Calculus And Programming

I'm posing a question here, and would appreciate any responses.

I am currently taking the Calculus sequence(differentiation,integration) at my university. I'm having a difficult time keeping it in my head as I can't seem to "connect it" to programming. I see how it could be useful for physics, or for certain problem domains - but I can't see how it's useful to programming.

Can anyone give examples of how software development is improved by mastering Calculus?

Check out the discussion on MathForProgrammers.

So the general consensus is that it's not particularly useful to programmers - except in an abstract sense (i.e. something to broaden one's mind). . . How anti-climactic.

We made extensive use of TheCalculus? and its fundamental Finite Difference techniques in solving the partial differential equations that were used in the Numerical Weather Prediction models of the 1970s. I recall it being:

quite complicated and severely limited to the capabilities of the existing hardware IBM System 360/75.
not profitable until doing consulting several years later when I won a contract from a major brewer based on the fact that I knew I could calculate an approximate relationship between a one degree increase in summer temperatures and beer sales.
- extended into a program for a beer delivery system based in AplLanguage, that was to have been temporary (being an Interpreter) but was so cheap to develop and maintain, it ran for half a decade because the re-redevelopment costs for a compiled environment could not be justified.

But I digress. -- HansWobbe

Would you say Calculus was used more for solving the problem domain (i.e. Weather), or as a tool to help in general development?

Many times I've done an analysis of our code, only to find that my programmers can't understand it. The common feature has always been that I was using techniques from calculus.

The "calculus" that you do in school, even in college or university, is as much related to what I was doing as arithmetic is to "real math". Once you have mastered how calculus works, the ideas, techniques, approaches and methods, you find that they apply in other areas.

Read RichardFeynman's analysis of the routing system in the ConnectionMachine to see how this sort of thing can work.

Your question about solving problems in the domain versus general development, I would say that calculus is never used specifically for program development except as required by the problem domain. I would add that calculus has been useful in every problem domain I've encountered, therefore in every commercial program I've written.

I read the page, but saw no mention of Calculus (or differentiation/integration). I'm not disagreeing with you here, I'm just not understanding you (since I don't have a mastery of Calculus) - could you explain what kind of analysis you did of the code (where Calculus is needed)? Was this related to profiling or analyzing the speed of algorithms?

It's referenced from RichardFeynmanAndTheConnectionMachine. Read the article pointed to from there.

By the end of that summer of 1983, Richard had completed his analysis of the behavior of the router, and much to our surprise and amusement, he presented his answer in the form of a set of partial differential equations. To a physicist this may seem natural, but to a computer designer, treating a set of boolean circuits as a continuous, differentiable system is a bit strange. Feynman's router equations were in terms of variables representing continuous quantities such as "the average number of 1 bits in a message address." I was much more accustomed to seeing analysis in terms of inductive proof and case analysis than taking the derivative of "the number of 1's" with respect to time. Our discrete analysis said we needed seven buffers per chip; Feynman's equations suggested that we only needed five. We decided to play it safe and ignore Feynman.
We decided to go ahead and make the chips with the smaller number of buffers. ... Fortunately, he was right.

The analysis we did was concerning the processing space and time needed for the algorithms we were using in the context we had. We were doing image processing to enhance image compression. Again, it seems like it's a domain thing, but it was intimately tied to the processing being done.

I don't know what you want. It sounds to me that you'd like to dismiss calculus as not useful in computing, and you're divorcing the problem domain from the computing. You can't do that. If you do accountancy you may not use calculus, but games programmers I know use it, every contract programmer I have worked with has either known it or explicitly said they wish they had.

It's like algebra. Analysing program behaviour is a lot easier if you're comfortable with writing and manipulating formulae, but it's not really a part of computing.

Actually I'm looking for a way to directly connect Calculus and programming mentally (right now they are separate spheres). I can see that Calculus is useful for many things, but why is it more useful to programming, than say music theory, or cellular automata? This may all seem obvious to you, but I'm looking at it from the other side and I don't "simply know" that Calculus is useful. I've used trigonometry and linear algebra in 3d programming, but I've yet to find a way to use Calculus - I'm just wondering what the canonical programmer's use for Calculus is.

Calculus is used in 3d programming too - you compute the normal to a surface by calculating the tangents (using partial derivatives) and then taking a unit vector that's orthogonal to them.

But in my experience, you're better off with music theory. Music theory, particularly species counterpoint, has a lot in common with programming. You're constrained by a set of rules, but within the rules, you want to construct the most aesthetically pleasing composition possible. And sometimes you find that your aesthetic choices have boxed you into a corner where you need to rewrite the whole thing, something you're bound to experience if you construct any non-trivial software.

If you want to learn math that's generally applicable to programming, I'd start with set theory and mathematical logic. In particular, learn how to write proofs. Those have direct connections to formal CS theory - you'll be hard-pressed to find an academic paper about advanced algorithm or compiler research that doesn't use formal methods. And if you want to do scientific programming, practically all of it requires calculus for problem-domain reasons (statistics, too, is a big one). But daily programming work really doesn't take much math beyond what's required by the problem domain. -- JonathanTang

Excellent response! I don't have much choice as to which math classes to take or which order. . . although I am suddenly interested in music theory. Thanks again!

The basis of DifferentialEquations? that can describe inputs and outputs of DynamicSystems?, is IntegrationAndDifferentiation?. Still, visualizing the results depends on calculable functions. These can simply be run as functions in a regular ProgrammingLanguage, view output as ASCII in notepad, CopyAndPaste to Excel for gui-style views. For instance, verifying the LyapunovFunction? V(x) > 0 to see if a system:

  dx1/dt=x2
  dx2/dt=-x2-x2^3-x1

is Stable, choosing V = x1^2+x2^2

 <?php 
 testV();

 function testV()
  {
    print "x1x2V\n";
    print "-----------------\n";
    for($x1=1;$x1<=3;$x1++)
    {  
      for($x2=1;$x2<=3;$x2++)
      { 
        print "$x1$x2".V($x1,$x2)."\n";
      }
    }
  }
  #changes for different problems
  function V($x1,$x2)
  {
    return pow($x1,2)+pow($x2,2);
  }
  ?>

Output (php v.php > v.txt):

x1x2V ----------------- 112 125 1310 215 228 2313 3110 3213 3318

We can see V is always greater than zero for the selected range. For Excel we RightClick?->Paste into a WorkSheet?, Insert->Chart(3-D Column) to see the result in ThreeDeeGraphics. The test dV/dt <=0 is more challenging, could be done manually and programmed as dVdt(), or both functions coded in a language such as Mathematica.