Derivatives and integrals: they sound like sixth form and suffering. To read the lessons you don’t have to work any out, just understand what they ask. And you already know what they ask: “how fast am I going?” and “how far have I gone?”.
dE/dt d/dt and the prime: the rate of change
- Say it
- dE/dt is read “dee E by dee t” or “the derivative of E with respect to time”. σ′ is read “sigma prime”.
- It means
- How much something changes for each sliver of time (or of whatever is underneath), at one particular instant. It's ΔE/Δt with a Δt as small as you like.
A car’s speedometer doesn’t show the average speed of the whole trip, but the speed right now. That’s a derivative: Δd/Δt with a Δt so small it’s almost zero. The small d in dE/dt is a Δ “in miniature”. The prime, as in σ′, is another way of writing the same thing.
Real example · in the lesson Joules, watts or calories? Energy and power without fear
P = dE/dt (instantaneous); P̄ = ΔE/Δt (mean).
∂ ∂: the partial derivative
- Say it
- “partial” or “del”. ∂L/∂w is read “the partial derivative of L with respect to w”.
- It means
- The same as d/dt, but for when something depends on many things at once: you change just one and keep the others still.
In a neural network, the error depends on thousands of weights. ∂L/∂w asks: if I nudge just this weight a little, how much does the error change? That’s how you know which way to turn each dial.
Real example · in the lesson Logistic regression or neural network? Same neuron
σ′(z) = σ(z)(1 − σ(z)): the slope of the sigmoid, the piece ∂L/∂W uses during training.
∇ ∇: the gradient
- Say it
- “nabla” or “the gradient of”. ∇L is read “the gradient of L”.
- It means
- An upside-down triangle that gathers all the partial derivatives into one arrow. That arrow points to where something climbs fastest.
Picture yourself on a mountain in the fog. ∇ tells you where the steepest climb is. To go down, you head exactly the other way: −∇. That’s how a neural network learns, stepping down the mountain of error.
Real example · in the lesson Logistic regression or neural network? Same neuron
−∇L = −Xᵀ(p − y)/n: which way the penalty goes down.
∫ ∫: the integral
- Say it
- “the integral of”. ∫ P(t) dt is read “the integral of P of t, dee t”.
- It means
- A stretched-out S, for “sum”. It adds up a huge number of very thin slivers: it's the area under a curve.
If you know how fast you’re going at every instant, how far have you gone? Cut the trip into slivers of time, multiply speed by time in each one and add it all up. The thinner the slivers, the more exact. The integral is that sum with infinitely thin slivers. The dt at the end says what you’re slicing.
Real example · in the lesson Joules, watts or calories? Energy and power without fear
E = ∫ P(t) dt: energy is power piled up over time.
Try it
A car speeds up, and after t seconds it has covered t² metres. In “rate of change”, make the jump Δt smaller and watch the average speed close in on 4 m/s, what the speedometer shows at t = 2 s. In “add up”, put more rectangles under the speed and watch the sum close in on 9 m.
Challenge
Sources
- d/dt, the prime, ∂, ∇ and ∫: ISO 80000-2:2019, Quantities and units — Part 2: Mathematics (iso.org).
- The derivative as the limit of the average speed and the integral as the limit of sums of rectangles: G. Strang, Calculus, 3rd ed., Wellesley-Cambridge Press, 2017, chapters 2 and 5 (MIT OpenCourseWare).
How this cheat sheet was made: written with Claude’s help from the sources above and translated from the Spanish original. The examples are copied from the site’s lessons. The widget’s numbers (4 m/s and 9 m) come from d = t² and a test checks them. The drawings are original.