As what I recall when I learned about the gradient, I was taught that the gradient represents the vector with the direction of fastest climbing. I didn’t really understand it. It is known that $\frac{\partial f}{\partial x}$ represents the rate of change for $f$ at $x$, then shouldn’t the gradient just tell you the rate of change for each direction?

I’m using $\nabla f = a\mathbf{i} + b\mathbf{j}$ for instance, but it should also apply to higher dimensions.

Proof: $\nabla f$ is in the direction where $f$ increases the fastest.

Let the direction of the gradient be $\mathbf{u} = \frac{\nabla f}{\|\nabla f\|}$. Say $\nabla f$ is not the direction where $f$ increases the fastest. Then $\exists$ unit vector $\mathbf{v}$ s.t. the directional derivative along $\mathbf{v}$ is greater than the gradient’s magnitude

So$\|\mathbf{v}\| \|\nabla f\| \cos(\theta) > \|\nabla f\|$Since $\mathbf{v}$ is a unit vector, $\|\mathbf{v}\|=1$ and $\cos(\theta) \leq 1$. $\Rightarrow \Leftarrow$ Thus, $\nabla f$ is in the direction where $f$ increases the fastest.