(This is not meant to show you how to solve the problem, just what the solution does in the most important special case).

Gaussian elimination is about *factorization*. Factorization is just
expressing something as a composition of simpler things. In this case,
expressing a linear function as a product of simpler linear functions.

Say we have an *invertible* linear function and we want
to find the (unknown) vector that maps to a *known* vector
. The assumption of invertibility is just to make the basic
idea easier to explain.

That means we have to solve .

Since is invertible, we just need to apply to both
sides to get . Gaussian elimination does that by
*factoring.*

The set of all invertible linear functions forms a group under the
operation of composition (AKA matrix multiplication), . This
group is generated by the *elementary matrices*. Let’s call the set of
those elementary matrices . Then . Each
elementary matrix is also invertible, and the inverse of the product is
the product of the inverses, *in reverse order*,
. Applying this inverse sequence to both
sides gives a fast way of finding the inverse function.

Each step of Gaussian elimination is equivalent to left-multiplying by an elementary matrix until you’ve inverted your original function.

Gaussian elimination is an algorithmic way of finding that factorization into elementary matrices that works over any field, not just the real numbers.

The funny thing is that you can do linear algebra without ever being able to solve such a simple problem. Not that that’s much fun.