Basic Idea Of Gaussian Elimination (If You Already Know Linear Algebra)


(This is not meant to show you how to solve the problem, just what the solution does in the most important special case).

Gaussian elimination is about factorization. Factorization is just expressing something as a composition of simpler things. In this case, expressing a linear function as a product of simpler linear functions.

Say we have an invertible linear function and we want to find the (unknown) vector that maps to a known vector . The assumption of invertibility is just to make the basic idea easier to explain.

That means we have to solve .

Since is invertible, we just need to apply to both sides to get . Gaussian elimination does that by factoring.

The set of all invertible linear functions forms a group under the operation of composition (AKA matrix multiplication), . This group is generated by the elementary matrices. Let’s call the set of those elementary matrices . Then . Each elementary matrix is also invertible, and the inverse of the product is the product of the inverses, in reverse order, . Applying this inverse sequence to both sides gives a fast way of finding the inverse function.

Each step of Gaussian elimination is equivalent to left-multiplying by an elementary matrix until you’ve inverted your original function.

Gaussian elimination is an algorithmic way of finding that factorization into elementary matrices that works over any field, not just the real numbers.

The funny thing is that you can do linear algebra without ever being able to solve such a simple problem. Not that that’s much fun.

Related Posts

Handy command line benchmarking tool

Stan Rogers

Ultimate Hot Couch Guy

Quote on Java Generics

The Programmer Tendency

Figure out undocumented JSON with gron

Mental Model of Dental Hygiene

Book Review: Swastika Night

Is there a name for this construction?

Fun with negation and idioms