Basic Idea Of Gaussian Elimination (If You Already Know Linear Algebra)


(This is not meant to show you how to solve the problem, just what the solution does in the most important special case).

Gaussian elimination is about factorization. Factorization is just expressing something as a composition of simpler things. In this case, expressing a linear function as a product of simpler linear functions.

Say we have an invertible linear function and we want to find the (unknown) vector that maps to a known vector . The assumption of invertibility is just to make the basic idea easier to explain.

That means we have to solve .

Since is invertible, we just need to apply to both sides to get . Gaussian elimination does that by factoring.

The set of all invertible linear functions forms a group under the operation of composition (AKA matrix multiplication), . This group is generated by the elementary matrices. Let’s call the set of those elementary matrices . Then . Each elementary matrix is also invertible, and the inverse of the product is the product of the inverses, in reverse order, . Applying this inverse sequence to both sides gives a fast way of finding the inverse function.

Each step of Gaussian elimination is equivalent to left-multiplying by an elementary matrix until you’ve inverted your original function.

Gaussian elimination is an algorithmic way of finding that factorization into elementary matrices that works over any field, not just the real numbers.

The funny thing is that you can do linear algebra without ever being able to solve such a simple problem. Not that that’s much fun.

Related Posts

Middle School by Bo Burnham

How to Disable Disqus Ads on your Blog

Derivation of Reservoir Sampling

Fun with Python Iterators: Linked Lists Made Easy

Notes for November 11, 2018

Underrated Vim Option: undofile and undodir

Hot Take on Solo Travel: Starve

Alan Perlis

Book Notes: The Map of My Life by Goro Shimura

Prague