A Prince, a Pauper, Power, Panama


Instrumental convergence is an issue I think about a lot because it's so natural. Being rich and powerful almost never works against you.

Take The Prince and the Pauper. It's asymmetric.

The pauper's side of the story can only happen because the prince allowed it, but the prince can just wear rags without anyone's permission. It's easy to go from prince to pauper, but not the other way around.

More abstractly, one with resources and power can always just leave it all behind and go live in the woods. Going the opposite way is The Story of Civilization.

What is power? I think of it as a large action space, with large orbits in the state space induced by that. A fancy way of saying "it's having a lot of options". Hence, the idea that power is the ability to do what you want. An omnipotent one would be maximally powerful, because it can do anything. Largest possible action (and state) space.

So one would want to gain power, pretty much independent of its goals. Other universal instrumental goals are lots of mass-energy under your control and minimizing energy usage for a given level of function. Influence --in the literal sense of matter flowing towards you-- and the Free Energy Principle.

Related Posts

My biggest takeaway from Redwood Research REMIX

finite, actual infinity, potential infinity

Actions and Flows

PSA: reward is part of the habit loop too

a kernel of lie theory

The hyperfinite timeline

Slick hyperfinite Ramsey theory proof

Gaoxing Guy

What it's like to dissect a cadaver

Braindead Way to Derive Taylor Series of Exponential Function