Today I grasped the difference between the differential and the
derivative (only took 6 years -_-), lost my fear of \(dx\) as a single
symbol (until learning the exterior derivative changes that into 2 symbols
again), and finally got a hint as to why transposes pop up in weird
places in “matrix calculus”.
Someone described a blog post of mine as “polemic and a quick example”. That’s an annoyingly good description. It’ll take some effort to level up to “polemic and the perfect example”.
Read half of a thesis about using algebra in reinforcement learning. It’s a bloody mess.
In a way, the action space acts on the state space via the transition dynamics, but it’s not a true semigroup action. They have the wrong type signature because of the probabilistic dynamics. Actions in RL give a distribution over (successor) states rather than a state. Deterministic transitions give a manifold with a reward structure, like a finite state machine that’s not finite. Not a great name. But I wonder if sufficiently nice actions and states (e.g. lie groups) make the problem tractable.
Terry Tao’s advice is real right now, as I made this list of topics I’d need to learn to even begin to know how to formalize the fuzzy ideas in my head:
- Differential geometry
- Dynamical systems
- Random dynamical systems
- Group theory
- Lie theory
- Automata theory
- Category theory
That list would take at least a year of solid work on nothing else. And that’s for one random idea. I’m not so worried about someone stealing an idea after thinking that over. I need all the help I can get.