Notes for March 6, 2018


Today I grasped the difference between the differential and the derivative (only took 6 years -_-), lost my fear of as a single symbol (until learning the exterior derivative changes that into 2 symbols again), and finally got a hint as to why transposes pop up in weird places in “matrix calculus”.

Someone described a blog post of mine as “polemic and a quick example”. That’s an annoyingly good description. It’ll take some effort to level up to “polemic and the perfect example”.

Read half of a thesis about using algebra in reinforcement learning. It’s a bloody mess.

In a way, the action space acts on the state space via the transition dynamics, but it’s not a true semigroup action. They have the wrong type signature because of the probabilistic dynamics. Actions in RL give a distribution over (successor) states rather than a state. Deterministic transitions give a manifold with a reward structure, like a finite state machine that’s not finite. Not a great name. But I wonder if sufficiently nice actions and states (e.g. lie groups) make the problem tractable.

Terry Tao’s advice is real right now, as I made this list of topics I’d need to learn to even begin to know how to formalize the fuzzy ideas in my head:

  • Differential geometry
  • Dynamical systems
  • Random dynamical systems
  • Group theory
  • Lie theory
  • Automata theory
  • Category theory

That list would take at least a year of solid work on nothing else. And that’s for one random idea. I’m not so worried about someone stealing an idea after thinking that over. I need all the help I can get.

Related Posts

Handy command line benchmarking tool

Stan Rogers

Ultimate Hot Couch Guy

Quote on Java Generics

The Programmer Tendency

Figure out undocumented JSON with gron

Mental Model of Dental Hygiene

Book Review: Swastika Night

Is there a name for this construction?

Fun with negation and idioms