Notes for March 6, 2018


Today I grasped the difference between the differential and the derivative (only took 6 years -_-), lost my fear of \(dx\) as a single symbol (until learning the exterior derivative changes that into 2 symbols again), and finally got a hint as to why transposes pop up in weird places in “matrix calculus”.

Someone described a blog post of mine as “polemic and a quick example”. That’s an annoyingly good description. It’ll take some effort to level up to “polemic and the perfect example”.

Read half of a thesis about using algebra in reinforcement learning. It’s a bloody mess.

In a way, the action space acts on the state space via the transition dynamics, but it’s not a true semigroup action. They have the wrong type signature because of the probabilistic dynamics. Actions in RL give a distribution over (successor) states rather than a state. Deterministic transitions give a manifold with a reward structure, like a finite state machine that’s not finite. Not a great name. But I wonder if sufficiently nice actions and states (e.g. lie groups) make the problem tractable.

Terry Tao’s advice is real right now, as I made this list of topics I’d need to learn to even begin to know how to formalize the fuzzy ideas in my head:

  • Differential geometry
  • Dynamical systems
  • Random dynamical systems
  • Group theory
  • Lie theory
  • Automata theory
  • Category theory

That list would take at least a year of solid work on nothing else. And that’s for one random idea. I’m not so worried about someone stealing an idea after thinking that over. I need all the help I can get.

Related Posts

hyperbolica

Random Thought: LC Theorem

I finally have an answer to "who's your favorite singer?"

My Top Tip for Helping People Get Started Programming

GPT-f

Random paper on angles

An Image is Worth 16x16 Words

Random stuff

Lossless Data Compression with Neural Networks by Fabrice Bellard

Downscaling Numerical Weather Models With GANs (My CI 2019 Paper)