Rethinking Moravec's Paradox

On March 13, I attended a robotics institute seminar, featuring Max Simchowitz, a CMU professor in the machine learning department. His talk was about the algorithmic Moravec’s Paradox and a mathematical description of some techniques that attempt to solve it.

Hans Moravec is a professor at CMU in the 80’s who pioneered autonomous cars. Then, he noticed a surprising phenomenon: computers were really good at doing the tasks that require immense human intelligence like playing checkers or passing intelligence tests, but cannot even rival a one-year-old in perception and mobility. This is Moravec’s Paradox, and it has been true for the following decades as we have systems that can play Chess and Go at the super human level, and yet, in 2015, the most advanced humanoid robots were still failing to turn a valve.

However, recent developments in robotics seem to challenge this observation. Dancing humanoid robots are everywhere on the internet and self-driving cars can be seen at every other intersection in San Francisco. Max notes how robotic arms folding shirts have now become extremely banal in academic circles. Rather than being the subject of a PhD thesis, master’s students are now creating shirt folding robots as a way to start their research. The technology behind this is called behavior cloning, where a human demonstrator would pilot a pair of robotic arms to collect many examples of the robot doing a task (like shirt folding) and then the robot would learn from those examples to be able to do it autonomously. Though this technique has been around since the 80’s (started at CMU), so why is it only now that robots are becoming much more capable?

According to Max, there are two reasons for this. Around 2023: We simply started to collect more robot data. There were algorithmic breakthroughs that enabled us to learn more from the data we started collecting.

Therefore, in the pre-2023 world, when robotics was hard, the observation of Hans Moravec could perhaps be explained by two problems:

• Getting a lot of data for robotics is hard. (Pragmatic Moravec)

• Robot learning can be fundamentally harder without a few key algorithmic choices. (Algorithmic Moravec)

  The focus of Max’s talk is to give a mathematical basis for the “Algorithmic Moravec”. He aims to: 1. Describe why learning from demonstrations can be fundamentally harder in continuous control settings as opposed to the discrete world of symbols. 2. How the key breakthroughs mitigate this fundamental challenge.

(I got carried away with other projects to finish this post. But in short, he showed that 1 exists because the continuous regime allows for infinitismal errors compound into unfixable errors, whereas this did not happen with discrete actions. For 2, he spoke about how action chunking and diffusion models mitigate the issue posed in 1.

Action chunking reparameterizes the continuous regime to be controllable by a policy by bounding the error the policy makes. The benefit of diffusion/flow models were already clear: it could learn multiple modes. So instead of showing why diffusion works for robotics, he instead challenged to assumption that it is only multimodality that made robotics work. He experimented with unimodelflow models in MIP (minimum iterative policy) and showed that the iterative process also made a big difference.)