Bitter lesson by @RichardSSutton is one of the most insightful essays on AI development of the last decades.
Recently, given our progress in robotics, I’ve been trying to predict what the next bitter lesson will be in robotics and how can we prevent it today.
Let me explain 🧵
Jan 9, 2023 · 6:00 PM UTC
Let's re-visit the original bitter lesson first:
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law (...)"
The reasoning goes like this:
1. We don't know how to build general-purpose AI yet but we know it will need a lot of compute.
2. We can recognize a trend outside of the field: computation is getting cheaper.
3. Let's work on AI methods that leverage that trend.
Let's look at robotics now. It's well understood that the biggest bottleneck in robotics is the lack of data (there is no internet-scale dataset of robot actions).
In fact, let's assume it isn't possible to generate enough data needed for general-purpose robots using robots alone
So we want to find something else that can provide that data to robotics.
In other words, we want robot learning to ride the wave of progress in other areas that will feed in more data to robots.
So our reasoning goes like this:
1. We don't know how to build general-purpose robots yet but we know they will require a lot of data.
2. Trend outside of robotics: ?
3. Work on robot learning methods that leverage that trend.
I call it Bitter Lesson 2.0 (robotics version)
To prevent learning this lesson in hindsight, let's try to predict what that trend in #2. could be and work on methods that leverage it.
We're looking for a trend that is outside of robotics (like Moore's law was happening outside of AI) that will bring more data into robotics.
Given the recent progress in AI, I propose this trend to be foundation models:
• there is huge interest and value in foundation models outside of robotics
• they scale with data and compute (1st bitter lesson)
• they are getting better at understanding the world
If we think of foundation models as a distilled internet-scale datasets, assuming that we can leverage them in robotics, they will provide a ton of data that robots so desperately need to understand the world around them.
To summarize, I believe that the next bitter lesson (in 70 years) will be:
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage foundation models are ultimately the most effective"
In the next longer 🧵 I'll present some examples of how we try to leverage this insight in our robotics work @GoogleAI.
And we're working on many more research directions along these lines - stay tuned!