“TextWorldExpress: Simulating Text Games at One Million Steps Per Second”, 2022-08-01 (; backlinks):
Text-based games offer a challenging test bed to evaluate virtual agents at language understanding, multi-step problem-solving, and common-sense reasoning. However, speed is a major limitation of current text-based games, capping at 300 steps per second, mainly due to the use of legacy tooling [such as Zork].
In this work we present TextWorldExpress, a high-performance implementation of 3 common text game benchmarks that increases simulation throughput by ~3 orders of magnitude, reaching over one million steps per second on common desktop hardware [including caching all possible paths through simple text games, storing in large 1GB JSON text files].
This largely reduces experiment runtime, enabling billion-step-scale experiments in about 1 day.