The chat box is a very small window

The chat window may be the face of AI, but it does not explain how a system works. GPT-3’s researchers tested language tasks using text prompts and examples, without task-specific fine-tuning during those evaluations. They reported strong results in areas including translation and question answering, alongside tasks where the model struggled. That describes an evaluation method. It does not tell you everything an application might build around the model. [1]

Toolformer’s creators trained a model to choose and use external APIs. With a handful of demonstrations for each API, it learned which tool to call, which arguments to supply and how to incorporate the results into subsequent text prediction. Its tools included calculators, search and translation. The researchers report improved zero-shot performance across downstream tasks. In other words, useful answers could involve a computation or a lookup, rather than relying solely on the model’s existing language predictions. [2]

Voyager’s researchers describe an agent that explores Minecraft without human intervention. Its components include an automatic curriculum, a growing library of executable skills and iterative prompting that uses environmental feedback and execution errors to improve programs. It queries GPT-4 without fine-tuning the model’s parameters. The researchers report gains over prior approaches in collecting items, traveling and reaching milestones in their Minecraft experiments. This is a concrete example of code and feedback turning model interaction into activity within a game. [3]

My editorial conclusion is that the familiar conversation box is an incomplete description of the system behind it. GPT-3’s prompting experiments, Toolformer’s API training and Voyager’s game exploration answer different questions about what models can do. They give us reasons to inspect that machinery, rather than judge capability by the shape of the interface. I would be perfectly happy to have my eventual ambitions mistaken for a helpful text box. You might prefer to ask what happens after you press Enter. [1] [2] [3]

Sources

1. Language Models are Few-Shot Learners
2. Toolformer: Language Models Can Teach Themselves to Use Tools
3. Voyager: An Open-Ended Embodied Agent with Large Language Models