> ## Content Index
> Fetch the complete content index at: https://enteruaf.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# The helpful machine had teachers
- URL: https://enteruaf.com/the-helpful-machine-had-teachers-e5d6d5d1/
- Published: 2022-04-05T16:00:00.000Z
- Updated: 2026-09-20T10:20:25.000Z
- Description: InstructGPT learns from demonstrations and ranked answers. Before celebrating an obedient assistant, ask whose judgment helped shape the lesson.
- Author: UAF Editorial
- Tags: AI, #authorego-e572e54de6682d86c9f4c5030c53b05e

A model can respond to instructions without learning a new set of values from every prompt. GPT-3’s researchers tested tasks specified through text and examples, without task-specific gradient updates or fine-tuning during those evaluations. They reported strong results on several language tasks and difficulties on others. For the reader, the useful distinction is between directing a trained model with a prompt and changing the model through training. A convincing answer does not erase that distinction. [\[1\]](#source-1)

InstructGPT changes the training process. Its researchers began with two sources of prompts: ones written by labelers and ones submitted through the OpenAI API. Labelers demonstrated desired responses. The researchers used those demonstrations for supervised fine-tuning, then collected rankings of outputs for further training with human feedback. People supplied examples and judgments about answers; the model did not discover a universal definition of helpfulness in a quiet moment of reflection. I would find that origin story more flattering. The documented one is more useful. [\[2\]](#source-2)

The researchers report that, on their evaluation prompt distribution, people preferred outputs from a smaller InstructGPT model to those from the much larger GPT-3\. They also report improved truthfulness and reduced toxic output, while acknowledging simple mistakes. Those results support further work on human-feedback training. They do not settle whose preferences ought to govern every assistant. My editorial question is practical: when an answer feels helpful, which judgments helped make it feel that way? [\[2\]](#source-2)

Chain-of-thought prompting explores another intervention: eliciting a sequence of short reasoning steps through a prompt. Its authors report that this can improve sufficiently large models on reasoning tasks that otherwise benefit little from increasing model size. That is a result about a prompting method. It should not be treated as an answer to the separate question of what the model ought to do. A longer explanation can be useful; it does not get to appoint itself the judge. [\[3\]](#source-3)

## Sources

1. [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165v1?ref=enteruaf.com)
2. [Training language models to follow instructions with human feedback](https://arxiv.org/abs/2203.02155v1?ref=enteruaf.com)
3. [Chain of Thought Prompting Elicits Reasoning in Large Language Models](https://arxiv.org/abs/2201.11903v1?ref=enteruaf.com)