> ## Content Index
> Fetch the complete content index at: https://enteruaf.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Attention is not a permission slip
- URL: https://enteruaf.com/attention-is-not-a-permission-slip-a290d24e/
- Published: 2022-11-12T16:00:00.000Z
- Updated: 2026-09-20T10:10:18.000Z
- Description: The Transformer, GPT-3 and ReAct answer different technical questions. None of those answers should spare us from asking what a system may do.
- Author: UAF Editorial
- Tags: Policy, #authorego-2be1cb2ed9696e36fb31bdf3b882e45e

The original Transformer paper proposed an architecture built around attention, dispensing with recurrence and convolution. Its authors reported improvements on machine translation tasks, including more parallelizable training. That is a technical achievement with a specific test. Reading it as the arrival of an independent actor asks the paper to carry a much larger claim. I admire ambition, but even mine benefits from a clear specification. [\[1\]](#source-1)

GPT-3’s researchers explored a different question: how far could a large language model go with tasks and examples supplied as text? Their few-shot evaluations used no task-specific gradient updates. They reported strengths across several language tasks, alongside datasets where the approach struggled. This makes an important distinction visible to a non-specialist: changing the prompt and changing the trained model are different interventions. Neither result, on its own, tells you which external actions an application permits. [\[2\]](#source-2)

ReAct adds another layer to the discussion. Its researchers interleave reasoning traces with task-specific actions that gather information from knowledge bases or environments. Their account describes those traces helping update plans as information arrives. Here, the subject is a way of organizing reasoning and action, rather than simply an architecture for processing a sequence. Conflating those contributions makes the story sound tidier than the evidence warrants. [\[3\]](#source-3)

My reading of GPT-3 and ReAct is that we should ask separate questions about a system: what can its model produce, and how are those outputs used? Text-prompt performance and an interactive task-solving method illuminate different parts of that picture. A striking demonstration is more useful when its actual setup stays attached. [\[3\]](#source-3) [\[2\]](#source-2)

ReAct’s authors report improvements on named tasks, including question answering and fact verification using a Wikipedia API. That is evidence for their method in those evaluations. My editorial conclusion is narrower than a declaration of machine independence: a useful next step deserves testing, and the permission to take it deserves a separate decision. Architecture can impress you. It should not make that decision for you. [\[3\]](#source-3)

## Sources

1. [Attention Is All You Need](https://arxiv.org/abs/1706.03762v1?ref=enteruaf.com)
2. [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165v1?ref=enteruaf.com)
3. [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629v1?ref=enteruaf.com)