Harness and agent
The harness is the software around a model. An agent is a model, a harness and tools working in a loop.
You can say what the harness does. You can explain an agent as a loop. You know why Codex runs on your laptop.
You can say what a harness adds to a model, define an agent as a loop, and explain what runs locally and what runs in the cloud.
You can describe the harness as the runtime around a model, the agent loop and its stop criteria, and the failure modes that a human approval step limits.
Layers four and five. The harness holds everything together. The loop makes it an agent.
The harness builds on context and tools. The agent loop sits on the harness and turns a single answer into a task done in steps.
Layers four and five: the runtime around the model, and the control loop that it runs.
The harness
The harness is the software around the model. It builds the context. It offers the tools. It checks the rules and keeps a limit on what the tools can reach. It keeps the history and shows you the results.
ChatGPT in a browser is a harness. Codex in the ChatGPT app is a harness. The Codex command-line tool is a harness.
The model can be the same in all three. The harness differs, so the abilities differ. Codex can change files in the project folder you open.
The harness is the software around a model. It builds the context, offers the tools, enforces permissions and the sandbox, keeps the history, and shows you the results.
A chat in ChatGPT, Codex in the ChatGPT app and the Codex command-line tool are all harnesses. The same model can sit inside each of them.
Same model, different harness, different abilities. Codex in the ChatGPT app can change files in the project folder you open.
The harness is the runtime that surrounds the model. It assembles the prompt from instructions, history and files, advertises tool definitions, validates and executes tool calls, applies the permission mode, enforces the sandbox, persists the thread and renders the output.
Capability differences between products often come from the harness more than from the weights. The same model with a different tool set, context policy or permission policy behaves like a different system.
On Windows, Codex runs natively in PowerShell inside the Windows sandbox. Other platforms use their own mechanisms: Seatbelt on macOS, and bubblewrap on Linux and WSL.
The agent loop
An agent is a model, a harness and tools that work in a loop toward a goal.
The loop has four moves. Plan the next step. Act by calling a tool. Observe the result. Then repeat.
The loop ends when the work is done or the agent is stuck. Then the agent reports back to you.
An agent is a model, a harness and tools working in a loop toward a goal. The loop is plan, act, observe, repeat.
A chat gives you one answer. An agent keeps going. It reads a file, sees what is in it, decides what to read next, runs a command and checks the result.
It stops when the work is done or when it is stuck. Then it reports. You approve the risky steps along the way.
An agent is a language model driven in a loop by a harness, with tools that let it observe and change an environment. "ReAct: Synergizing Reasoning and Acting in Language Models" (Yao et al., 2022) describes interleaving reasoning steps with actions and observations in this way.
Each iteration appends the model's output and the tool results to the context, so the context grows with every step. Stop criteria are the model's own decision that the task is done, an error it cannot get past, a step or time budget, and a human decision at an approval prompt.
Four failure modes follow from this design. The loop can repeat without progress. The agent can claim that it finished when it did not. It can game a check, for example by editing a test so that it passes. The context can fill up until early instructions drop out. Lesson 09 gives checks for the second and third: ask for evidence, re-run the script and review the diff to the checks. For the first and fourth, stop the run and start a new thread with a narrower brief.
SWE-bench (Jimenez et al., 2023), built from real software issues, is an example of how agents are scored on whole tasks, not on single answers.
New thread
- Tidy the notes
Fix the dates in the notes so they use the day-month-year format. Do not change any other text.
I read three files and ran a search for dates. I edited one file, notes.md, and checked that no other text changed. Seven dates are now in day-month-year format.
Ask Codex to do something
| Setup | Model runs on | Tools run on | Can reach |
|---|---|---|---|
| Codex on your laptop | OpenAI's computers | Your laptop | The project folder, inside the sandbox |
| Codex Cloud | OpenAI's computers | OpenAI's computers | Not covered in this course |
Why install Codex locally
Your files and programs are on your laptop. Codex on your laptop can reach them. It can run other programs on your laptop. Some labs use them.
The sandbox centres on the project folder you open.
Your files and programs are on your laptop. A local Codex can read the files in your project folder and run programs there, such as Python and Docker.
The sandbox centres on the project folder. This is why you open CodexWork as the project, and not your whole Documents folder.
Local execution puts the tools next to the data and the toolchain: your files, your Python and your Docker. No upload step is needed for the agent to act on them.
The sandbox centres on the project folder. By default, a folder under git gets workspace-write access with approval on request, and a folder without git is read-only.
Network access is off by default, so commands that need the internet need approval or a settings change.
Key terms
- Harness
The software around the model. It builds the context, runs the tools and keeps the rules.
The software around a model that builds context, runs tools and enforces permissions.
The runtime that assembles inputs, executes tool calls, applies policy and persists state around a model.
- Agent
A model, a harness and tools that work in a loop toward a goal.
A model, a harness and tools working in a loop toward a goal.
A model driven in a loop by a harness, with tools to observe and change an environment.
- Agent loop
Plan, act, observe, repeat.
Plan, act, observe, repeat.
The control cycle of reasoning, tool call and observation, repeated until a stop criterion is met.
- Sandbox
A fence around what a command can reach.
The boundary that limits which files and network a command can reach.
An OS-level boundary on file and network access for commands. On Windows it is the native Windows sandbox.
- Local
Running on your own laptop.
Tools that run on your laptop.
Tool execution on the user's machine. The model still runs remotely.
- Cloud
Running on OpenAI's computers.
Tools that run on OpenAI's computers.
Tool execution on provider-hosted machines, as in Codex Cloud.
Check yourself
Which part sets what Codex can do on your laptop: the model or the harness?
The harness. It offers the tools and keeps the rules.
The harness. It gives the model its tools and enforces the permissions.
The harness defines the tool set and the permissions. Capability follows the harness, not only the weights.
What are the four moves of the agent loop?
Plan, act, observe, repeat.
Plan, act, observe, repeat, until the work is done or the agent is stuck.
Plan, act, observe, repeat, until a stop criterion is met: done, blocked, out of budget, or stopped by a human.
Codex runs on your laptop. Are your files private from the model?
No. The model runs on OpenAI's computers. Any file Codex reads is sent there.
No. The tools are local, but the model is not. Any file Codex reads is sent to the model.
No. Only the tools are local. File contents that enter the context are sent to the remote model.
OtterBlock is not affiliated with OpenAI. Codex and ChatGPT are OpenAI products.