ADMINHow the assistant works in a Windows window#
Vision follows a loop: observe the screen → perform a permitted action → check the result. Agent supplies the intended session's image, and the assistant uses available mouse and keyboard actions.
- Choose a computer
- Observe
- Act
- Check
Prepare the workspace
Check Agent Online, the correct user's open session and the required application. The AI model and connection path must support images. Agent does not unlock Windows or bypass protected Windows prompts.
Describe a specific task
For example: “Open this form on the selected PC, enter test data and show the result before saving.” State what may change. With ordinary restricted access, review and approve the task; permanent Full Control follows separate access rules.
Let the assistant check state
A click or text entry needs a subsequent check. If the window changes or the result is uncertain, obtain a current observation first. Do not blindly repeat an uncertain action.
Verify the outcome
Successful text delivery does not prove that a form is correct. Clicking Save does not prove that a record exists. Check field values, application messages and the resulting record in the intended system.
Why every image is not a new full screenshot#
The first observation is a full screen. Later observations can contain changed areas only, or an unchanged-state message. Core retains bounded current visual state for checks and recovery. This optimisation is automatic; you do not assemble fragments yourself.
Codex receives new changes within its visual context. The ordinary Native path includes the required baseline and patch sequence in model requests. These paths differ; a new full screenshot is not needed after every click. Lost context requires full-image recovery.
Text entry and keys#
Agent supports typing, clipboard paste and key combinations. Predictable forms can use bounded action sequences, followed by outcome verification. Uncertain delivery is not a reason to type the same text twice automatically. The clipboard does not become the assistant's permanent memory.