OpenAI has introduced a research preview of Operator, an agent powered by Computer-Using Agent (CUA), which combines GPT-4o's vision capabilities with reinforcement learning to interact with graphical user interfaces. CUA processes raw pixel data and uses a virtual mouse and keyboard to navigate multi-step tasks across diverse digital environments without relying on OS-specific APIs.
- CUA achieves state-of-the-art results, scoring 38.1% on OSWorld, 58.1% on WebArena, and 87% on WebVoyager.
- The agent operates through an iterative loop of perception, reasoning via chain-of-thought, and action, adapting to unexpected changes.
- Safety measures include refusals for harmful tasks, blocklists for prohibited sites, real-time moderation, and user confirmations for sensitive actions.
The research preview is available to Pro Tier users in the U.S. at operator.chatgpt.com to gather feedback and refine safety measures as the technology evolves.