OpenAI Launches Chronicle Feature in Codex, Making AI Directly Read Screen Context the Default Interaction Method
OpenAI has launched the Chronicle feature in Codex, allowing AI to continuously take screenshots and perform OCR analysis in the background to automatically understand the current screen content of users. This enables direct references in conversations such as "this error" or "that file" without the need for manual copying and pasting. This feature, an extension of the previous Memories, has been rolled out in a limited capacity for ChatGPT Pro users on macOS.
Chronicle periodically records screen activity through a local background process and generates structured memory files to enhance cross-session understanding. However, this data needs to be uploaded to OpenAI's servers for processing, and the generated plaintext Markdown files are stored locally, posing potential privacy risks and prompt injection attack vulnerabilities; the official source has clearly indicated that this feature may amplify the impact of malicious web commands.
A similar "AI screen reading" approach has been explored by Microsoft Recall and some IDE tools, but implementing it as a continuously running agent that automatically builds long-term memory and integrates into programming tool ecosystems is currently a bold advancement among mainstream vendors.
Source: Public Information
ABAB AI Insight
This is not just a simple feature update, but a shift in the human-computer interaction paradigm: from "user providing context" to "AI actively acquiring context." The core bottleneck of past LLMs was the context window and information input costs; Chronicle attempts to bypass this limitation by using system-level data collection, essentially replacing the prompt itself with data flows from the operating system.
Behind this is a larger competitive direction: whoever controls the "real user behavior flow" will control the next generation of AI entry points. The core asset in the search era was query terms, in the mobile internet era it was clicks and dwell time, while this generation of AI competes over continuous screen semantic flows. Chronicle directly transforms "what you are doing" into model input, effectively turning the operating system into part of the training and inference interface.
Privacy and security issues are not just ancillary costs, but structural contradictions. This model inherently expands the attack surface: prompt injection evolves from "text pollution" to "environmental pollution," where any webpage, document, or even terminal output could become a source of commands. This means that traditional "model dialogue security" is shifting towards "system-level security," with boundaries extending from APIs to the entire user device.
From an industrial perspective, this direction explains why Microsoft, OpenAI, and IDE vendors are simultaneously advancing towards "long-term memory + system-level perception." The difference lies not in model capabilities, but in who can embed deeper into the operating system and development environment. Once AI reliably masters continuous context, the interface layer value of traditional applications will be diminished, and the main entry point for software may shift from "clicking applications" to "describing goals."