ChatGPT Gets 'Agentic' Upgrade: OpenAI Lets AI Work Directly Across Your Desktop Apps

San Francisco, CA – OpenAI has introduced significant updates to its ChatGPT desktop applications for macOS and Windows, focusing on enhanced system integration. Announced as part of OpenAI’s "12 DAYS OF OPENai," specifically "Day 12 – "A New Way to work with ChatGPT”", the update allows ChatGPT to interact directly with other applications running on the user's computer.

The announcement opened a discussion about the existing Mac and Windows desktop apps and the concept of agentic models. A key feature highlighted is the ability for ChatGPT to "Work With Apps on the computer," allowing it to pull context directly from the application currently being viewed by the user. The announcement specified that ChatGPT focuses on the app the user chooses to show it, even if other previously viewed apps are open.

A demonstration showcased the Warp console, linked to GitHub, being used with ChatGPT. Because the Warp console was visible to ChatGPT, the user did not need to explicitly describe the desired output format. The announcement stated that the model can access both the internal content of the app and the visual elements the user sees.

OpenAI noted that utilizing accessibility APIs for this type of integration can be challenging without coding knowledge. The new features are available for use with the o1-Pro model, which was identified as one of several "chain-of-thought" (CoT) models.

A key goal outlined in the announcement is to eliminate the need for users to copy and paste information between applications, provided the relevant window is within ChatGPT's view. Supported applications at launch or soon after include VSCode, Jetbrains products (Pycharm, Ruby Mine), Text Mate, and Deviant Edit. Support for Apple Notes, Notion, and a platform referred to as "Quint" was also announced, with the implication of near-term availability.

The announcement mentioned that highlighting text within an application helps the model understand what to focus on. The desktop ChatGPT applications also now include a search functionality. A demonstration illustrated how ChatGPT can adapt its writing style to match a sample provided from the document it has in view.

The updated desktop apps also support Advanced Voice Mode within the app view. For Mac users, a hotkey combination of Option + Shift + 1 was introduced to quickly bring up ChatGPT. Users can also receive live feedback from ChatGPT on their text input.

The announcement confirmed that all these features for the desktop apps are intended to be available now, requiring users to update their applications. The presentation concluded with a teaser for something "super exciting" planned for the final day of the "12 DAYS OF OPENai" initiative.