Sign up for our daily and weekly newsletters for the latest updates and exclusive content on the industry’s best AI coverage. Learn more
OpenAI has expanded the number of applications that can be used with the desktop app, including allowing advanced voice modes to be used with other apps, bringing ChatGPT closer to use on computers.
The desktop app introduced integrations with the initial four applications in November. OpenAI has announced several new integrated development environments (IDEs), terminals, and text apps that will be supported during Day 11 of its “12 Days of OpenAI” event.
ChatGPT now supports BBEdit, MatLab, Nova, Script Editor, TextMate as an IDE, VS Code for VSCode Insiders, VSCodium, Cursor, WindSurf, JetBrains IDE suite in Android Studio, AppCode, CLion, DataGrip, GoLand, IntelliJ IDEA, and PHPStorm. do. PyCharm, RubyMine, RustRover, and WebStorm. We’ve also added the Warp and Prompt terminal apps as integrations. These applications combine VS Code, Xcode, Terminal, iTerm 2, and TextEdit into an integrated app.
However, coding applications are not the only applications that the ChatGPT desktop app has access to. OpenAI has also added Apple Notes, Notion, and Quip to its integration. Advanced Voice Mode can work with these applications by taking the context of the project into account in the integration.
OpenAI emphasized that users must grant ChatGPT permission to access these applications.
Allow ChatGPT to use your computer on your behalf
Of course, app integration with AI chatbots is nothing new. In October GitHub Copilot adds coding platform integration. When you connect your application to ChatGPT or Copilot, you bring the context of those platforms into your chat experience. Developers can ask ChatGPT for coding help for their projects in VS Code, and the chatbot will understand what the developer is working on.
Kevin Weil, OpenAI’s chief product officer, said during the live stream that improving the desktop app will help the company move closer to a more engaged user experience for ChatGPT.
“We’ve been putting a lot of effort into our desktop app,” Weil said. “As our model becomes more powerful, ChatGPT will become more and more of an agent. This means we go beyond just questions and answers. ChatGPT starts the work for you.”
He added that desktop apps are a “big part” of this change.
“Being a desktop app allows you to do a lot more than just a browser tab,” he said. “This includes the ability to see what’s on your screen with your permission and automate many of the tasks you’re doing on your desktop. “We will have more to say about this by 2025.”
As OpenAI allows ChatGPT to see more of your computer, ChatGPT will get closer to the capabilities of Anthropic’s Claude computing. This allows Claude to click around a person’s computer, navigate the screen, and even type text.
OpenAI has already announced fairly similar functionality for the mobile version of ChatGPT, but the chatbot doesn’t yet have access to your computer or phone in the same way. Users can share their screens with the chatbot to “see” what they are reading. microsoft and Google It has also developed similar features to Copilot Vision and Project Astra.
Connection method
Users who want to open ChatGPT while using other applications on MacOS can use “Option + Space” to bring up ChatGPT and select the required application through the buttons on the chat screen. Another shortcut “option + Shift + 1” brings up the top application used.
From that window, users can also access Advanced Voice Mode in the same way. Voice mode automatically detects the context of your application.
The integration is available to ChatGPT Plus, Pro, Team, Enterprise, and Edu users. However, Enterprise and Edu subscribers will need to ask their IT administrator to turn this feature on.
