iOS 18.2 Beta 2  adds APIs to improve Siri  awareness and Apple Intelligence Content

Apple’s new API in iOS 18.2 Beta 2 will  allow users  to ask  Siri “Hey Siri,  what’s this document  about?” » to get document summaries directly from the assistant.

iOS 18.2 Beta 2 was released to developers on Monday.

The update includes an API  that supports Apple Intelligence  to recognize screen  content.

Users can share their screen content with Siri for analysis. 

On Monday, Apple released iOS 18.2 Beta 2 to developers and testers through the  Developer Beta Channel. This update,  scheduled for early December,  improves the capabilities of Apple  Intelligence. In particular, it introduces an API that  allows developers  to give the system access to  content on the screen, which allows Siri and Apple Intelligence to interact with third-party services  to process the content. Apple’s new API for  on-screen Siri content recognition

In  the documentation  published on  Apple’s developer site (via  MacRumors ), Apple detailed the API called  “Make content  on the screen available to Siri and Apple Intelligence.” This API allows Siri and Apple Intelligence to interpret the content  that users  see in supported  applications.

By integrating this API, developers can enable their apps to share screen content with Siri or Apple Intelligence when users ask, allowing third-party services  (eg OpenAI’s ChatGPT) to process the information.

Apple  showed how Siri can interpret  the content on the screen: a user browsing the web  can ask “Hey Siri,  what is this  document?” » to  get an overview. Developers can add this  on-screen recognition feature to browsers, document readers, file managers, email clients, photos, presentations, spreadsheets, and word processors. Apple  has hinted that more apps may eventually support the API.

Although iOS 18.2  will not include the new Siri with  enhanced functionality (which is expected in iOS 18.4 in April  2025), this API  gives developers time to  integrate content  recognition features  into their apps  before future  improvements to Siri.