This article is a companion piece to the Tech Made Fun Episode – TMF 019 – Apple Intelligence – AI for the ‘Best’ of Us – All You Need to Know About AI in Pakistan
Announcing a bunch of AI features at this year’s WWDC, Apple unveiled “Apple Intelligence”, a term it uses to describe a suite of AI features releasing this fall on select products in the Apple lineup. After all of the big tech, Finally, Apple has given its users some taste of AI with some groundbreaking efforts in user privacy and data, Some of the highlights of this event are discussed below:
AI Writing tools
At the forefront of the plethora of announcements come the brand-new system-wide AI writing tools, giving users the ability to rewrite, proofread and summarize text nearly everywhere they write. Rewrite changes the tone, Proofread checks for writing mistakes, Summarise does as the name suggests. In Mail, users have a priority view at the top where they can interact with important emails. Mail users will also have access to Smart Reply, a feature that analyzes the email content including questions asked and suggests replies based on the analysis. Priority notifications appear are stacked on the top of the lock screen and summaries help scan long or stacked notifications.
The new Siri
Siri is getting an overhaul not just with its logo but with its functionality and features. Siri will now talk more naturally and be able to understand better thanks to rich AI features coming its way. Instead of the Siri bubble, a glowing light would illuminate edges of the screen. Siri will now be able to maintain the context across user requests. The voice assistant will also be able to perform hundreds of new actions across Apple and third-party apps, like opening articles from a Reading List or looking up a specific photo in your library or a message or email that you just received. In place of the traditional mode of speaking to Siri, users can now engage with it through text prompts if they like.
Improved Media search
Apple users get improved search for photos and images across the device. Users could type in a description to bring out photos and videos matching it with the ability to look for specific moments in a clip. The new clean up tool allows users to remove unwanted objects in the background of a photo. Using Memories, users could craft a storyline using existing photos and videos with chapters and segments, based on identified themes from the photos just by typing in a description.
Image Generation
‘Apple Intelligence’ users would be able to generate images choosing between three different styles – animation, illustration and sketch, across various native apps using the Image playground feature. It’s built into apps like Messages and Notes and is also available as a dedicated app. Apple claims all images are created on-device. Apple also announced a similar feature called Genmoji that lets users create an emoji by typing in a description through a prompt that could be shared in-line with messages or as a sticker. Users can generate emojis by addings pictures of themselves or their friends or family.
ChatGPT
Across the suite of AI features that Apple announced, there would be the integration of ChatGPT that users could use to generate text or images using the GPT4o model across Apple AI features. Users won’t need to sign up for a paid subscription for ChatGPT unless they want to. ChatGPT is a start with other AI models to come. Apple says users are asked before any questions are sent to ChatGPT, along with any documents or photos, and Siri then presents the answer directly.
Privacy & Security
Through Apple AI, the company has made a strong emphasis on privacy and security of user data. Here are some of the ways it aims to achieve that:
- On-Device Processing
Like other players such as Google and Samsung, Apple has prioritized on-device processing of data and claims that many of the models that power its AI features run on the device where that’s possible. Though complex requests are sent to Apple servers for further processing. In context of its AI features, the company says that user photos and videos are kept private on device and are not shared with Apple or anyone else.
- Private Cloud Compute
Such requests that can’t be processed on the device itself are sent to Apple servers for further processing. The company says its using its own apple silicon servers to process AI requests and even it cannot see or store user data sent to its servers. Apple claims that independent experts can review the code that runs on Apple Silicon servers to verify privacy.
- Third-party Data sharing
In addition to its own AI features, Apple has introduced some level of privacy and security in its relation with third party services like ChatGPT. It claims to obscure the user’s IP address when initiating a request to OpenAI’s servers and that OpenAI won’t store user requests either.
Questions
How is Apple’s take different?
Privacy – Instead of adding this as an afterthought, Apple here has made it a significant part of its event introducing Apple Silicon based servers for AI computer offloading, the new hardened OS that’s going to run on those servers, their promise of allowing independent experts to review the privacy and security of their servers, cryptographic verification of remote servers by the device, give Apple an upperhand to all the rest and an example the industry could use as a lesson.
Baked-in Integration – Apple Intelligence is baked into various device-native apps like Messages, rather than begin a separate app or assistant like Google Assistant or Bixby. Apple’s greater control over their apps and hardware helps in that.
Phased Rollout – The company doesn’t want to bring all of its experimental features out at once, instead it plans on an incremental rollout of its AI features divided between months aiming to provide a better experience than the rest.
Why is Private Cloud Compute revolutionary?
There are multiple reasons that make it groundbreaking in the privacy and security space. Some of these are discussed below:
Private Cloud Compute Node – It is the custom-built hardware using Apple Silicon that Apple claims is more secure than any other processor out there. This specialized hardware is using Apple’s same security technology as being used in other Apple products including Secure Boot, Secure Enclave. Couple all this with a new purpose-built operating system designed specifically for security in mind with additional checks like Code Signing and sandboxing. With all that, Apple’s not done here, It has excluded traditional data center administration tools like remote shells and system inspection tools that allow unrestricted access to the complete server with all its data to the administrator.
Stateless Computation – Unlike messaging services like Whatsapp or iMessage that don’t need to have access to the actual data that is being transmitted, E2E or End-to-End encryption can’t be used to offload AI work to the cloud since the remote computer needs to see the data in plain text to work on it. To combat that Apple has enforced some new guarantees explained below.
- Data leaves the Apple device encrypted after cryptographically verifying the PCC node, with the PCC node’s public key so no one but that PCC node could read or tamper with the data being transmitted.
- PCC node receives that data, decrypts it using its private key and performs the required computation on it.
- After the computation the data is encrypted again but this time with the Apple Device’s public key that sent it so it can’t be tampered with or be ready by anyone during transmission.
- As the data reaches the back to the client. The PCC node deletes the data. User data is never available to Apple — even to staff with administrative access to the production service or hardware.
The above explanation was made to be short and simple for the average Bashir. If you’re interested in diving deep, check the reference articles down below.
Why did Apple go to ChatGPT?
Because they don’t have a model that could compete with ChatGPT yet when it comes to advanced Siri queries. Though Apple went with ChatGPT, they plan to go with Google too in near future.
Does this mean AI is here to stay?
Yes.
On Device AI Processing Impact
- Enhanced Privacy and Security – The less data that leaves the device the more private the system since there won’t be risks in transmission and other endpoints to worry about.
- Improved Performance – Since data doesn’t have to travel to the other side of the world and then come back with many more complex processes in between, for the end user it means quicker responses to their AI requests.
- Offline processing – On-device processing of AI features doesn’t require a constant internet connection that may be better for areas with poor internet connection.
- Cost Savings – You own the hardware and don’t need to pay for subscriptions or bear costs for using someone else’s cloud processing infrastructure.
- Personalized Experience – AI models running on-device can tailor interactions and responses more closely to the user’s behavior and preferences since they have access to data on the device unlike the cloud that gets access to only the data they need.
A guy trying his hardest to not end up as a Writer. Alternatively someone who has deep passion for the field of CyberSecurity and overlapping ideologies. I am the one and only researcher for Tech Made Fun Tech Podcast – patent pending.
