WWDC 2024

OS Announcements

Intro

Great opening!

AppleTV+

Lots of new shows and series

VisionOS

Version 2 announced.
Photos
- Converts standard photos into “Spatial” photos using ML to create a Left and Right eye part of the photo.
New gestures for control center-like actions.
Mac hardware display options with higher resolutions*
Train support to complement Flight mode.
New APIs: Volumetric, TableTop, Enterprise
Spatial content improvements via iPhone 15 Pro/Max.
Cannon offers a new Spatial lens for commercial captures.
Vimeo announced a new app with Spatial video support.
Apple immersive video partners: BlackMagicVideo for now, expanding to more providers including Red Bull, The Weekend, and Submerged Short Film.
Expanding to eight more countries this year.

iOS

Home Screen
- Icons and widgets can be arranged anywhere.
- Dark mode icons.
- Complimentary color palettes to match the background photo or custom color.
Control center
- Swipe between screens to show different controls on individualized pages.
- Continuous swipe will scroll through all the various pages.
- New controls gallery to customize.
- Developers can include their own controls.
- Change layout and sizes.
- Controls API
- Lock Screen now also includes Controls.
- Action Button can be customized using these controls.
Privacy
- Lock and app: FaceID/Touch ID/Passcode lock apps that you don’t want others to open.
- Hide an app
- Contacts access: can be limited to which contacts.
- Accessories and network: limit access for limited access.
Messages
- Tapbacks: adding any emoji
- Send Later!
- Text formatting: Bold, italics, underline.
- Text effects expanded.
- Messages via satellite: 14 and later, text off-grid without needing cellular. Fully encrypted, SMS also supported.
Mail
- On device categorization*: Primary, Transactions, Updates, Promotions
- Digest message collects everything together from various businesses. Can be recategorized.
Maps
- Topographic: trails and hiking maps. Nation Parks included and offline.
Wallet
- Tap to cash: tap iPhone to pay someone.
- Apple pay Online
- Event tickets with new design and event guides.
Journal
- Log state of mind
- Track goals
- Insides dashboard
- Search to find past entries.
Gaming
- Game Mode: for iPhone
Photos
- Biggest redesign ever!
- Keeps library organized
- Single view with photo grid on top and options on the bottom
- Filters to limit what you do and do not want to see.
- Intelligence is the buzzword they keep repeating.
- Trips are organized by destinations
RCS Support confirmed.
Reminders integration with Calendar

Audio & Home

AirPods
- Head nod or shake to silently confirm/deny Siri interactions.
- Voice Isolation for AirPod Pro. Expanded to gaming.
Home and tvOS
- Insight: Like Amazon’s X-Ray. Shows actors and songs playing.
- Enhanced dialogue to improve sound quality. Subtitles can automatically come on at key moments, like muting or rewinding.
- Support for 21:9 projector formats.
- More screen savers: Additional Ariels, photo modes, movies and TV, Snoopy!
- Improved Fitness+ app experience.
WatchOS
- Training load for workout tracking. Gives effort rating to provide additional trends.
- Customize the summary tab and activity rings can be adjusted by the day of the week. Can also pause for injury or illness to not miss your streak.
- Smart Stack now includes new things, similar to the old Siri watch face. Auto updates based on need.
- Translation now on Watch.
- Double-tap API for developers to create their own features.
- Photos Watch Face uses ML to find the best photos.
Health
- Vitals app
- Cycle Tracking: Pregnancy improvements.

iPadOS

IOS customizations are coming as well. So are the Photos changes.
Productivity improvements!
Floating tab bar, moves from top and can migrate to the sidebar.
Document browsing is much easier.
Refined animations.
API Elements
SharePlay
- Screen sharing
  - Tap and draw on the screen to show them
  - Can remotely control iPad or iPhone!!!!
- Freeform
  - Scenes
Calculator!
- History
- Apple pencil support!
- Math Notes
  - Creates results from handwritten equations. Recognizes equal sign to solve.
  - Very advanced functions can be created with variables and can create graphs. Insanely impressive!
  - Also works in Notes! All features are included there as well.
Notes
- Smart script: improves handwriting clarity and legibility. Paste typed info in and it turns it into handwritten notes.
- Collapsible sections

MacOS

Crack marketing team named it: Sequoia.
Math notes with typing.
Hike planning in Maps.
Continuity:
- iPhone mirroring: see and control iPhone remotely.
- iPhone notifications: integrates with Mac notifications and can open mirrored iPhone right up on the Mac.
- iPhone screen stays locked. Works with standby seamlessly and stays active while used.
- Drag and drop support between Mac apps and iPhone mirrored apps.
Windows arrangement: Snaps to position (Sherlocked Rectangle)
Video conferencing: presenter preview and backgrounds.
Passwords: independent app! Really pretty, similar to Reminders. Bonus: Bringing this to Windows!
Safari
- World’s fastest browser.
- Privacy: industry-leading tracking prevention and private browsing.
- WebKit based
- Trying to reintroduce Safari.
- New features
  - Highlights: show important information on a webpage using ML
  - Reader: redesigned, summary and table of contents
  - Viewer: full video controls and PIP integration. Slam against YouTube.
Gaming
- Baulders Gate 3 coming soon.
- Unified gaming platform due to integrated lineup.
- Game Porting Toolkit 2 announced.
  - Improved windows compatibility
  - Assists with migrating to iPhone and iPad.
- More games!
  - Frost[something]
  - Ubisoft: Assassin’s Creed just released, Prince of Persia, Assassin’s Creed Shadows.
  - A bunch of new ports coming!
Whole bunch of new APIs.

Artificial Intelligence and Machine Learning Announcements

Apple Intelligence: next big step from Apple
Personal intelligence system
Generative models using your personal self while staying private. Deeply integrated with platforms and apps.
Language
- Priority notifications
- Writing tools: rewrite, proofread
Images
- Photos, emojis, GIFs
- Knows the people in your photo library, and generates an AI image of them for their birthday. Mom can be a superhero. Different styles can be applied.
- Notes, Keynote, Pages, Freeform.
Actions
- Pull up the files that were shared last week.
Personal context
- Personal information can be analyzed and used
- What’s on the screen
Privacy
- On-device processing
- On-device semantics index to understand relevant personal data
- Can tap into external sources using the cloud: Private Cloud Compute. Focus on keeping private and secure connections to remote clouds so no data is leaked. Intelligently sends only needed data out and only when needed.
- New standard for privacy and AI.
Experiences
- Siri enhanced
  - 1.5 billion requests a day
  - 13 years ago released
  - Now more natural relevant, and personal.
  - New look.
  - Natural language can find out what you need, even when you make a mistake.
  - Conversational context lets you chain requests together.
  - Now you can type to Siri, double tap the bottom of the screen to privately use without using out loud commands.
  - Groundwork for improved future uses: product knowledge for features and settings to tell you how to use your devices. Can describe a feature and Siri will tell you how to use it. Will continue to roll out features for this.
  - On-screen awareness: can get an address from the screen and add to a contact card.
  - In-app actions: App Intents (framework) improvements. Adding App Intents API for developers to use.
  - Personal context: can be linked between apps and services. can ask to find things on-device and it will pull it up for when you don’t know what app it was sent in.
  - Can take info from photo ID and fill out a form.
  - Siri can help with timing for things like flight tracking and will be aware of texts of your lunch plans; can all be done without opening those apps.
- Writing Tools
  - Rewrite
    - Can look at different versions (verses) for different contexts for writing suggestions.
    - Correct for tone.
    - Change text into a poem for a unique party invite
    - Proofread
    - Summarize to pull out key points and make a TL;DR at the top automatically.
  - Smart Reply
    - Suggested responses with intelligent responses.
  - Summarize
    - Inbox Summaries created from the email content
    - Summarize long messages
  - Priority
    - Surface more important emails to the top of the inbox
- Notifications can also take advantage of these features.
  - Reduce interruptions
- Images
  - Create images on the fly
  - Genmoji – custom emoji based on prompts. can make one’s based on friends in photos. And can be created inline.
  - Image playground: fun way to create images easily using categories to generate images. Integrated into apps, an independent app, and developers can integrate with API.
  - Notes
    - ImageWand: circle sketches and use the words to create images. Can circle a space to create an image based on the notes around the space.
  - Photos
    - Clean Up: removes back grad objects.
    - Improved search with natural language for photos. Videos can be found inside the video itself to pull you right to the point in the video when your search mentions.
    - Memory movie: interpret a typed description into a new memory. Creates a storyline and chapters with a narrative arch.
Notes: record and transcribe notes. AI creates a summary after.
Phone: record and transcribe calls.
All free updates.
ChatGPT integration, 4.0 model.
- Siri can take questions and submit to ChatGPT after confirmation.
- Can also submit photos, PDFs, and documents for suggestions.
- Compose for writing and art.
- Free!
- Subscribers can access paid features.
Future integrations coming later.
Developers and AI.
- SDKs
  - ImagePlayground
  - Writing Tools
  - SiriKit
  - App Intents
- Generative xCode: code completion and smart assist for code questions.
AI for the rest of us.
Rolled out with this year’s OSs and additional features over the next year. All AI starts in Beta.