Every new feature Apple unveiled for iPhone, iPad, Mac, and more this fall (including AI)

Apple unveiled what it calls its new “private intelligence” system below the title Apple Intelligence. The discharge places generative fashions on the coronary heart of the ecosystem of Apple gadgets. 

With Apple Intelligence, your iPhone can prioritize notifications to make sure you get notified solely when it is essential all through your day.

The discharge consists of writing instruments that leverage AI, together with rewriting, proofreading, and summarizing textual content options accessible throughout mail, keynotes, third-party apps, and extra. 

Customers can now create personalised photographs within the picture library, together with sketches, illustrations, and animations. This characteristic is on the market in Messages, Apps, Freeform, Keynote, and Pages. 

Apple Intelligence can faucet into instruments and perform duties in your behalf, reminiscent of “Present me all of the photographs,” “Play the podcast,” or “Pull the information that my coworker shared with me final week.” 

As a result of it is grounded in your private info and context, and may retrieve knowledge from throughout your apps and reference the content material in your display screen, Apple Intelligence is positioned to be your private assistant.

Apple emphasised the security and privateness precautions constructed into Apple Intelligence, significantly for on-device intelligence processing. The corporate touted the safety of Apple’s silicon, A17 Professional, and its M household of chips (M1, M2, M3, and M4). 

For duties which can be too massive for on-device processing and must be accomplished within the cloud, Apple unveiled Personal Cloud Compute, which protects customers’ privateness by operating on servers specifically created utilizing Apple Silicon. When customers make requests, Apple Intelligence first exams on-device functionality, however calls on Personal Cloud Compute if the duty requires extra energy. Apple reiterated that consumer knowledge isn’t saved or offered to exterior events. 

Siri lastly obtained the AI makeover it deserves, first with a brand new look: when tapped, gentle wraps across the edges of your display screen. Siri can now higher perceive customers, even when they stutter, because of extra superior pure language processing (NLP). It now has conversational context, remembering what you simply mentioned and utilizing it to finish the following process. Customers may also sort requests to Siri. As a result of it has in-depth product data, Siri can reply questions on performance on iPad, iPhone, and Mac. Siri will even have Apple Intelligence’s on-screen consciousness, permitting it to behave on what it sees. The voice assistant may also take actions throughout apps, together with picture enhancing. With entry to your private context, Siri can perceive and full new instructions, reminiscent of pulling your driver’s license info from a photograph and inputting it right into a type. The Siri updates are coming to iPad and Mac, too. 

Apple Intelligence additionally powers new options in Mail, together with Rewrite, which gives customers completely different variations of what they’ve already drafted. Solutions are proven in-line, and Proofread edits for grammar, phrase selection, and sentence construction. You may also use Summarize to transform your textual content into bullet factors. Good Reply identifies related choices of an e-mail and makes use of them to assist craft a customized message. Summaries will now seem on the prime of emails, making shopping an inbox simpler. Apple Intelligence may even assist prioritize your emails, putting what’s most necessary on the prime of your inbox. 

There may be an all-new focus possibility: cut back interruptions. When on this setting, your telephone will solely present you what’s most necessary primarily based in your private exercise and context. 

Genmoji permits customers to create AI-generated emojis primarily based on what they sort. You may also create a Genmoji primarily based on a photograph of a buddy. Genmojis might be included in-line in Messages and even used for Tapbacks. 

Picture Playground permits customers to leverage AI on-device to create photographs from textual content prompts, which might be simply shared in iMessage and elsewhere. The characteristic can also be accessible in Keynote, Pages, and Freeform, and as a stand-alone Picture Playground app. 

Picture Wand within the Notes app transforms a tough sketch into a sophisticated picture and is on the market instantly within the device palette. For instance, you’ll be able to circle a tough sketch in Notes and open Picture Playground to rework your doodle right into a fully-fledged picture. 

Apple Intelligence will even improve the Photographs app with a brand new clean-up device that removes undesirable objects. Search in movies permits customers to simply discover particular snippets of content material, and customers can create Recollections on-demand, utilizing textual content to edit and set up photographs into motion pictures. 

Within the Notes app, customers can document and transcribe audio, which Apple Intelligence will generate a textual content abstract of. This expertise can also be accessible within the Notes telephone app. 

Apple Intelligence is free on iOS 18, iPadOS 18, and MacOS Sequoia, and might be accessible to strive in English solely this summer season. 

iPhone and iPad customers will have the ability to customise their dwelling display screen additional by putting apps wherever they’d like on the display screen, versus the standard mounted grid. App icon colours will even be customizable, permitting customers to make apps any coloration they need and even match their dwelling display screen. Customers may also change app icons to darkish mode. 

After 5 years of remaining untouched, the Management Heart obtained a number of upgrades, together with the flexibility to customise its toggles, reminiscent of flashlight, display screen recording, calculator, auto-rotate, display screen mirroring, and extra, by tapping, holding, and rearranging. The Management Heart toggle will even characteristic completely different pages with utterly customizable consumer controls, and customers can change controls on the underside of the Lock Display. 

Apple additionally added privateness choices, together with the flexibility to lock an app, which requires customers to authenticate with Face ID or passcode earlier than accessing the app. Customers may also disguise an app, which makes it disappear from the house display screen to a hidden a part of their app library.  

Messages obtained a number of upgrades. Tapbacks, the characteristic permitting customers to react to messages by holding them down, was upgraded to characteristic completely different colours and embody emojis. Customers can add textual content results to particular phrases or phrases as an alternative of the complete phrase. Texts will also be personalized additional with formatting choices like daring, underline, italics, and strikethrough. Lastly, customers will have the ability to schedule messages.

iPhone 14 and later fashions can have a brand new Messages through Satellite tv for pc characteristic, which permits customers to ship messages through satellite tv for pc once they haven’t got Wi-Fi or mobile service. 

The Mail app will mechanically categorize emails, a characteristic that might be accessible later this yr. 

The Pockets app now permits customers to faucet telephones collectively to trade Apple Money with out requiring them to share private info like telephone numbers. 

The Journal app will now present extra statistics and insights, together with what number of entries you’ve got had this yr, what number of days you journaled, and extra. 

There’s a new Recreation Mode for iPhone, meant to assist avid gamers optimize their gaming expertise. This consists of minimizing background exercise and utilizing extra responsive equipment, reminiscent of controllers. 

The Photographs app obtained what Apple dubbed its “largest ever redesign,” that includes a cleaner design, a brand new carousel with highlights that replace every day, the flexibility to pin collections, and an improved search. 

The Messages app now helps Wealthy Communication Providers (RCS).

The Safari app was upgraded to incorporate key details about a webpage. You possibly can learn extra concerning the Safari upgrades below the MacOS part of this text. 

The Calendar app can now pull from the Reminders app for a extra seamless schedule overview. 

The brand new coaching load permits customers to achieve insights into how their exercises’ depth impacts their long-term efficiency. 

The new Vitals app will give customers a fast take a look at their most necessary well being metrics, together with coronary heart charge, respiratory charge, wrist temperature, sleep period, and blood oxygen. It would additionally present context to assist them make extra knowledgeable choices. If one thing appears out of the bizarre, customers will obtain pings alerting them of the anomaly. 

When a consumer logs a being pregnant within the Well being App on iPhone or iPad, the Cycle Monitoring app on Apple Watch is upgraded to indicate gestational age, and permits customers to log signs skilled throughout being pregnant. Pregnant customers may also ask to be reminded to take a psychological well being evaluation each month. Utilizing the Strolling Steadiness characteristic, customers will also be alerted of elevated fall danger. 

Customers will even expertise extra customizable Exercise Rings, which permit them to pause their rings once they wish to take a time without work with out impacting their award streaks. 

Apple Health+ was upgraded to incorporate personalised For You tabs, Discover and Library tabs, search options, and enhanced awards.  

Good Stack can also be getting extra clever: it is now capable of recommend widgets when wanted mechanically, amongst different enhancements. 

With watchOS, customers will get solutions on one of the best picture choices for his or her watch face from their picture library primarily based on standards reminiscent of aesthetics, composition, facial expressions, and extra. The face will also be personalized with completely different sizes, layouts, and fonts. 

Test In is on the market on Apple Watch, even throughout exercises, to assist customers keep secure. Translate can also be coming to Apple Watch so customers can translate textual content proper from their wrists. 

One other notable change is that the double faucet gesture can be utilized to scroll by apps. 

The replace will characteristic a redesigned tab bar that floats above app content material. Customers can customise it to showcase their favourite apps and entry crucial sections of an app. You may also long-tap the bar to maneuver it round. The tab bar additionally morphs into the sidebar for added insights. 

Shareplay will permit customers to remotely management another person’s iPad or iPhone and share drawings on their screens.

In a long-awaited launch, iPads will now have a calculator app for the primary time, full with the identical interface because the one presently discovered on iPhones. Plus, customers can use it with the Apple Pencil by a brand new Math Notes expertise, which permits customers to write down expressions that the calculator app will resolve for them as soon as they sort the equal signal, in their very own handwriting, 

Handwriting in Notes additionally obtained an improve with Good Script, which refines customers’ writing to make it extra legible whereas holding the authenticity of the consumer’s handwriting model. The characteristic may also match copied and pasted textual content to the consumer’s handwriting. Typed textual content was additionally enhanced, with 5 new spotlight colours and the flexibility to toggle sections below headings or subheadings. 

iPadOS 18 helps screen-sharing through SharePlay and the identical Management Heart customizations, Photographs app upgrades, Safari updates, and emoji Tapbacks present in iOS 18.

Apple unveiled MacOS Sequoia, which is able to embody lots of the new options that had been added to iOS 18 and iPadOS 18, together with the up to date Safari, Photographs, Messages, and the brand new Passwords administration app.

The brand new iPhone mirroring functionality on Mac permits customers to expertise their telephone nearly totally from their Mac. For instance, iPhone notifications will now be accessible on Mac, permitting customers to work together with them and open corresponding apps, although the iPhone itself will seem locked. 

Window tiling was made doable to assist customers keep extra organized. When customers drag a window to the sting of the display screen, macOS mechanically suggests a tile place. 

Video conferences are additionally getting an improve, with new backgrounds and a preview expertise that means that you can see what you might be about to share earlier than sharing it. This characteristic works with in style video conferencing functions reminiscent of FaceTime and Zoom. 

The AI summarization device will dwell in Safari to assist customers course of content material like internet pages and articles extra effectively. Safari will even help customers in discovering extra useful details about a web page they’re shopping when related, reminiscent of instructions. 

Apple additionally launched a brand new Viewer expertise, which does for video what Reader does for textual content. 

Apple unveiled the primary main improve to its lately launched VisionOS — VisionOS 2. 

In Imaginative and prescient OS 2, Photographs will get an improve that enables customers to rework any 2D picture right into a Spatial Picture, with added depth from moments already of their digicam rolls.

Spatial Personas within the Photographs app lets customers view photographs collectively, making a extra shared expertise. 

VisionOS 2 additionally helps new hand movement gestures, permitting customers to entry some settings extra simply. For instance, customers can open their arms and faucet to succeed in the house display screen or flip their wrists to see the battery stage. 

Customers who mirror their MacOS to their Imaginative and prescient Professional will quickly get the next decision and larger dimension, creating an ultrawide monitor view equal to 2 4K screens aspect by aspect. 

The Imaginative and prescient Professional will even embody prepare assist for journey mode, making working throughout your commute simpler. 

Imaginative and prescient Professional will now characteristic a Visitor Consumer possibility that enables further customers to save lots of their eye and hand knowledge for 30 days. 

Customers can now personalize their Dwelling View, putting apps wherever they need.

Customers will now have the ability to watch movies in an Atmosphere when utilizing Safari, even on websites reminiscent of YouTube and Netflix. 

Apple TV brings multiview to Imaginative and prescient Professional, which is very helpful when watching sports activities video games. 

Customers can solid content material from their iPhone, iPad, or Mac to the Imaginative and prescient Professional utilizing AirPlay. 

Apple can also be making the Imaginative and prescient Professional accessible in additional nations beginning June 28. You possibly can see the complete checklist right here. 

When customers watch an Apple TV+ present or film, the brand new InSight characteristic on tvOS will embody further info reminiscent of actor names and music titles. Customers can then simply add these music titles to their Apple Music playlist. When utilizing an iPhone as a distant, the InSight info will even seem on the smartphone. 

The Improve Dialogue characteristic was upgraded to ship better vocal readability over different parts of the film or present, reminiscent of music or background noise, on Apple TV 4K. 

Subtitles had been optimized to mechanically seem when the language doesn’t match one of many gadgets, when customers mute, or when skipping again. 

Apple added assist for 21:9 formatting for viewing content material on projectors. 

There are new, enjoyable screensavers, together with Portraits, TV and films, and Snoopy. 

When FaceTiming on tvOS 18, customers will now have the choice of Stay Captions for English within the US and Canada.  

tvOS will even characteristic a redesigned Apple Health+ expertise, together with new For You, Discover, and Library areas.