Microsoft wants to bring generative artificial intelligence to the forefront of Windows and the computers that run it.
In several keynotes this week during the annual Build developer conference, the company unveiled a new line of Windows machines it’s calling Copilot+ PCs, as well as generative AI-powered features like Recall that help users find apps, files and other content. viewed in the past. Copilot, Microsoft’s brand of generative artificial intelligence, will soon be more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are on the way.
We’ve rounded up all the key announcements from Monday and Tuesday here.
Volumetric programs
Microsoft brings Windows Volumetric Apps—basically spatially aware, interactive VR apps—to the Meta Quest headset. Through the partnership with Meta, Microsoft says it will bring Windows 365 and local PC connectivity to the Quest headset, allowing developers to extend their apps into the 3D space.
During Tuesday’s unveiling, Microsoft showed off a digitally exploded 3D view of the Xbox controller from the perspective of the Meta Quest 3 headset — a digital object that the user can manipulate with their hands. “We’re deepening our partnership with Meta to make Windows a world-class experience on Quest devices,” said Pavan Davuluri, CVP of Windows and Microsoft’s devices, during the demo.
Developers can sign up for a preview Get access to Microsoft’s new volume API.
Copilot+ PC
Copilot+ PCs are Microsoft’s vision of AI-first advanced Windows hardware. All include special chips called NPUs to power AI experiences like Recall. And they ship with a minimum of 16GB of RAM combined with SSD storage.
The first Copilot+ PCs will pack Qualcomm’s Snapdragon X Elite and Plus chips, which Microsoft claims provide up to 15 hours of web browsing and 20 hours of video battery life. Chipmakers Intel and AMD are also committed to creating processors for Copilot+ devices, partnering with a number of manufacturers including Acer, Asus, Dell, HP, Lenovo and Samsung.
Copilot+ computers start at $999, and some are available for pre-order today.
Surface Pro and Surface Laptop
Microsoft’s newly introduced Surface devices, the Surface Laptop and Surface Pro, focus on performance and battery.
Available with a 13.8- or 15-inch display, the latest Surface Laptop has been redesigned with “modern lines” and slimmer display bezels. The company says it lasts up to 22 hours on a charge and is up to 86% faster than the Surface Laptop 5. It also supports Wi-Fi 7 and has a touch feedback sensor.
As for the new Surface Pro, Microsoft says it’s up to 90% faster than the previous-generation Surface Pro (Surface Pro 9), and it features a new OLED with HDR display, Wi-Fi 7 (and optional 5G), and improved ultra wide front camera. Also, its detachable keyboard, reinforced with extra carbon fiber, now has haptic feedback.
Remember
Windows 11’s upcoming Recall feature can “remember” programs and content a user accessed on their computer weeks or even months ago, helping them find a Discord chat where they discussed clothes they were thinking of buying, for example. Users can use the Recall graph to “Scroll Back” to view files such as PowerPoint presentations to see what they worked on in the recent past and display information potentially relevant to their search.
Microsoft says that Recall can make associations between colors, images, and more to let users search for virtually anything on their computer in natural language (not unlike the startup’s Rewind technology); developers will be able to enhance Reminder by adding contextual information to their apps. And Microsoft claims that all user data associated with Recall is kept private and on-device — and, crucially, not used to train AI models.
Here it is more From Microsoft: “Your photos are yours; they remain local to your computer. You can delete individual images, adjust and delete time ranges in Settings, or pause at any time from the System Tray icon in your Taskbar. You can also filter apps and websites without ever saving them.”
Photo editing and live translations
Windows now has more AI than ever, and some of it is only on the new Copilot+ PCs.
A new feature called Super Resolution can restore old photos by automatically zooming them in. And Copilot can now analyze images to give users ideas for creative compositions. Through a feature called Cocreator, users can create images and also ask an artificial intelligence model to follow their drawing to modify or re-compose the image.
Elsewhere, Live Subtitles, with live translations, translates any audio coming through the computer—whether from YouTube or a local file—into the language of the user’s choice. Live translations will initially support around 40 languages, including English, Spanish, Mandarin and Russian.
A separate but related new feature in Microsoft Edge offers real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, Bloomberg and more. Coming in the near future, the feature supports Spanish to English and English to German, Hindi, Italian, Russian, and Spanish, live translation of spoken content with both dubbing and subtitles.
Team Copilot and extensions
Team Copilot is the latest extension of Microsoft’s growing Copilot suite of generative AI technology. It integrates with the company’s video conferencing software, Teams, to help manage meeting agendas and notes that anyone in the meeting can co-author. And it extends to Loop and Planner, Microsoft’s collaboration and scheduling platforms, for creating and assigning tasks, tracking deadlines, and notifying team members when they need to be included.
In somewhat related Copilot news, Microsoft has launched Copilot Extensions (in private preview), which allow developers to extend GitHub’s code generation tool GitHub Copilot with third-party apps and capabilities. Launch partners include DataStax, Docker and LambdaTest; extensions will live on the GitHub Marketplace, but developers will also be able to create their own custom extensions to integrate with their internal systems and APIs.
Windows Copilot Runtime
Powering capabilities like Recall and Super Resolution is the Windows Copilot Runtime, a suite of ~40 generative AI models that make up what Microsoft describes as a “new layer” of Windows. In tandem with semantic indexing, a native vector-based system for personal Copilot+ PCs, Windows Copilot Runtime enables generative artificial intelligence applications, including third-party applications, to run without the need for an internet connection.
“[The runtime] Includes Studio Effects, Live Subtitle translations, OCR, recall by user activity and ready-to-use AI APIs. [more]It will be available to developers in June,” Davuluri said on Tuesday.
Popular video editor CapCut from TikTok owner ByteDance will use the Windows Copilot Runtime and its accompanying new Windows Copilot Library, API and set of AI tools to accelerate AI features, Microsoft says. And Meta will add the aforementioned Studio Effects to WhatsApp to introduce features like background blur and eye contact during video calls.
Improved bot builders
Azure AI Studio, a set of tools in Microsoft’s Azure OpenAI Service that allows customers to integrate an artificial intelligence model and create an application “based” on that information, will soon enable developers to build applications using increasingly payment APIs. through which developers can access and fine-tune generative AI models deployed on Azure infrastructure. Microsoft calls it the “as-a-service model,” and it’s starting with the Nixtla and Core42 models.
In its neighboring Copilot Studio suite of products, Microsoft is launching Copilot agents, which the company describes as AI bots that can “autonomously manage tasks tailored to specific roles and functions.” (Copilot Studio provides tools to connect Copilot for Microsoft 365, the AI-powered “copilot” in applications like Excel and Word, to third-party data.) Using memory and context knowledge, Copilot agents can manage various business workflows, learn from user feedback and asking for help when faced with situations they don’t know how to handle.
Snapdragon Dev Kit
There’s a new development kit from Qualcomm aimed at developers building apps for Arm-chip Copilot+ computers.
By the way, the $899.99 Snapdragon Dev Kit for Windows, which is the same width, height and length as Apple’s Mac Mini, has Qualcomm’s Snapdragon X Elite chip paired with 32GB of RAM, 512GB of storage and lots of I/O. The Dev Kit supports Wi-Fi 7 and Bluetooth 5.4 and can control up to three 4K monitors simultaneously via various USB-C and HDMI ports.
Phi-3
Microsoft has announced an addition to its Phi-3-vision family, a generative artificial intelligence model capable of performing common visual analysis and reasoning tasks, such as answering questions about graphs and images. The model can read both text and images and is efficient enough to work on a mobile device.
Phi-3-vision is available in preview, while the model’s previously announced text-only counterparts – Phi-3-mini, Phi-3-small and Phi-3-medium – are now generally available.
In partnership with Khan Academy
Microsoft is donating access to cloud computing infrastructure with Khan Academy, enabling Khan Academy to offer teachers in the US free access to Khan Academy’s AI-powered tools. Microsoft said Tuesday that the two companies will also collaborate to explore opportunities to improve artificial intelligence applications for math classes through generative artificial intelligence.
Introducing the AI Newsletter! Subscribe here To start receiving it in your inbox from June 5th.