What is an AI PC? (And should you buy one?)

Think of an AI PC as a supercharged personal computer, tailor-made with the right hardware and software to handle AI and machine learning tasks like a pro. It's all about giving those number-crunching, data-sifting AI and machine learning tasks the power boost they need.

These tasks cover a wide spectrum of workloads including generative AI programs like Stable Diffusion, clever chatbots powered by local language models, comprehensive data analysis, training of AI models, and running intricate simulations and sophisticated AI-driven applications.

Also: This mini PC is basically a Mac Mini for Windows users

Along with power processors (CPUs) and graphics cards (GPUs) to give the systems multitasking might, and ample RAM and swift storage options, these PCs boast something new: an NPU, or neural processing unit, designed to turbocharge AI tasks.

Intel and Microsoft have also come together to define what an AI PC is, and in addition to needing a CPU, GPU, and NPU, these PCs will also have a physical Microsoft Copilot key on the keyboard, and be able to run Copilot locally. 

And, as we'll see when we come onto the measure of the power and performance of an NPU, this has performance requirements that right now are hard to meet. 

The goal is to create systems that are quicker at executing AI tasks, more energy-efficient, and eliminate the need to send data -- especially sensitive data -- to cloud-based AI servers for processing. This approach ensures that systems can operate independently of an internet connection, and enhance security by retaining the data locally.

An NPU, short for neural processing unit, is a specialized processor that's tailor-made to manage the heavy lifting for AI tasks, a job typically assigned to the GPU.

While a PC's GPU can certainly power through AI workloads, it can be quite a power-hungry endeavor. That's not ideal for laptops where battery life is precious, and it's not the most efficient choice for desktops either.

Also: What is deep learning? Everything you need to know

Right now, NPUs aren't quite ready to take the reins from GPUs entirely. It's now more like a partnership, with the two working in tandem to slash processing times while at the same time curbing power usage.

But this is just the beginning, and next-gen NPUs will likely be up to speeding through AI tasks all on their own, leaving GPUs to focus on what they do best. While that's the endgame, we're just not quite there yet.

Tech titans Intel and AMD are wading into the AI PC arena in a big way.

Processor ranges such as Intel's Core Ultra or AMD's Ryzen 8000G (previous to this, AMD had the Ryzen 7040 series which featured the "Ryzen AI" engine), respectively, each feature an NPU to take on some of the AI workload -- workloads such as AI-effects on video calling and for video generation, better multi-tasking with AI-accelerated software, and AI assistants.

You're going to see "productivity" and "efficiency" and "advanced collaboration" bandied around a lot.

Also: Your next phone will be able to run generative AI tools (even in Airplane Mode)

But remember, Intel and AMD are somewhat late to this game. When Apple kicked Intel to the curb and replaced the chip giant with its own line of M-series processors in 2020, these chips all featured "neural engine" NPUs.

But the story goes further back. In September 2017, Apple unveiled the A11 Bionic chip for the iPhone, which was the first of its chips to feature a neural engine. Some Qualcomm Snapdragon mobile processors also featured neural engines back in 2018.

Intel and AMD aren't so much leading the way, but instead playing catchup.

NPU performance will be measured in TOPS -- Trillion Operations Per Second -- and this figure is likely to become the de facto measurement for an NPU. 

For example, a report by Tom's Hardware with Todd Lewellen, the Vice President of Intel's Client Computing Group, said running Microsoft's Copilot AI service locally rather than in the cloud would require an NPU with a minimum of 40 TOPS of performance.

The downside of finding this out is that the current crop of silicon from Intel and AMD -- Meteor Lake and Ryzen Hawk Point, respectively --  fall short of this by over a half, unable to hit 20 TOPS.

Qualcomm will likely have its Snapdragon Elite X chips out this year that will feature the company's Hexagon NPUs that will be capable of 45 TOPS.

It's hard to take benchmarks and translate these into real-world numbers but Intel has published data on how its Core Ultra chips perform under different workloads compared to the competition (the competition in this case being AMD).

There's nothing like a new release of Windows to get both consumers and businesses to upgrade their hardware, so PC makers are hoping that Windows 12 will make a big AI push.

Also: Windows 12 FAQ: Yes, it's coming in 2024 (and more surprising predictions)

And if just some of the hints are rumors are true, AI is going to be a cornerstone to the new Windows release, and it will be a way for Microsoft to give Windows Copilot -- formerly Bing Chat, the company's centralized AI assistant -- more features, and closing the gap between Windows and Apple's MacOS.

Big name OEMs such as Dell, Lenovo, and HP are all rolling out AI PC systems, powered by Intel and AMD processors that feature NPUs.

Good question! On one hand, the true potential of AI PCs might not fully become clear until Windows 12 rolls out (likely later this year), which will give us a better idea of what the current crop of AI PCs will be capable of.

Also: I turned my laptop into a desktop PC and I've never been more productive

On the other hand, if you're in the market for a new PC anyway, opting for one with AI capabilities seems like a smart move. It not only future-proofs your investment but also offers immediate benefits, like blurring your video call background or quickly summarizing lengthy documents. If this sounds useful to you, it might just be worth considering.

While we don't have any official word on what the system requirements of Windows 12 will be, you can be sure that this current crop of AI PCs will have been built by the OEMs with a lot of feedback from Microsoft.