Windows 11’s AI Recall feature is blasted by a security expert as ‘one of the most ridiculous security failings I’ve ever seen’

• June 3, 2024

Microsoft has already been dragged over the coals regarding its Recall functionality inbound for Windows 11 by security researchers and privacy watchdogs alike – and it’ll need a flame-retardant suit for the latest fiery outpouring against the AI-powered feature.

This comes from security expert Kevin Beaumont, as highlighted by The Verge. The site notes that Beaumont worked for Microsoft briefly a few years ago.

To recap – in case you missed it somehow – Recall is an AI feature for Copilot+ PCs, which launches later this month and acts as a photographic timeline – essentially a history of everything you’ve done on your PC, recorded via screenshots that are taken regularly in the background of Windows 11.

Beaumont got Recall working on a normal (non-Copilot+) PC – which can be done, though it isn’t recommended performance-wise – and has been messing around with it for a week.

He’s come to the conclusion that Microsoft has made a giant mistake here, at least going by the feature as currently implemented – and it’s about to ship, of course. Indeed, Beaumont asserts that Microsoft is “probably going to set fire to the entire Copilot brand due to how poorly this has been implemented and rolled out,” no less.

So, what’s the big problem? Well, principally, it’s the lack of thought around security and how there’s a major discrepancy between Microsoft’s description of the way Recall is apparently kept watertight and what Beaumont has found.

As you can see in the above post on X (formerly Twitter), one of the security expert’s main beef with Microsoft is that it informed media outlets that a hacker can’t possibly nab Copilot+ Recall data remotely. In other words, an attacker would need to access the device physically, in-person – and this isn’t true.

In a long blog post on this topic, Beaumont explains: “This is wrong. Data can be accessed remotely.” Note that Recall does work entirely locally, as Microsoft said – it’s just that it isn’t impossible to tap into the data remotely, as suggested (if you can access the PC, of course).

As Beaumont elaborates, the other big problem here is the Recall database itself, which contains all the data from those screenshots and the history of your PC usage – as all of this is stored in plain text (in an SQLite database).

This makes it very easy to snaffle all the Recall-related info of exactly how you’ve been using your Windows 11 PC – assuming an attacker can get access to the device (either remotely, or in-person).

Analysis: Recall the Recall feature, or regret it

There are lots of further concerns here, too. As Microsoft pointed out when it revealed Recall, there are no limits to what can be captured in the AI-powered history of the activity on your PC (save for some slight exceptions, like Microsoft Edge’s private browsing mode – but not Chrome Incognito, tellingly).

Sensitive financial info, for example, won’t be excluded, and Beaumont further points out that auto-deleting messages in messaging apps will be screenshotted, too, so they could be accessed via a stolen Recall database. Indeed, any message you delete from the likes of WhatsApp, Signal, or whatever could be read via a Recall compromise.

But wait a minute, you might be thinking – if your PC is remotely accessed by a hacker, aren’t you in deep trouble anyway? Well, yes, that’s true – it’s not like these Recall details can be accessed unless your PC is actively exploited (though part of Beaumont’s problem is Microsoft’s apparently errant statement that any kind of remote access to Recall data wasn’t possible at all, as mentioned above).

The real kicker here is that if someone does access your PC, Recall seemingly makes it very easy for that attacker to grab all these potentially hugely sensitive details about your usage history.

While info stealer Trojans already exist and scrape victims at a large scale on an ongoing basis, Recall could enable this kind of personal data hoovering to be done ridiculously quickly and easily.

This is the crux of the criticism, as Beaumont explains it: “Recall enables threat actors to automate scraping everything you’ve ever looked at within seconds. During testing this with an off the shelf infostealer, I used Microsoft Defender for Endpoint – which detected the off the shelve infostealer – but by the time the automated remediation kicked in (which took over ten minutes) my Recall data was already long gone.”

This is a major part of the reason why Beaumont calls Recall “one of the most ridiculous security failings I’ve ever seen.”

If Microsoft doesn’t take action before it ships, mind – as there’s still time, in theory anyway, although the release of Copilot+ PCs is very close now. (However, Recall could still be kicked temporarily to touch while it’s further worked on – perhaps).

If Recall does ship as it’s currently implemented, Beaumont advises turning it off: “Also to be super clear you can disable this in Settings when it ships, and I highly recommend you do unless they rework the feature and experience.”

Herein lies another thorny issue: the AI-powered functionality is on by default. Recall is highlighted during the Copilot+ PC setup experience, and you can switch it off, but the way this is implemented means you have to tick a box to enter settings post-setup, and then turn off Recall there – otherwise, it will simply be left on. And some Windows 11 users will likely fall into the trap of not understanding what the tick box option means during setup and just end up with Recall on by default.

This is not the way a feature like this should operate – particularly given the privacy concerns highlighted here – and we’ve made our feelings on this quite clear before. Anything with wide-ranging abilities like Recall should be off by default, surely – or users should have a very clear choice presented to them during setup. Not some kind of weird ‘tick this box, jump through this hoop later’ kind of shenanigans.

You might also like…

Source

Microsoft has already been dragged over the coals regarding its Recall functionality inbound for Windows 11 by security researchers and privacy watchdogs alike – and it’ll need a flame-retardant suit for the latest fiery outpouring against the AI-powered feature. This comes from security expert Kevin Beaumont, as highlighted by The…