Windows AI feature that screenshots everything labeled as security ‘disaster’

Microsoft is about to launch a new AI-powered Recall feature that captures screenshots of everything you do on your PC. Recall is part of the new Copilot Plus PCs that will be launched on June 18th, but experts who tested the feature are already warning that Recall could be a “disaster” for cybersecurity.

Recall is designed to use local AI models to capture images of everything you see or do on your computer and then provide the ability to search and retrieve anything in seconds. There’s even an explorable timeline you can scroll through. Everything in Recall is designed to stay local and private on the device, so no data is used to train Microsoft’s AI models.

Despite Microsoft’s promises of a secure and encrypted Recall experience, cybersecurity expert Kevin Beaumont has found that the AI-powered feature has some potential security flaws. Beaumont, who briefly worked at Microsoft in 2020, tested Recall last week and found that the feature stores data in a database in plain text. This could make it trivial for an attacker to use malware to extract the database and its contents.

“Every few seconds, screenshots are taken. They are automatically OCRed by Azure AI, run on your device, and written to an SQLite database in the user folder,” he explains. Beaumont in a detailed blog post. “This database file records everything you’ve ever seen on your PC in plain text.”

Beaumont shared an example of a plain-text database in X, chiding Microsoft for telling media outlets that a hacker can’t exfiltrate Recall activity remotely. The database is stored locally on a PC, but can be accessed in the AppData folder if you are a PC administrator. Two Microsoft engineers demonstrated this in Build recently, and Beaumont claims that the database is accessible even if you are not an administrator.

The fear is that Recall will make it easier for malware and attackers to steal information. InfoStealer trojans already exist to steal credentials and information from PCs, and hackers currently distribute this type of malware to steal and sell information. “Recall allows threat actors to automate the collection of everything you’ve ever seen in seconds,” says Beaumont.

Beaumont exfiltrated his own Recall database and created a website where you can upload a database and search it instantly. “I’m deliberately withholding the technical details until Microsoft releases the feature because I want to give them time to do something,” he says.

Microsoft is planning to enable Recall by default on Copilot Plus PCs. In my own testing on a pre-release version of Recall, the feature is enabled by default when you set up a new Copilot Plus PC, and there is no option to disable it during the setup process unless you check an option that open the Settings panel. . Microsoft is supposedly discussing whether we should change this configuration process.

The reaction to Microsoft’s recall announcement was swift, with privacy advocates calling it a potential “privacy nightmare” and the UK Information Commissioner’s Office stepping in to ask Microsoft questions about its use of the AI-powered feature.

Microsoft says that Recall is an optional experience and that it has built privacy controls into the feature. You can disable certain URLs and applications, and Recall will not store any material protected by digital rights management tools. “Recall also does not take snapshots of certain types of content, including InPrivate web browsing sessions in Microsoft Edge, Firefox, Opera, Google Chrome, or other Chromium-based browsers,” Microsoft says in its statement. FAQ explanatory page.

However, Recall does not perform content moderation, so it does not hide information such as passwords or financial account numbers in its screenshots. “This data may be in snapshots stored on your device, especially when sites do not follow standard Internet protocols such as password entry hiding,” warns Microsoft.

Microsoft’s FAQ page does not address the potential for malware to attempt to steal the Recall database. “Recall snapshots are kept on the Copilot Plus PCs themselves, on the local hard drive, and are protected using data encryption on your device and (if you have Windows 11 Pro or a Windows 11 enterprise SKU) BitLocker,” says the Microsoft.

As Beaumont points out, disk encryption is only good for certain scenarios. “When you are connected to a PC and run the software, things are decrypted for you,” explains Beaumont. “Encryption at rest only helps if someone comes to your house and physically steals your laptop – that’s not what criminal hackers do.”

Microsoft may very well need to rework Recall, or recall it, if you will. There are clearly some obvious holes in the way data is stored here that need to be addressed, and making this an opt-out experience worries privacy advocates. Recall’s launch comes just weeks after Microsoft CEO Satya Nadella called on employees to make security Microsoft’s “top priority,” even if that means prioritizing it over new features.

“If you are faced with the trade-off between security and another priority, your answer is clear: Be safe,” Nadella said (his emphasis) in an internal memo obtained by On the edge. “In some cases, this will mean prioritizing security above other things we do, like releasing new features or providing ongoing support for legacy systems.”

On the edge reached out to Microsoft for comment about Recall’s security and privacy concerns, but the company did not respond in time for publication.