Microsoft’s Copilot AI now understands your terrible handwriting

Microsoft is preparing to allow Copilot in OneNote to read and analyze handwritten notes. The feature has entered beta testing at the end of last monthand will allow OneNote users to write handwritten notes using a pen and then summarize them, ask questions, or even generate to-do lists based on the notes.

The AI-powered Copilot feature in OneNote will even be able to convert your handwritten notes to text for easy editing and sharing. Microsoft first released Copilot within OneNote in November, and this update will be available to existing Copilot for Microsoft 365 subscribers and Copilot Pro users once it is rolled out more broadly.

I’ve been briefly testing Copilot’s ability to read my handwriting, and I’m impressed that it’s able to decipher it. Summary works well on small handwritten notes and even larger notes. I asked Copilot to rewrite an entire paragraph of handwritten notes that he was able to format into easy-to-read text that was faithful to the original but also a little more breezy. This is impressive, as generative AI models tend to invent things half the time.

If you use OneNote for handwritten to-do lists, this feature definitely makes it easier to convert them to text later. I created a handwritten list and Copilot was able to accurately convert it to a text list in seconds.

I’m not sure how Copilot will handle even worse handwriting. Samsung’s Galaxy AI has a similar feature that can automatically format handwritten lists, but it struggles with some of the worst handwriting I’ve seen it (sorry, Allison!).

If you want to test Copilot’s handwritten note recognition, you’ll need to be a Microsoft 365 Insider running the latest version of OneNote on Windows (17628.20006 or later) and have a subscription to Copilot Pro or Copilot for Microsoft 365.