Nibsy makes manual project documentation obsolete

Documenting a project is just as much work as the project itself. Capturing photos and video, listing parts, and describing each step takes a lot of time — which is why so many people don’t bother. But that documentation is critical if you want to share your work, which is why Kevin McAleer had the genius idea to create a “self-documenting workshop” called Nibsy.

While the details of a project vary dramatically, write-ups, and tutorials tend to follow pretty standard formulas. That includes a list of components, a description of each step, and visuals. Because there is a consistent workflow, it is possible to automate the documentation process. AI makes that automation much more effective, as it can make sense of everything and wrap it up into a presentable package.

McAleer used the dual-brain Arduino® UNO Q to pull off that automation. The STM32 microcontroller detects the start of work through either a button press or sensing current on McAleer’s soldering iron. That tells the Qualcomm Dragonwing QRB2210 processor brain to start recording the environment.

It records two things: audio as McAleer speaks and explains what he’s doing, plus images from an overhead webcam that points down at the desk. McAleer’s spoken explanations give the AI context, including critical information like the names of components and information about the connections he’s making. The explanations, plus image analysis, let the AI determine which image frames to keep for the documentation. All the while, the UNO Q’s onboard LED matrix displays icons representing the system’s status.

The AI in question is Claude’s Sonnet model, which offers a good balance of performance, cost, and capability. That does most of the work of sifting through the recorded data. Then, at the end of a session, Claude’s top-of-the-line Opus models refines that and distills it down into a project log or tutorial, ready to publish to McAleer’s blog.

The post Nibsy makes manual project documentation obsolete appeared first on Arduino Blog.