Clearing up a robot’s fuzzy memory

MIT researchers have built a memory system that lets robots flag and fix their own mistaken descriptions of objects, according to MIT News. Called UQ-DAAAM, short for Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment, it extends the team's earlier DAAAM system, which uses a vision-language model to store detailed captions in a 3D map of a robot's surroundings. Poor lighting, blur or partial views can produce errors in that map, such as a gray chimney recorded as a squirrel.

The new system scores the model's confidence in its wording and checks that caption against descriptions of the same object from other angles, merging both into one uncertainty figure. Above a set threshold, a simple rule picks extra views, and a language model fuses the captions, favoring newer and less ambiguous ones. The researchers mathematically proved this rule can find the best frames without rerunning the vision model, sharply cutting computation so the process can run in real time on a moving robot.

MIT News reports that refinement resolved ambiguity for about 92 percent of objects using just two more captions each, and raised description accuracy by 5 percent over the state of the art. Answers about what objects were, where they were and when they were last seen also beat baseline methods.

Senior author Luca Carlone argues that physical AI needs more than a disclaimer about possible errors. The paper, co-led by graduate students Harry Zhang and Nicolas Gorlo, will be presented at the Conference on Neural Information Processing Systems. Backers include the U.S. Army Research Laboratory and the Office of Naval Research.