Struggling readers and writers have had an emerging set of tools that have progressed in the past few years. Voice to text helps struggling writers and even those that would rather just speak than type to create accurate text from speaking in real time. Text to voice tools read different texts from the internet, personal writing or even from a picture of typed text. These tools originally had a reputation for slow generating times and computer-generated voices, but they have begun to sound more natural in recent months.
Seeing AI goes one step further by eliminating many of the waiting times and moving beyond the constraints of just text-based reading.
Seeing AI allows you to point your camera at a wide range of items and almost instantaneously provides an audio description of the item. Even abstract items, like my wife laying on the couch were resolved within about a second.
There was, however, some regression in the voices with a loss in intonation and flowing words that have progressed in other tools.
Microsoft’s angle for Seeing AI is primarily a tool for people with visual needs. In education, this could help a much broader range of students such as those already using reading and writing tools and students who require SEL support. The almost instantaneous response will eliminate a barrier of framing, scanning, and selecting the right tool for processing. The ability to read faces, scenes, money, barcodes and more could have future possibilities for school attendance, security, supervision, and resources.
It could also have applications for inquiry and for content learning in education. Students can wonder “what is that” and get instant information to move further along in their explorations.
Seeing AI was part of a Microsoft Garage project. Many of these projects become part of other applications. Dictate, Microsoft’s version of speech to text was a garage project that led to voice typing with context so that pronunciation was less of an issue and typing became more accurate. Now Dictate is a standard feature in many of their tools.
See all the tutorials and try it for yourself.
Where would you see yourself using Seeing AI?
Hi Ryan,
I enjoyed reading this, well-articulated. Thank you.
Seeing AI is amazing, in particular for sight-impaired people. SAI enables millions of people – to varying degrees – to learn about their environment. If I recall correctly, this project used to be called Deep Vision, not sure what triggered the name change, but I like Seeing AI better anyways. To be honest, I feel like people with disabilities to some extents have been overlooked when it comes to technology development, in particular when talking about mobile technologies. Seeing AI or similar projects certainly require advances across multiple domains, including software, hardware, systems, and learning algorithms improvements, but I am quite happy to see that work is being done in this field.
Saeid