To study a concept, LEVAN scans millions of books and images online. It learns as many visual variations as it can. For example, when learning about the concept “horse” the algorithm would keep phrases like “jumping horse” or “eating horse” but discard non-visual phrases like “my horse” and “last horse.”
“It is all about discovering associations between textual and visual data,” said Ali Farhadi, a UW assistant professor of computer science and engineering. “The program learns to tightly couple rich sets of phrases with pixels in images. This means that it can recognize instances of specific concepts when it sees them.”
The software works great for concrete concepts like airplane or chair, less well for vaguer concepts like innovation (which turns up a large number of images of people talking, often in suits). The list of known concepts is finite but growing. Since its March launch, LEVAN has tagged over 13 million images with 65,000 phrases.
“Major information resources such as dictionaries and encyclopedias are moving toward the direction of showing users visual information because it is easier to comprehend and much faster to browse through concepts,” said Santosh Divvala, a research scientist at the Allen Institute and affiliate scientist at UW in computer science and engineering.
However, Divvala goes on to point out the approach is limited by the need for human workers to sort images. The best part about LEVAN is that once on the scent of a concept, it works the rest out all by itself. Though it currently takes awhile to finish its learning—12 hours of compute time per concept—that’s a vast improvement on manual human curation.
Learn more at Science Daily, “New computer program aims to teach itself everything about any visual concept,” or try out LEVAN for yourself here.
This entry was posted in AI and tagged ai, Ali Farhadi, allen institute for artificial intelligence,Learning Everything About Anything, LEVAN,natural language processing, Santosh Divvala,University of Washington.