A New Tech Problem: AI-Powered Search Delivers Wrong Answers
What makes it worse is that companies are likely to heavily push the results.
Search is a basic need for any type of business. Major tech companies are pushing their own generative artificial intelligence to be a front for search. However, there are complications.
One is that the combinations of technologies don’t necessarily do what a business user needs. The Verge reported that Google recently gave a demo of its generative artificial intelligence included in its search systems, using video search.
They showed a brief video of a film camera that had a stuck film advance lever, then suggested ways of fixing the camera. The third suggestion was to open the back door of the camera and then remove the film if it’s jammed. Google highlighted that answer, offering it as a particularly good one.
Except that would be a terrible step as it would expose much of the film, likely destroying all the images on the roll. The real solution would be to go into a darkroom or use something called a changing bag to have access to the contents of the camera without exposing the contents to light.
This was during a product demo. Not the first time things have gone wrong in such circumstances, but usually when something goes wrong in a demonstration, it’s obvious to everyone, including those who are giving it. But people who haven’t used film cameras might not recognize that a mistake just took place. Apparently, this is the second time in a public demo that Google’s products made a big mistake
And that is a problem, because it has been an ongoing aspect of generative AI — hallucination. The software can put together associations of words that sound coherent but express things that are imaginary, which is exactly not what someone would want from a search engine, which is supposed to help someone find real answers to real questions.
That brings up the second complication, which is the marketing intentions of companies promoting their AI products, especially when some of those companies are heavily involved in search. AI-generated results are likely to get pushed to the top and, given typical patterns of how far into results people delve, what is at the top will be the most likely to be used.