Um, none of those numbers add up.
Google is dumb because its AI is focused on your searches sooner or later. The company has already been test Over the course of about 11 months, this new Google, called Search Generative Experience, or SGE, started showing AI answers in Google’s top results with volunteers. even for untested people.
The new Google can do some useful things. But as you can see, he sometimes makes up facts, misinterprets questions, conveys outdated information, and generally lies. Worse, researchers find that AI often elevates low-quality sites as reliable sources of information.
Normally, I wouldn’t review an unfinished product. But this test of Google’s future has been going on for nearly a year, and the choices made now will affect how billions of people access information. Also at stake is the basic idea behind the current AI frenzy: technology can replace the need to investigate things ourselves by simply giving us answers. If a company with Google’s money and computing power can’t make it work, who can?
SGE combines the search engine you know with the capabilities of a chatbot. In addition to traditional results, SGE writes direct responses to queries, interspersed with links to dig deeper.
SGE is a response to the reality that some people, including myself, have started turning to AI like ChatGPT for more complex questions or when we don’t want to read many different sites. Alonethe search optimization firm estimates that using SGE can shorten a user’s overall research journey by 10 to 20 times by aggregating pros and cons, pricing and other information in one place.
An omniscient reply bot sounds useful when we’re narrowing our focus. But Google has a lot to work with. We expect searches to be fast, but it takes a painful second or two to generate Google’s AI answers. Google must balance the internet’s already fragile economy, where its AI answers can steal traffic from publishers who do the expensive and difficult work of doing actual research.
And most importantly, the new Google must deliver on its promise of being able to consistently and accurately answer our questions. This is where I focused my testing – and I continued to find instances where AI-powered Google was worse than its predecessor.
Testing Google’s AI answers
Most of the time, when you’re working on Google, what you really want is a summary or a link. On a day-to-day basis, the new Google is often annoying because its AI is so chatty.
A silly example: “What do Transformers eat?”
The AI’s response told me that while fictional robots need some kind of fuel, they don’t actually need to eat or drink. Meanwhile, my old Google search had a one-word answer: Energon. (It’s some kind of magic fuel.) You got this answer from the new Google just by scrolling down the page.
This doesn’t just happen with alien robots. SE Ranking, a firm dedicated to search engine optimization, tested With 100,000 keyword queries, SGE found that the average response it generated was 3,485 characters, or about a third of that column. One of Google’s challenges is figuring out when its AI is best to keep quiet; sometimes SGE asks you to click “generate” before writing a response.
We expect the right information when we’re looking for it the most. Google claims that SGE has a leg up on ChatGPT because its knowledge is up-to-date.
However, I found that the new Google is still struggling with the latest work. Three days after I recently searched for “Oscars 2024” in the Academy Awards. He told me the Oscars were still to come and listed some of the nominees.
And nothing has shaken my faith in Google’s AI answers more than watching it confidently create things.
It really includes facts about you. I asked him about an award-winning series I wrote for The Washington Post, and he attributed it to some stranger — then linked to another website.
Then there was the time when SGE was quite happy to make up information about something that didn’t even exist. I asked about a San Francisco restaurant called Danny’s Dan Dan Noodles, and he told me they had “crazy wait times” and described his food.
The thing is, it’s an imaginary store named after my favorite Chinese food. Google’s AI had no problem inventing information about it.
So-called hallucinations of real and fake subjects are a known problem with current artificial intelligence. The disclaimer above the SGE results says “General AI is experimental,” but that doesn’t address the issue. Google should know to say “I don’t know” when it’s not confident.
To give us all the answers, Google’s AI has to decide which sources are reliable. I’m not too sure about his verdict.
Remember our unfortunate take on Zuckerberg’s net worth? A professional researcher – and also plain old Google – might suggest checking the list of billionaires Forbes. Google’s AI response was based on something very strange ZipRecruiter Page for “Mark Zuckerberg Jobs”, something that doesn’t exist.
In my tests, questionable sources were the example. At Onely’s suggestion, I asked the new Google which was more reliable: Apple iPhones or Samsung phones. As a longtime reviewer, I can tell you there are many good sources of information on this, including professional journalists and repair organizations like iFixit.
Instead, the AI references random images of people pulled from social media. Aside from the limited usefulness of a single Reddit user’s experience, how does Google know this isn’t a fake review posted by a phone manufacturer?
“Google SGE plays by a different set of rules than the traditional search engine we know today,” said Tomek Rudzki, Onely’s head of research and development.
SEO firms try to conduct quantitative studies of the value of SGE, although they are limited by Google’s requirements for test accounts. But they found a similar pattern in the disconnect between old and new Google sites. SEO software company authority tested searches for a thousand shopping terms in late March and found that 77 percent of the time, the domain of the #1 traditional search result did not appear anywhere in the AI-written response.
And while researching 100,000 keyword searches, In SE Rating found that Q&A service Quora was the most linked source by SGE; LinkedIn and Reddit were fifth and sixth. How often can these sources be used in an eighth grade coursework?
Active searches about technology topics — including many “how-to” questions — SE Ranking found the most related domains simplilearn.com. I had never heard that before; the site describes itself as an “online boot camp.”
“This trend not only lowers the quality of search results, but also reduces traffic and revenue for many small businesses, including affiliate websites,” says Anastasiya Kotsiubinska, head of SEO at SE Ranking.
Google says that SGE has a connection experience. But Google has already surpassed it expected end last December and has offered no update on when he will come looking for everyone. It’s possible that Google doesn’t think SGE is accurate, fast, or profitable enough and will eventually change it drastically.
They’re wise to go slow, even if they make Google look like they’re lagging behind in the AI race. Bing, a rival search engine from Microsoft, underwent a similar AI overhaul in February 2023, but its AI is still notorious for going off the rails.
Google vice president Elizabeth Reid, who heads SGE, described it as a work in progress in an interview.
“We’re really focused on getting the experience right. There are a lot of different factors that go into it—things like latency, accuracy, helpfulness,” Reid said. trying to figure out where to draw the line.
When I shared examples in this column, Reid told me that SGE’s hallucination rates were “very low” and had dropped “significantly” since SGE’s launch in May, though he declined to be specific.
“I don’t want to minimize it — it’s a problem with technology,” and “we’re really working on it,” Reid said. He added that putting links next to AI answers is important to allow people to check the facts for themselves.
Here’s the suggestion: Since Google recognizes that getting facts right is a problem, it should disclose its data on accuracy before rolling out SGE to a wider audience. With billions of searches daily, even 0.001 percent can add a lot of wrong information.
Another area of focus for Google is “trying to help us get to the core of the question as quickly as possible and then provide additional details,” Reid said.
Citing low-quality sources, Google disputed outside research on SGE, saying it was based on more limited searches than what Google sees in practice. But he refused to share his information.
Reid said SGE is not held to a different standard than the old Google. “We are seeing a greater variety of sources emerging. But the goal is to continue to put really high-quality content at the highest level,” he said.
It is hard enough for people to choose who to believe. What makes Google take the current AI technology known as LLM or large language models up to the task?
“They’re not perfect,” Reid said. “We want to take this thoughtful approach because the brand of trust people have in Google is really important.”
The future of our information depends on it.