The information wars are heating up with search startup ChatGPT, a direct competitor to Google that promises a more “natural, conversational” experience over traditional search engines.
On top of the usual AI-powered text summaries, ChatGPT search links to the most direct sources and provides a short list of articles for further browsing. It’s currently available to those with a $20/month Plus or business account, and will be rolling it out to free users in the coming weeks.
ChatGPT research on the meaning of cat sounds, with a video and resources on the side. (Credit: ChatGPT)
While this may seem like ChatGPT is Google growing up, it mirrors what Google launched in May with AI Briefings. Most Google searches now display an AI-powered summary, with links to a few top sources, followed by the traditional blue link list. Google also has an advantage with the simple fact that most people are already using Google all day, every day across devices.
As the two tools are becoming more and more similar, the question arises: When is it better to use Google and when is ChatGPT more useful? We tested both, and here’s what we found.
Research test #1: How to cook scallops?
Winner: ChatGPT
We started our test with a simple internet search to help with dinner. This is one area where an AI overview can help avoid the cumbersome experience of browsing the web for recipes today.
When I asked Google how to cook scallops, it was linked to several recipes. I knew that reading any of them would involve wading through a barrage of pop-up ads and paragraphs of filler content before getting to the actual instructions. All I wanted was a basic technique for a casual dinner, so this wasn’t ideal.
(Credit: Google)
ChatGPT gave me more of what I was looking for: A basic list of ingredients and general techniques. He cited Once Upon a Chef and Martha Stewart, removing the need to scroll past ads or unnecessary information. This was great for my casual use case, but if I was looking for a more elaborate dish, it probably wouldn’t satisfy. The recipes on Google looked more delicious, with creamy garlic sauces and suggestions for side dishes.
(Credit: ChatGPT)
It’s not an easy win for ChatGPT, especially because its output is almost identical to another search product in the game: Google Gemini. This chatbot-style ChatGPT competitor lives on a separate website. Her answer cited The Kitchn and All Recipes, but gave similar answers.
(Credit: Google Gemini)
For the rest of our testing, we’ll leave Gemini aside and focus on ChatGPT and Google’s flagship search product. ChatGPT is by far the most used AI chatbot and Google dominates global search volume, making them the most compelling comparison.
Research Quiz #2: What Does Early Voting Data Say?
Winner: Tied, both offer a unique experience
More complex entities provided a more interesting comparison between ChatGPT and Google. I asked both of them about early voting data (before the election, of course).
Google did not attempt an AI overview, perhaps for fear of inaccuracies. He linked to reputable publications that provided a wealth of additional information. For example, NBC News featured a helpful infographic, plus an overview of the situation and a list of real-time election updates.
(Credit: Google)
(Credit: NBC)
ChatGPT’s response provided a dry, point-by-point summary of the key discussions in the press. He scored points by posing my follow-up question since AI Summaries don’t allow for continuations. ChatGPT’s initial response and subsequent response were helpful, curated summaries, but not ideal for someone wanting to explore the wealth of information out there.
Quoted ChatGPT The Wall Street JournalAssociated Press, and New York Postall of which have content licensing agreements with OpenAI. This begs the question: Isn’t additional information included there from those agreements and therefore not included in the answer?
ChatGPT’s first answer, left and middle, and the follow-up question, right. (Credit: ChatGPT)
Research Quiz #3: What are easy songs to play on guitar?
Winner: Google
AI chatbots are considered a complement to tutors and teachers, much like YouTube was back then, especially because students can ask questions and learn quickly. Did research bring ChatGPT closer to reality? Potentially, but not in my test.
I searched for easy songs to play on the guitar, but the result did not help me play a song. From the list that came up, below, I searched for the chords of George Ezra’s “Budapest.”
(Credit: ChatGPT)
He suggested using a more complex chord than is appropriate for a beginner (B minor). It also did not adjust the chords of the music, which is required to be played. When I asked it to do this, the confused interface interpreted the music as “code”. At the end, it linked to a video of a guitarist showing how to play it.
This arrangement from my favorite site, Ultimate Guitar Chords, uses three of the easiest chords (G, C, D). It shows you where to place your fingers, provides a pattern of the strum, and a video to guide you through it. Ultimate Guitar, accessed through Google, would have been more useful from the start than using ChatGPT.
(Credit: ChatGPT)
Research Quiz #4: Why is NJ so warm this year?
Winner: Google
This fall, New Jersey has seen many uncharacteristically warm days in the 70s and 80s, culminating in the hottest Halloween ever. What gives?
Recommended by our Editors
Google’s AI summary attributes the record temperatures to global warming, rising temperatures in the boundary ocean and the heat island effect in cities. I also learned that the state is warming faster than others nearby. (Not good!)
(Credit: Google)
ChatGPT’s first reply was incorrect, citing “high pressure systems”, which he removed from an article on Asbury Park Press for the lack of rain. He quotes David Robinson, state climatologist at Rutgers University: “We owe this drought to an unusually persistent ridge of high pressure over the region through late August, which has produced clear skies and dry air.”
(Credit: ChatGPT)
Research Quiz #5: Where to Watch the Tesla Robotaxi Reveal?
Winner: Google
We know ChatGPT favors quick summaries over lengthy explanations, which is sometimes a matter of preference, but in this case it prevented me from surfacing the result I wanted at all.
On October 10, Elon Musk hosted a spectacular unveiling party in Hollywood for Tesla’s upcoming robotaxi lineup. I asked ChatGPT where I could watch the event and it suggested a 7-minute recap video. This is potentially useful as the full video of the event ran for over an hour, but it wasn’t what I was looking for.
(Credit: ChatGPT)
In a follow-up question, I asked ChatGPT to link to the original video and it failed. Google displayed the original video first in its results. This reinforced to me that Google will remain the primary choice for those looking to get their information from the original source.
(Credit: ChatGPT)
Final score: 3 Google, 1 equal, 1 ChatGPT
ChatGPT search is a big step forward for OpenAI, especially with how it connects to resources. It’s very similar to Google, where both offer an overview of AI along with a list of links, but for now I expect most people to stick with Google.
In some scenarios, ChatGPT search can be a great option, especially for a quick summary of an issue without ads and without serious need to dig deeper. In most cases, Google will likely suffice, offering a similar AI-powered summary of information, plus more resources to dig further and take ownership of the information you consume.
OpenAI must also build trust with consumers; Google has extremely high brand recognition. However, with a few more product releases, ChatGPT search may be on the way.
OpenAI unveils its AI voice assistant ChatGPT
Get our best stories!
Register for What’s new now? to get our top stories delivered to your inbox every morning.
This newsletter may contain advertisements, deals or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You can unsubscribe from newsletters at any time.
About Emily Dreibelbis Forlini
Senior reporter
