A good car mechanic is hard to find. Could AI help me locate one? It was worth a shot.
First, a note about the tool I used: There are a few differences between the publicly available free ChatGPT and the $20/month subscription service, ChatGPT Plus. In addition to ChatGPT Plus being based on the GPT-4 model, I find the biggest differences to be the availability of Code Interpreter (which offers a lot of neat analytics) and ChatGPT plugins.
Also: The 10 best ChatGPT plugins right now
The plugins are something of a game-changer because they enable — among many other things — ChatGPT to do its magic using current data available on the web. Sans plugins, everything ChatGPT knows ends in 2021. For this project, I used the WebPilot plugin.
About the car: Our 13-year-old Ford has reached the point where it needs a lot of work. It’s a heck of a car, and we hope to have it for many more years, which means that every so often it needs some tending to.
We didn’t want to take it to the dealer but preferred to find a mechanic locally, here in town. I used ChatGPT Plus to help identify the person to whom we eventually brought our car. It turns out he’s a guy who does not own a cell phone and refuses to use email. (So he’s clearly a much happier and smarter person than you or I will ever be.)
To protect his identity, as well as to repeat the experiment, I decided to run the same queries using Melbourne, Florida, a town on the other side of the country, as an example.
Also: 7 advanced ChatGPT prompt-writing tips you need to know
The plan was simple: get ChatGPT to analyze Yelp and Google reviews to find out which mechanic local residents seem to like best.
With ChatGPT Plus, you have to enable plugins at the start of a session. So, to start, I enabled WebPilot.
One quick note: Previously, I tried to get ChatGPT to scan both Yelp and Google reviews at the same time and the AI always got confused. So, for this demonstration, I’m separating those queries. (And it still gets a little lost.)
Also: How this simple ChatGPT prompt tweak can refine your AI-generated content
Here was my first prompt:
Use WebPilot to scan Yelp reviews for car repair shops local to Melbourne, Florida. Aggregate the review results and let me know which repair shops, based on the average review value and the number of reviews, produce the highest level of customer satisfaction. Do not include any car dealers or repair shops that are part of car dealers, and be sure to stay solely within Melbourne.
I got back a fairly useful response:
Then I told it:
Do the same for Google reviews.
As you can see, it got lost, confusing LA Fitness and Home Depot with auto mechanics. I then tried to get it to re-run the search, but I explicitly told it to substitute Google for Yelp:
Do the same, but use Google reviews instead of Yelp reviews
This time it just failed. Rather than looking up results, it told me how to search for them myself. Sometimes when the AI gets confused, a more comprehensive prompt can help. So I fed it the original prompt I used above, but with “Google” in place of “Yelp”:
Use Webpilot to scan Google reviews for car repair shops local to Melbourne, Florida. Aggregate the review results and let me know which repair shops, based on the average review value and the number of revews, produce the highest level of customer satisfaction. Do not include any car dealers or repair shops that are part of car dealers, and be sure to stay solely within Melbourne.
As you can see, that worked. Next, I asked it to do a sentiment analysis:
Now perform a sentiment analysis on both sets of shops based on their Yelp and Google reviews and rank them in terms of how customers rate them, along with a short descriptive summary that reflects overall customer satisfaction or lack of satisfaction. Present the shops in order of their overall ranking, with the most positive sentiment as the first shop presented.
This did not result in an aggregated ranking. I decided again to add more specificity to my prompt. Here’s the prompt that yielded the best results:
Now perform a sentiment analysis on both sets of shops based on their COMBINED Yelp and Google reviews (REVIEW COUNT AND NUMBER OF STARS), AS WELL AS ALL THE CUSTOMER COMMENTS, and rank them in terms of how customers rate them OVERALL, along with a short descriptive summary that reflects overall customer satisfaction or lack of satisfaction.
Also: We’re not ready for the impact of generative AI on elections
The all-caps words are words I added that gave the prompt more clarity. They’re upper-cased for you to see my changes (ChatGPT doesn’t pay attention to case). I also modified this sentence as shown in all-caps:
Present the top five shops in order of their overall ranking ACROSS BOTH YELP AND GOOGLE, with the most positive sentiment as the first shop presented.
The results were much more in line with what I wanted. First, it presented a combined detailed report:
Then, as part of the same answer, it presented an overall ranking.
When I did this here, locally, the repair shop recommended by ChatGPT Plus is the one we took the Ford to. So far, it’s been a very satisfying experience. Among other things, something another shop quoted as costing more than $2,000 to fix wound up costing about $250 with the mechanic recommended by AI.
Also: These are my 5 favorite AI tools for work
I took one final step, which I hadn’t tried before. As a college professor, I often told my students to show their work, to help me see how they got to their conclusions. I did the same with ChatGPT Plus. It was interesting.
Show your work
Overall, I’m pretty intrigued by the possibility of letting the AI aggregate public sentiment data. But it’s important to be careful because this data is hard to confirm. And, as we’ve seen, the AI regularly loses the thread, so you really never know how reliable its conclusions will be.
Also: Everyone wants responsible AI, but few people are doing anything about it
As an experiment, though, it was fun. And, fortunately for our over-the-hill Ford, it’s proving to be a good result.
You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.
Artificial Intelligence
