Google Gemini
Tested gemini.google.com/app via Chrome browser – checked version and it says ‘Gemini family of models, including 1.0 Pro’ – March 2024
Answer to Question 1
Screenshot of Google Gemini answer to Question 1 – When was the Battle of Hastings? – showing the way it correctly answered the question, though a bit sparsely with only two sentences.
Accuracy Issues
I deducted quite a few points off Google Gemini for accuracy issues, as for some questions it doesn’t even attempt an answer, such as Question 2 – Who was the Chancellor during Tony Blair’s goverment? where it answers with ‘I’m still learning how to answer this question. In the meantime, try Google Search.’ as seen in the screenshot. It also offered the same for the follow up question – When did Gordon Brown become Prime Minister?
It also struggled (the same as the others) with follow up question for Question 6 – How many times has Francis Ford Coppola won the Best Director Oscar? answering twice, as the others do, when he has won it once.
There was also confusion over the answer to Question 7 – Name an actor who has been nominated for most Oscars without winning one? as it does correctly state that Peter O’Toole and Glenn Close were both nominated for 8 Oscars, but then lists the films out and the lists only have seven films and some are totally wrong. For example, Glenn Close was not even in The Shawshank Redemption?!?
Google Gemini scores were as follows:
- Ease of Use / Usability / UX / UI – Score = 8
- Easy to use as just type in questions and view the answers. Just a bit limited as text only and often not much text in the answer. I’d class them as sparse.
- Accuracy – Score = 5
- Quite a few points off here, as couldn’t even answer some of the questions, and for some of those it could answer, there were accuracy issues.
- Response time – Score = 9
- Some responses are a bit slow, so 1 point off for response time
- Error handling – Score = 10
- Worked well with garbled questions and nonsense questions and garbage data
- Source quotation/Provenance – Score = 0
- No sources given at all, so have to score it 0
- Working with the response e.g. further options – Score = 6
- Doesn’t provide that much further to work with, apart from Share and Copy, so a fairly low mark here.
- Overall rating and score – Overall Score = 6
- Where it can provide a response, some good responses, although some are a bit sparse. Worst thing was that couldn’t answer some questions the other chatbots did. Accuracy issues also, plus some answers were a bit slow to be provided.
- Ranking = 4
- Overview – Average performance, some questions it couldn’t answer, some it got accuracy errors for, and sometimes slow to provide answers, so 4th place of 4.
More info on Google Gemini – main website
According to the Wikipedia page on Google Gemini, it was formerly known as Bard and then in February 2024 unified with Duet AI, under the Gemini brand.
All the posts
I have created a post for each AI Chatbot test in order to fully explore the test results, along with a Results post, where I crown the winning AI Chatbot. You can jump straight there or read each test post via the links below.
– Testing the AI Chatbots: Questions
– Testing the AI Chatbots: Perplexity AI
– Testing the AI Chatbots: OpenAI ChatGPT
– Testing the AI Chatbots: Microsoft Copilot
– Testing the AI Chatbots: Google Gemini
– Testing the AI Chatbots: Results
Note: The heading images used in these posts were created via Bing Image Creator
About my testing services: iOS App Testing / Android App Testing / Website Testing / AI Chatbot Testing