# Assessing AI Chatbots Performance in Comprehensive Standardized Test Preparation; A Case Study with GRE

**Authors:** Mohammad Abu-Haifa, Bara'a Etawi, Huthaifa Alkhatatbeh, Ayman Ababneh  
**arXiv:** [2312.03719](https://arxiv.org/abs/2312.03719)  
**Date:** 2023-11-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.03719v1/

## Key points

- Research paper evaluates performance of three AI chatbots: Bing, ChatGPT, and GPT-4
- Focuses on their abilities in addressing standardized test questions using GRE as a case study
- Administered 137 quantitative reasoning questions and 157 verbal questions
- Questions categorized into varying levels of difficulty
- Evaluated chatbots' proficiency in addressing image-based questions and measured uncertainty level
- Results show varying degrees of success across chatbots, influenced by model sophistication and training data
- GPT-4 emerged as most proficient, particularly in complex language understanding tasks
- Demonstrates evolution of AI in language comprehension and ability to pass exams with high scores
- Explores chatbots' abilities in assessing verbal reasoning, quantitative reasoning, critical thinking, and analytical writing skills
- Provides valuable insights into utilization of AI in standardized test preparation.

## Summary

This research paper presents a comprehensive evaluation of the performance of three artificial intelligence chatbots: Bing, ChatGPT, and GPT-4. The study focuses on their abilities in addressing standardized test questions and uses the Graduate Record Examination (GRE) as a case study. This exam encompasses both quantitative reasoning and verbal skills and is commonly used by universities to evaluate applicants to their graduate programs. A total of 137 quantitative reasoning questions and 157 verbal questions were administered to assess the chatbots' capabilities. These questions were categorized into varying levels of difficulty. The study also evaluated the chatbots' proficiency in addressing image-based questions and measured their uncertainty level. The results revealed varying degrees of success across the chatbots, highlighting the influence of model sophistication and training data. GPT-4 emerged as the most proficient, particularly in complex language understanding tasks. This demonstrates the evolution of artificial intelligence in language comprehension and its ability to pass exams with high scores. In addition to evaluating the chatbots' performance on standardized test questions, this paper explores their abilities in assessing verbal reasoning, quantitative reasoning, critical thinking, and analytical writing skills – all crucial factors for graduate school applicants. Overall, this research provides valuable insights into the utilization of artificial intelligence in standardized test preparation.