LLMs Like ChatGPT Can Perform Data Analysis At A Fraction Of

A recent study conducted by researchers from Damo Academy (Alibaba Group Holding’s research arm) and Singapore’s Nanyang Technological University highlights the potential threat to job security posed by the increasing adoption of generative artificial intelligence (AI) in various industries.

The study reveals that using large language models (LLMs) in data analysis yields comparable performances to human analysts. It compares the cost of employing GPT-4, the latest version of OpenAI’s LLM, with that of hiring human data analysts. The findings indicate that GPT-4 costs only 0.45% of hiring a senior data analyst with an annual salary of approximately $90,000, or 0.71% of hiring a junior-level employee.

The researchers conducted experiments where they provided questions and data to both GPT-4 and human analysts. GPT-4 automated the entire data analytics process, extracting and analyzing data to generate insights and graphs.

The results of the analysis were then compared in terms of performance, time, and costs. The study demonstrates that GPT-4 is significantly cheaper than human analysts and completes tasks much faster. The AI model outperforms entry-level human analysts in terms of performance, evaluated based on metrics such as chart correctness, fluency, and insights produced.

Additionally, GPT-4 exhibits “comparable performance” to senior-level analysts, although the advantages vary depending on the case and metrics used.

While GPT-4 surpasses human analysts in some aspects, such as the correctness and complexity of generated insights, it falls behind in correctly presenting data in graphs and formatting. Nevertheless, the study emphasizes that GPT-4 can still generate accurate analyses despite errors in some figures. These findings underscore the expanding application of LLMs in different industries, which promise improved efficiency but also pose a threat to human jobs.