Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? An Examination on Several Typical Tasks

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Proche Pas pertinent

   FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets

   Neng Wang, Hongyang Yang, Christina Dan Wang · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   In the swiftly expanding domain of Natural Language Processing (NLP), the potential of GPT-based models for the financial sector is increasingly evident. However, the integration of these models with financial datasets presents challenges, notably in determining their adeptness and relevance. This paper introduces a distinctive approach anchored in the Instruction Tuning paradigm for open-source large language models, specifically adapted for financial contexts. Through this methodology, we capitalize on the interoperability of open-source models, ensuring a seamless and transparent integration. We begin by explaining the Instruction Tuning paradigm, highlighting its effectiveness for immedi…

2. Proche Pas pertinent

   FinGPT: Democratizing Internet-scale Data for Financial Large Language Models

   Xiao-Yang Liu, Guoxuan Wang, Daochen Zha · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Large language models (LLMs) have demonstrated remarkable proficiency in understanding and generating human-like texts, which may potentially revolutionize the finance industry. However, existing LLMs often fall short in the financial field, which is mainly attributed to the disparities between general text data and financial text data. Unfortunately, there is only a limited number of financial text datasets available (quite small size), and BloombergGPT, the first financial LLM (FinLLM), is close-sourced (only the training logs were released). In light of this, we aim to democratize Internet-scale financial data for LLMs, which is an open challenge due to diverse data sources, low signal-to…

3. Proche Pas pertinent

   A Categorical Archive of ChatGPT Failures

   Ali Borji · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Large language models have been demonstrated to be valuable in different fields. ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation by comprehending context and generating appropriate responses. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries, with fluent and comprehensive answers surpassing prior public chatbots in both security and usefulness. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study. Ten categories of failures, including reasoning, factual errors, math, coding, and bias, are presented and discussed. The risks…

4. Proche Pas pertinent

   Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors

   Ridong Han, Tao Peng, Chaohao Yang et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   ChatGPT has stimulated the research boom in the field of large language models. In this paper, we assess the capabilities of ChatGPT from four perspectives including Performance, Evaluation Criteria, Robustness and Error Types. Specifically, we first evaluate ChatGPT's performance on 17 datasets with 14 IE sub-tasks under the zero-shot, few-shot and chain-of-thought scenarios, and find a huge performance gap between ChatGPT and SOTA results. Next, we rethink this gap and propose a soft-matching strategy for evaluation to more accurately reflect ChatGPT's performance. Then, we analyze the robustness of ChatGPT on 14 IE sub-tasks, and find that: 1) ChatGPT rarely outputs invalid responses; 2) …

5. Proche Pas pertinent

   ChatGPT-Crawler: Find out if ChatGPT really knows what it's talking about

   Aman Rangapur, Haoran Wang · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Large language models have gained considerable interest for their impressive performance on various tasks. Among these models, ChatGPT developed by OpenAI has become extremely popular among early adopters who even regard it as a disruptive technology in many fields like customer service, education, healthcare, and finance. It is essential to comprehend the opinions of these initial users as it can provide valuable insights into the potential strengths, weaknesses, and success or failure of the technology in different areas. This research examines the responses generated by ChatGPT from different Conversational QA corpora. The study employed BERT similarity scores to compare these responses w…

6. Proche Pas pertinent

   Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks

   Agam Shah, Sudheer Chava · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Recently large language models (LLMs) like ChatGPT have shown impressive performance on many natural language processing tasks with zero-shot. In this paper, we investigate the effectiveness of zero-shot LLMs in the financial domain. We compare the performance of ChatGPT along with some open-source generative LLMs in zero-shot mode with RoBERTa fine-tuned on annotated data. We address three inter-related research questions on data annotation, performance gaps, and the feasibility of employing generative models in the finance domain. Our findings demonstrate that ChatGPT performs well even without labeled data but fine-tuned models generally outperform it. Our research also highlights how ann…

7. Proche Pas pertinent

   Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study

   Zengzhi Wang, Qiming Xie, Yi Feng et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Recently, ChatGPT has drawn great attention from both the research community and the public. We are particularly interested in whether it can serve as a universal sentiment analyzer. To this end, in this work, we provide a preliminary evaluation of ChatGPT on the understanding of \emph{opinions}, \emph{sentiments}, and \emph{emotions} contained in the text. Specifically, we evaluate it in three settings, including \emph{standard} evaluation, \emph{polarity shift} evaluation and \emph{open-domain} evaluation. We conduct an evaluation on 7 representative sentiment analysis tasks covering 17 benchmark datasets and compare ChatGPT with fine-tuned BERT and corresponding state-of-the-art (SOTA) mo…

8. Proche Pas pertinent

   Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions

   Chen Feng Tsai, Xiaochen Zhou, Sierra S. Liu et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Large language models (LLMs) such as ChatGPT and GPT-4 have recently demonstrated their remarkable abilities of communicating with human users. In this technical report, we take an initiative to investigate their capacities of playing text games, in which a player has to understand the environment and respond to situations by having dialogues with the game world. Our experiments show that ChatGPT performs competitively compared to all the existing systems but still exhibits a low level of intelligence. Precisely, ChatGPT can not construct the world model by playing the game or even reading the game manual; it may fail to leverage the world knowledge that it already has; it cannot infer the g…