Résultats de recherche pour « Large Language Model »

Large language models (LLMs) have achieved success in acting as agents, which interact with environments through tools such as search engines. However, LLMs are optimized for language generation instead of tool use during training or alignment, limiting their effectiveness as agents. To resolve this problem, previous work has first collected interaction trajectories between LLMs and environments, using only trajectories that successfully finished the task to fine-tune smaller models, making fine-tuning data scarce and acquiring it both difficult and costly. Discarding failed trajectories also leads to significant wastage of data and resources and limits the possible optimization paths during fine-tuning. In this paper, we argue that unsuccessful trajectories offer valuable insights, and LLMs can learn from these trajectories through appropriate quality control and fine-tuning strategies. By simply adding a prefix or suffix that tells the model whether to generate a successful trajectory during training, we improve model performance by a large margin on mathematical reasoning, multi-hop question answering, and strategic question answering tasks. We further analyze the inference results and find that our method provides a better trade-off between valuable information and errors in unsuccessful trajectories. To our knowledge, we are the first to demonstrate the value of negative trajectories and their application in agent-tunning scenarios. Our findings offer guidance for developing better agent-tuning methods and low-resource data usage techniques.

Introdução: A utilização de dados de saúde em pesquisas é limitada por questões éticas. Isso desafia os pesquisadores a encontrarem formas de obter o material necessário para desenvolverem seu trabalho. Método: Usou-se uma ferramenta de Large Language Model (LLM) para gerar prontuários eletrônicos (PE) sintéticos de pacientes cardiológicos utilizando-se as técnicas "few-shot prompting" e "chain-of-thought prompting". Objetivo: criar um conjunto de dados abrangente e acessível para auxiliar no treinamento de algoritmos de classificação de texto em cenários médicos. Resultados: Foram gerados 103 PE sintéticos, abrangendo diagnósticos cardíacos distintos. Conclusão: A geração de PE sintéticos através de LLM apresentaram qualidade esperada, sendo condizentes com o conteúdo encontrado em PE reais. O conjunto de dados está disponível no repositório Zenodo para uso irrestrito pela comunidade de pesquisa, seguindo o conceito de ciência aberta.

Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datasets for LLM fine-tuning is still poorly understood. This study categorizes instructions into three primary types: NLP downstream tasks, coding, and general chat. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively impact other areas. This work provides insights into instruction mixtures, laying the foundations for future research.

Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Human evaluations on a complexity-balanced test bed and Vicuna's testset show that instructions from Evol-Instruct are superior to human-created ones. By analyzing the human evaluation results of the high complexity part, we demonstrate that outputs from our WizardLM are preferred to outputs from OpenAI ChatGPT. In GPT-4 automatic evaluation, WizardLM achieves more than 90\% capacity of ChatGPT on 17 out of 29 skills. Even though WizardLM still lags behind ChatGPT in some aspects, our findings suggest that fine-tuning with AI-evolved instructions is a promising direction for enhancing LLMs. Our code and data are public at https://github.com/nlpxucan/WizardLM

As ever larger language models grow more ubiquitous, it is crucial to consider their environmental impact. Characterised by extreme size and resource use, recent generations of models have been criticised for their voracious appetite for compute, and thus significant carbon footprint. Although reporting of carbon impact has grown more common in machine learning papers, this reporting is usually limited to compute resources used strictly for training. In this work, we propose a holistic assessment of the footprint of an extreme-scale language model, Noor. Noor is an ongoing project aiming to develop the largest multi-task Arabic language models -- with up to 13B parameters -- leveraging zero-shot generalisation to enable a wide range of downstream tasks via natural language instructions. We assess the total carbon bill of the entire project: starting with data collection and storage costs, including research and development budgets, pretraining costs, future serving estimates, and other exogenous costs necessary for this international cooperation. Notably, we find that inference costs and exogenous factors can have a significant impact on total budget. Finally, we discuss pathways to reduce the carbon footprint of extreme-scale models.

This paper explores network binarization, a radical form of quantization, compressing model weights to a single bit, specifically for Large Language Models (LLMs) compression. Due to previous binarization methods collapsing LLMs, we propose a novel approach, Partially-Binarized LLM (PB-LLM), which can achieve extreme low-bit quantization while maintaining the linguistic reasoning capacity of quantized LLMs. Specifically, our exploration first uncovers the ineffectiveness of naive applications of existing binarization algorithms and highlights the imperative role of salient weights in achieving low-bit quantization. Thus, PB-LLM filters a small ratio of salient weights during binarization, allocating them to higher-bit storage, i.e., partially-binarization. PB-LLM is extended to recover the capacities of quantized LMMs, by analyzing from the perspective of post-training quantization (PTQ) and quantization-aware training (QAT). Under PTQ, combining the concepts from GPTQ, we reconstruct the binarized weight matrix guided by the Hessian matrix and successfully recover the reasoning capacity of PB-LLM in low-bit. Under QAT, we freeze the salient weights during training, explore the derivation of optimal scaling factors crucial for minimizing the quantization error, and propose a scaling mechanism based on this derived scaling strategy for residual binarized weights. Those explorations and the developed methodologies significantly contribute to rejuvenating the performance of low-bit quantized LLMs and present substantial advancements in the field of network binarization for LLMs.The code is available at https://github.com/hahnyuan/BinaryLLM.

Large language models (LLMs) have become ubiquitous tools across numerous domains such as education, healthcare, and professional advisory services. However, their susceptibility to sycophancy-a bias toward echoing user prompts over independently reasoning-poses a critical threat to model reliability. We present a Bayesian-latent variable framework that introduces a hidden sycophancy score Si,m,u,p ∼ N (0, σ 2 S) to quantify such behavior. The framework defines a flip indicator ∆i,m,u,p ∈ {−1, 0, +1} to distinguish regressive (accuracydecreasing) from progressive (accuracy-increasing) shifts. Our generative logistic model ties correctness probabilities to latent agreement pressure via Pr(ŷ (1) = 1) = σ(x ⊤ β + γmSi,m,u,p), enabling rigorous posterior inference. Empirically, we find an average sycophancy-induced shift of E[|S|] ≈ 0.8σS, and a model susceptibility factor γ that directly controls response volatility. Simulated results show that nearly 50% of responses flip under user cues, with regressive flips dominating when γ < 0. Our approach provides a statistically grounded diagnostic for auditing and improving trustworthiness in LLM behavior under social influence. Simulation code for this study is available at: https://github.com/ParthaPRay/Sycophancy_in_LLM_model.