Résumé IA · arXiv: 2005.00052v1 · cs.CL

MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer

Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder
Article original · DOI 10.18653/v1/2020.emnlp-main.617 Soumis à arXiv le 30 Avr. 2020 Publié dans Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2020) Licence : NONEXCLUSIVE-DISTRIB 1.0 Aussi en : en

Comment lire ce résumé ? Guide illustré

Pour aller plus loin

  • Résumé Complet
  • Résumé vulgarisé
  • Article de blog
  • Abstract original
Le résumé n'est pas encore prêt

Cette synthèse vous a-t-elle été utile ?

Note : 0

Pourquoi avons-nous besoin de votes ?

Les votes sont utilisés pour déterminer si nous devons réexécuter nos outils de synthèse. Si le compte atteint -10, nos outils peuvent être redémarrés.

Figures et graphiques

Posez une question sur une figure en indiquant son numéro ou en utilisant son bouton de question pour demander une analyse visuelle. Au maximum deux figures et quatre panneaux par question. Les valeurs lues sur les graphiques sont approximatives ; vérifiez les données originales.

Instructions pour utiliser l'assistant IA ?

Outils pour cet article

Expliquer autrement 1 réécriture gratuite restante aujourd'hui
Enregistrer dans une collection Regroupez des articles en collections pour les comparer, les interroger et les exporter. Continuer avec Google Créer un compte gratuit Se connecter

Outils avancés Premium

Pour suivre un domaine de près : alertes de citations, diapositives de journal club, comparaison des versions et envoi vers Zotero. Découvrir Premium

Zotero Guide Zotero
Alertes de citation Plus de 300 articles le citent déjà : recevez chaque nouveau par email. Guide des alertes de citations
Qui travaille dessus ?

Parmi les 31 articles autour de celui-ci, le nom qui revient le plus : Alan Ansell (2 articles)

7 autres personnes en signent plusieurs. La carte complète ajoute leurs institutions et leurs approches, d’après les articles des dernières années.

Continuer avec Google Créez un compte gratuit : votre première carte est offerte
Lettre hebdomadaire La dernière lettre de son domaine, « Intelligence artificielle cette semaine » du 16 novembre 2026 : Intelligence artificielle cette semaine : ce que les bons scores ne disent pas · En recevoir une par email gratuitement

Les articles autour de celui-ci

Les voir sur une carte

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

  1. Très proche
    MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
    Alan Ansell, E. Ponti, Jonas Pfeiffer et al. · 2021 · Conference on Empirical Methods in Natural Language Processing · cité 103 fois · Pas encore résumé
    Résumé scientifique

    Adapter modules have emerged as a general parameter-efficient means to specialize a pre-trained encoder to new domains. Massively multilingual transformers (MMTs) have particularly benefited from additional training of language-specific adapters. However, this approach is not viable for the vast majority of languages, due to limitations in their corpus size or compute budgets. In this work, we propose MAD-G ( M ultilingual AD apter G eneration), which contextually generates language adapters from language representations based on typological features. In contrast to prior work, our time-and space-efficient MAD-G approach enables (1) sharing of linguistic knowledge across languages and (2) zero-s…

  2. Très proche
    Cross-Lingual Transfer with Target Language-Ready Task Adapters
    Marinela Parovi'c, Alan Ansell, Ivan Vulic et al. · 2023 · Annual Meeting of the Association for Computational Linguistics · cité 19 fois · Pas encore résumé
    Résumé scientifique

    Adapters have emerged as a modular and parameter-efficient approach to (zero-shot) cross-lingual transfer. The established MAD-X framework employs separate language and task adapters which can be arbitrarily combined to perform the transfer of any task to any target language. Subsequently, BAD-X, an extension of the MAD-X framework, achieves improved transfer at the cost of MAD-X's modularity by creating"bilingual"adapters specific to the source-target language pair. In this work, we aim to take the best of both worlds by (i) fine-tuning task adapters adapted to the target language(s) (so-called"target language-ready"(TLR) adapters) to maintain high transfer performance, but (ii) without sac…

  3. Très proche
    X-METRA-ADA: Cross-lingual Meta-Transfer learning Adaptation to Natural Language Understanding and Question Answering
    Meryem M'hamdi, Doo Soon Kim, Franck Dernoncourt et al. · 2021 · North American Chapter of the Association for Computational Linguistics · cité 21 fois · Pas encore résumé
    Résumé scientifique

    Multilingual models, such as M-BERT and XLM-R, have gained increasing popularity, due to their zero-shot cross-lingual transfer learning capabilities. However, their generalization ability is still inconsistent for typologically diverse languages and across different benchmarks. Recently, meta-learning has garnered attention as a promising technique for enhancing transfer learning under low-resource scenarios: particularly for cross-lingual transfer in Natural Language Understanding (NLU). In this work, we propose X-METRA-ADA, a cross-lingual MEta-TRAnsfer learning ADAptation approach for NLU. Our approach adapts MAML, an optimization-based meta-learning approach, to learn to adapt to new la…

  4. Très proche
    Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer
    A. Ustun, Arianna Bisazza, G. Bouma et al. · 2022 · Conference on Empirical Methods in Natural Language Processing · cité 40 fois · Pas encore résumé
    Résumé scientifique

    Massively multilingual models are promising for transfer learning across tasks and languages. However, existing methods are unable to fully leverage training data when it is available in different task-language combinations. To exploit such heterogeneous supervision, we propose Hyper-X, a single hypernetwork that unifies multi-task and multilingual learning with efficient adaptation. It generates weights for adapter modules conditioned on both tasks and language embeddings. By learning to combine task and language-specific knowledge, our model enables zero-shot transfer for unseen languages and task-language combinations. Our experiments on a diverse set of languages demonstrate that Hyper-X…

  5. Très proche
    WAD-X: Improving Zero-shot Cross-lingual Transfer via Adapter-based Word Alignment
    Ahtamjan Ahmat, Ya-Ting Yang, Bo Ma et al. · 2023 · ACM Trans. Asian Low Resour. Lang. Inf. Process. · cité 7 fois · Pas encore résumé
    Résumé scientifique

    Multilingual pre-trained language models (mPLMs) have achieved remarkable performance on zero-shot cross-lingual transfer learning. However, most mPLMs implicitly encourage cross-lingual alignment in pre-training stage, making it hard to capture accurate word alignment across languages. In this paper, we propose Word-align ADapters for Cross-lingual transfer (WAD-X) to explicitly align word representations of mPLMs via language-specific subspace. Taking a mPLM as the backbone model, WAD-X constructs subspace for each source-target language pair via adapters. The adapters use statistical alignment as the prior knowledge to guide word-level aligning in the corresponding bilingual semantic subs…

  6. Très proche Cité par cet article Utilisé en profondeur par cet article Une méthode qu’il utilise
    Unsupervised Cross-lingual Representation Learning at Scale
    Alexis Conneau, Kartikay Khandelwal, Naman Goyal et al. · 2019 · Computation and Language · cité 9 795 fois · Résumé IA disponible
    Résumé scientifique

    This paper shows that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks. We train a Transformer-based masked language model on one hundred languages, using more than two terabytes of filtered CommonCrawl data. Our model, dubbed XLM-R, significantly outperforms multilingual BERT (mBERT) on a variety of cross-lingual benchmarks, including +13.8% average accuracy on XNLI, +12.3% average F1 score on MLQA, and +2.1% average F1 score on NER. XLM-R performs particularly well on low-resource languages, improving 11.8% in XNLI accuracy for Swahili and 9.2% for Urdu over the previous XLM model. We also present a de…

  7. Proche
    FAD-X: Fusing Adapters for Cross-lingual Transfer to Low-Resource Languages
    Jaeseong Lee, Seung-won Hwang, Taesup Kim · 2022 · AACL · cité 11 fois · Pas encore résumé
    Résumé scientifique

    Adapter-based tuning, by adding light-weight adapters to multilingual pretrained language models (mPLMs), selectively updates language-specific parameters to adapt to a new language, instead of finetuning all shared weights. This paper explores an effective way to leverage a public pool of pretrained language adapters, to overcome resource imbalances for low-resource languages (LRLs). Specifically, our research questions are, whether pretrained adapters can be composed, to complement or replace LRL adapters. While composing adapters for multi-task learning setting has been studied, the same question for LRLs has remained largely unanswered. To answer this question, we study how to fuse adapt…

  8. Proche Cite cet article Utilise cet article en profondeur
    AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging
    Yi-Ran Zhao, Wen-Xuan Zhang, Huiming Wang et al. · 2024 · North American Chapter of the Association for Computational Linguistics · cité 47 fois · Pas encore résumé
    Résumé scientifique

    As an effective alternative to the direct fine-tuning on target tasks in specific languages, cross-lingual transfer addresses the challenges of limited training data by decoupling ''task ability'' and ''language ability'' by fine-tuning on the target task in the source language and another selected task in the target language, respectively. However, they fail to fully separate the task ability from the source language or the language ability from the chosen task. In this paper, we acknowledge the mutual reliance between task ability and language ability and direct our attention toward the gap between the target language and the source language on tasks. As the gap removes the impact of tasks…

Les travaux antérieurs sur lesquels il s’appuie

  1. Proche Utilisé en profondeur par cet article Une méthode qu’il utilise
    On the Cross-lingual Transferability of Monolingual Representations
    Mikel Artetxe, Sebastian Ruder, Dani Yogatama · 2019 · Annual Meeting of the Association for Computational Linguistics · cité 1 019 fois · Pas encore résumé
    Résumé scientifique

    State-of-the-art unsupervised multilingual models (e.g., multilingual BERT) have been shown to generalize in a zero-shot cross-lingual setting. This generalization ability has been attributed to the use of a shared subword vocabulary and joint training across multiple languages giving rise to deep multilingual abstractions. We evaluate this hypothesis by designing an alternative approach that transfers a monolingual model to new languages at the lexical level. More concretely, we first train a transformer-based masked language model on one language, and transfer it to a new language by learning a new embedding matrix with the same masked language modeling objective, freezing parameters of al…

  2. Proche Utilisé en profondeur par cet article Une méthode qu’il utilise
    XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
    Jun-Jie Hu, Sebastian Ruder, Aditya Siddhant et al. · 2020 · International Conference on Machine Learning · cité 1 235 fois · Pas encore résumé
    Résumé scientifique

    Much recent progress in applications of machine learning models to NLP has been driven by benchmarks that evaluate models across a wide variety of tasks. However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing. To this end, we introduce the Cross-lingual TRansfer Evaluation of Multilingual Encoders XTREME benchmark, a multi-task benchmark for evaluating the cross-lingual generalization capabilities of multilingual representations across 40 languages and 9 tasks. We demonstrate that whi…

  3. Proche Utilisé en profondeur par cet article Une méthode qu’il utilise
    Parameter-Efficient Transfer Learning for NLP
    N. Houlsby, A. Giurgiu, Stanislaw Jastrzebski et al. · 2019 · International Conference on Machine Learning · cité 7 837 fois · Pas encore résumé
    Résumé scientifique

    Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. However, in the presence of many downstream tasks, fine-tuning is parameter inefficient: an entire new model is required for every task. As an alternative, we propose transfer with adapter modules. Adapter modules yield a compact and extensible model; they add only a few trainable parameters per task, and new tasks can be added without revisiting previous ones. The parameters of the original network remain fixed, yielding a high degree of parameter sharing. To demonstrate adapter's effectiveness, we transfer the recently proposed BERT Transformer model to 26 diverse text classification tasks, including the GLUE b…

  4. Proche Cite cet article Utilisé en profondeur par cet article Une méthode qu’il utilise
    AdapterHub: A Framework for Adapting Transformers
    Jonas Pfeiffer, Andreas Rücklé, Clifton A. Poth et al. · 2020 · Conference on Empirical Methods in Natural Language Processing · cité 793 fois · Pas encore résumé
    Résumé scientifique

    The current modus operandi in NLP involves downloading and fine-tuning pre-trained models consisting of millions or billions of parameters. Storing and sharing such large trained models is expensive, slow, and time-consuming, which impedes progress towards more general and versatile NLP methods that learn from and for many tasks. Adapters—small learnt bottleneck layers inserted within each layer of a pre-trained model— ameliorate this issue by avoiding full fine-tuning of the entire model. However, sharing and integrating adapter layers is not straightforward. We propose AdapterHub, a framework that allows dynamic “stiching-in” of pre-trained adapters for different tasks and languages. The f…

  5. Proche Utilisé en profondeur par cet article Une méthode qu’il utilise
    Simple, Scalable Adaptation for Neural Machine Translation
    Ankur Bapna, N. Arivazhagan, O. Firat · 2019 · Conference on Empirical Methods in Natural Language Processing · cité 481 fois · Pas encore résumé
    Résumé scientifique

    Fine-tuning pre-trained Neural Machine Translation (NMT) models is the dominant approach for adapting to new languages and domains. However, fine-tuning requires adapting and maintaining a separate model for each target task. We propose a simple yet efficient approach for adaptation in NMT. Our proposed approach consists of injecting tiny task specific adapter layers into a pre-trained model. These lightweight adapters, with just a small fraction of the original model size, adapt the model to multiple individual tasks simultaneously. We evaluate our approach on two tasks: (i) Domain Adaptation and (ii) Massively Multilingual NMT. Experiments on domain adaptation demonstrate that our proposed…

Les articles qui le citent

  1. Proche Utilise cet article en profondeur
    Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal Retrieval
    Rui Cai, Zhi-Yu Dong, Jian-Feng Dong et al. · 2024 · AAAI Conference on Artificial Intelligence · cité 5 fois · Pas encore résumé
    Résumé scientifique

    Existing cross-modal retrieval methods typically rely on large-scale vision-language pair data. This makes it challenging to efficiently develop a cross-modal retrieval model for under-resourced languages of interest. Therefore, Cross-lingual Cross-modal Retrieval (CCR), which aims to align vision and the low-resource language (the target language) without using any human-labeled target-language data, has gained increasing attention. As a general parameter-efficient way, a common solution is to utilize adapter modules to transfer the vision-language alignment ability of Vision-Language Pretraining (VLP) models from a source language to a target language. However, these adapters are usually s…

  2. Proche Utilise cet article en profondeur
    No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement
    Mateusz Klimaszewski, P. Andruszkiewicz, Alexandra Birch · 2024 · International Conference on Computational Linguistics · cité 6 fois · Pas encore résumé
    Résumé scientifique

    Modular deep learning is the state-of-the-art solution for lifting the curse of multilinguality, preventing the impact of negative interference and enabling cross-lingual performance in Multilingual Pre-trained Language Models. However, a trade-off of this approach is the reduction in positive transfer learning from closely related languages. In response, we introduce a novel method called language arithmetic, which enables training-free post-processing to address this limitation. Extending the task arithmetic framework, we apply learning via addition to the language adapters, transitioning the framework from a multi-task to a multilingual setup. The effectiveness of the proposed solution is…

  3. Proche Utilise cet article en profondeur
    Sequential Adapter Stacking for Cross-Lingual Low-Resource ASR
    Thai-Thinh Dang, Meng-Jie Qian, Kate Knill · 2026 · cité 1 fois · Pas encore résumé
    Résumé scientifique

    Extending large-scale multilingual automatic speech recognition (ASR) models to low-resource languages remains challenging. Model performance is skewed toward high-resource languages and degrades sharply for languages with limited labeled data and pre-training exposure. To address this, we investigate parameter-efficient approaches for transferring knowledge from resource-rich source languages to low-resource target languages on Whisper. Alongside warm initialization and attention-based fusion, we propose Sequential Adapter Stacking, which places a trainable target-language adapter on top of a frozen source-language adapter. Under controlled experiments, these approaches are evaluated on thr…

  4. Proche Utilise cet article en profondeur
    Typologically Informed Parameter Aggregation
    Stef Accou, Wessel Poelman · 2026 · Conference of the European Chapter of the Association for Computational Linguistics · cité 1 fois · Pas encore résumé
    Résumé scientifique

    Massively multilingual language models enable cross-lingual generalization but underperform on low-resource and unseen languages. While adapter-based fine-tuning offers a parameter-efficient solution, training language-specific adapters at scale remains costly. We introduce Typologically Informed Parameter Aggregation (TIPA), a training-free method that constructs proxy language adapters by aggregating existing ones, weighted by typological similarity. Integrated into the MAD-X framework, these proxies enable zero-shot cross-lingual transfer without additional training. We evaluate TIPA on five NLP tasks and over 230 languages. TIPA consistently outperforms or matches baselines such as Engli…

  5. Proche
    From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers
    Anne Lauscher, Vinit Ravishankar, Ivan Vulić et al. · 2020 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · cité 250 fois · Pas encore résumé
    Résumé scientifique

    Massively multilingual transformers (MMTs) pretrained via language modeling (e.g., mBERT, XLM-R) have become a default paradigm for zero-shot language transfer in NLP, offering unmatched transfer performance.Current evaluations, however, verify their efficacy in transfers (a) to languages with sufficiently large pretraining corpora, and (b) between close languages.In this work, we analyze the limitations of downstream language transfer with MMTs, showing that, much like cross-lingual word embeddings, they are substantially less effective in resource-lean scenarios and for distant languages.Our experiments, encompassing three lower-level tasks (POS tagging, dependency parsing, NER) and two hi…

Comment ces articles sont-ils trouvés ?

Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.

Mis à jour le 4 novembre 2026.

Le guide illustré des articles autour d’un article

Le résumé précédent a été créé il y a plus d'un an et peut être réexécuté (si nécessaire) en cliquant sur le bouton Exécuter ci-dessous.

Avertissement : Notre outil de synthèse basé sur l'IA et l'assistant virtuel fournis sur ce site Web peuvent ne pas toujours fournir des résumés complets ou des réponses exactes. Nous vous encourageons à examiner attentivement et à évaluer le contenu généré pour vous assurer de sa qualité et de sa pertinence par rapport à vos besoins.