Search Results for « Low-resource language modeling »

This paper proposes an approach for aspect-based sentiment analysis of Arabic social data, especially the considerable text corpus generated through communications on X (formerly known as Twitter) for expressing opinions in Arabic-language tweets during the COVID-19 pandemic. The proposed approach examines the performance of several pre-trained predictive and autoregressive language models; namely, Bidirectional Encoder Representations from Transformers (BERT) and XLNet, along with topic modeling algorithms; namely, Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), for aspect-based sentiment analysis of online Arabic text. In addition, Bidirectional Long Short Term Memory (Bi-LSTM) deep learning model is used to classify the extracted aspects from online reviews. Obtained experimental results indicate that the combined XLNet-NMF model outperforms other implemented state-of-the-art methods through improving the feature extraction of unstructured social media text with achieving values of 0.946 and 0.938, for average sentiment classification accuracy and F-measure, respectively.

This paper looks at improving multilingual language models for Nepali, which is a language with limited resources. We take a two-step approach: first, we train the model using Masked Language Modeling (MLM) on Nepali-only text, then we continue training with Translation Language Modeling (TLM) using English-Nepali parallel texts. Our experiments show that adapting the XLM-R Base model this way helps reduce Nepali perplexity and boosts performance on tasks like machine translation, sentiment analysis, and question answering. We also share our methods and results to support future research on underrepresented South Asian languages.

Various facts, including politics, entertainment, industry, and research fields, are connected to analyzing the audience's emotions. Sentiment Analysis (SA) is a Natural Language Processing (NLP) concept that uses statistical and lexical forms as well as learning techniques to forecast how different types of content in social media will express the audience's neutral, positive, and negative emotions. There is lack of an adequate tool to quantify the characteristics and independent text for assessing the primary audience emotion from the available online social media dataset. The focus of this research is on modeling a cutting-edge method for decoding the connectivity among social media texts and assessing audience emotions. Here, a novel dense layer graph model (DLG-TF) for textual feature analysis is used to analyze the relevant connectedness inside the complex media environment to forecast emotions. The information from the social media dataset is extracted using some popular convolution network models, and the predictions are made by examining the textual properties. The experimental results show that, when compared to different standard emotions, the proposed DLG-TF model accurately predicts a greater number of possible emotions. The macro-average of baseline is 58%, the affective is 55%, the crawl is 55%, and the ultra-dense is 59%, respectively. The feature analysis comparison of baseline, affective, crawl, ultra-dense and DLG-TF using the unsupervised model based on EmoTweet gives the precision, recall, and F1-score of the anticipated model are explained. The micro- and macro-average based on these parameters are compared and analyzed. The macro-average of baseline is 47%, the affective is 46%, the crawl is 50%, and the ultra-dense is 85%, respectively. It makes precise predictions using the social media dataset that is readily available. A few criteria, including accuracy, recall, precision, and F-measure, are assessed and contrasted with alternative methods.

Offensive content is pervasive in social media and a reason for concern to companies and government organizations. Several studies have been recently published investigating methods to detect the various forms of such content (e.g., hate speech, cyberbullying, and cyberaggression). The clear majority of these studies deal with English partially because most annotated datasets available contain English data. In this article, we take advantage of available English datasets by applying cross-lingual contextual word embeddings and transfer learning to make predictions in low-resource languages. We project predictions on comparable data in Arabic, Bengali, Danish, Greek, Hindi, Spanish, and Turkish. We report results of 0.8415 F1 macro for Bengali in TRAC-2 shared task [23], 0.8532 F1 macro for Danish and 0.8701 F1 macro for Greek in OffensEval 2020 [58], 0.8568 F1 macro for Hindi in HASOC 2019 shared task [27], and 0.7513 F1 macro for Spanish in in SemEval-2019 Task 5 (HatEval) [7], showing that our approach compares favorably to the best systems submitted to recent shared tasks on these three languages. Additionally, we report competitive performance on Arabic and Turkish using the training and development sets of OffensEval 2020 shared task. The results for all languages confirm the robustness of cross-lingual contextual embeddings and transfer learning for this task.

Continuous proliferation of hate speech in different languages on social media has drawn significant attention from researchers in the past decade. Detecting hate speech is indispensable irrespective of the scale of use of language, as it inflicts huge harm on society. This work presents a first resource for classifying the severity of hate speech in addition to classifying offensive and hate speech content. Current research mostly limits hate speech classification to its primary categories, such as racism, sexism, and hatred of religions. However, hate speech targeted at different protected characteristics also manifests in different forms and intensities. It is important to understand varying severity levels of hate speech so that the most harmful cases of hate speech may be identified and dealt with earlier than the less harmful ones. In this work, we focus on detecting offensive speech, hate speech, and multiple levels of hate speech in the Urdu language. We investigate three primary target categories of hate speech: religion, racism, and national origin. We further divide these categories into levels based on the severity of hate conveyed. The severity levels are referred to as symbolization , insult , and attribution . A corpus comprising more than 20,000 tweets against the corresponding hate speech categories and severity levels is collected and annotated. A comprehensive experimentation scheme is applied using traditional as well as deep learning–based models to examine their impact on hate speech detection. The highest macro-averaged F-score yielded for detecting offensive speech is 86% while the highest F-scores for detecting hate speech with respect to ethnicity, national origin, and religious affiliation are 80%, 81%, and 72%, respectively. This shows that results are very encouraging and would provide a lead towards further investigation in this domain.