Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

AI-generated keywords: SBERT BERT Semantic Similarity Sentence Embeddings NLP Applications

AI-generated Key Points

  • Sentence-BERT (SBERT) is a modification of the BERT network
  • SBERT utilizes siamese and triplet network structures to derive semantically meaningful sentence embeddings
  • SBERT allows for efficient semantic similarity search and unsupervised tasks like clustering without requiring both sentences to be fed into the network
  • SBERT significantly reduces computational overhead for finding similar pairs in a collection of 10,000 sentences from 65 hours with BERT to just 5 seconds while maintaining accuracy
  • SBERT outperforms other state-of-the-art sentence embedding methods in various sentence-pair regression tasks and transfer learning tasks
  • SBERT enables BERT to be applied to new tasks that were previously not feasible
  • SBERT achieves improved performance in several NLP applications.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Nils Reimers, Iryna Gurevych

Published at EMNLP 2019
License: CC BY-SA 4.0

Abstract: BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art performance on sentence-pair regression tasks like semantic textual similarity (STS). However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10,000 sentences requires about 50 million inference computations (~65 hours) with BERT. The construction of BERT makes it unsuitable for semantic similarity search as well as for unsupervised tasks like clustering. In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT. We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.

Submitted to arXiv on 27 Aug. 2019

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 1908.10084v1

The paper introduces Sentence-BERT (SBERT), a modification of the BERT network that utilizes siamese and triplet network structures to derive semantically meaningful sentence embeddings. SBERT allows for efficient semantic similarity search and unsupervised tasks like clustering without requiring both sentences to be fed into the network. The authors demonstrate that SBERT significantly reduces the computational overhead required for finding the most similar pair in a collection of 10,000 sentences from 65 hours with BERT to just 5 seconds while maintaining accuracy. They evaluate SBERT on various sentence-pair regression tasks and transfer learning tasks where it outperforms other state-of-the-art sentence embedding methods. Overall, SBERT enables BERT to be applied to new tasks that were previously not feasible and achieves improved performance in several NLP applications.
Created on 24 Jul. 2023

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.