Back to search

McMaster University

Domain-Specific Text Embedding Models for Information Retrieval

Abstract

dc:description.abstract

Large Language Models (LLMs) have shown advanced capabilities across various fields. However, using these models out of the box, especially in specialized domains like chemistry, often leads to issues such as context limitations, hallucinations, difficulty updating their parametric knowledge, and unclear sources for generated responses. To tackle these challenges, Retrieval-Augmented Generation (RAG) enables language models to use external knowledge sources during inference, improving factual accuracy and allowing dynamic knowledge retrieval without costly retraining. A critical part of any RAG system is the text embedding model, which searches for the most relevant documents in a knowledge base given a query. However, standard embedding models trained on general datasets perform poorly in chemistry because of the field's unique vocabulary, specialized terms, and complex semantics. This thesis introduces the Chemical Text Embedding Benchmark (ChemTEB), designed specifically to evaluate embedding models on chemical tasks. ChemTEB systematically measures models' performance, clearly identifying their strengths and weaknesses when handling chemistry-related texts. Using insights from ChemTEB, we developed ChEmbed, a family of text embedding models fine-tuned specifically for chemistry. To achieve this, we gathered domain-specific text from chemistry research articles and public resources. We then generated synthetic queries for these texts using LLMs, creating realistic query-passage pairs for training. This approach effectively addresses issues of limited data availability and improves the model’s ability to represent chemistry-specific language. Additionally, we introduced a new domain-specific tokenizer that efficiently integrates chemical terms into an existing pretrained model, enhancing the accuracy of text representations. Together, ChemTEB and ChEmbed offer the first domain-adapted solution for chemical text retrieval, overcoming the performance limitations of general embedding models. This contributes to more accurate and interpretable AI-based chemical research and discovery. Although the focus here is chemistry, the methods can serve as a practical framework for adapting embedding models to other specialized fields.

Degree

thesis:*
Department dc:contributor.department
Computational Engineering and Science
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Shiraee Kasmaee, Ali
Advisor dc:contributor.advisor
  • Mahyar, Hamidreza

Subjects

dc:subject × 3

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
McMaster University
Base URL
macsphere.mcmaster.ca/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Shiraee Kasmaee, Ali. Domain-Specific Text Embedding Models for Information Retrieval. 2025. http://hdl.handle.net/11375/32255