National and Kapodistrian University of Athens
Design and development of bioinformatics tools for the effective and systematic distinction between true (biological) and artificial somatic variants in pathological conditions, focusing on low frequency variants
Abstract
dc:descriptionAdvances in next-generation sequencing (NGS) technologies have enabled the detection of genomic alterations at unprecedented scale and resolution, leading to the development of numerous computational tools for variant discovery. However, despite continuous methodological improvements, the detection of somatic variants remains challenging. This is particularly true for variants occurring at low allele frequencies, where technical artifacts can substantially affect performance.Over the past decade, a large number of variant calling algorithms have been developed, each employing different computational principles, including statistical models, heuristic filtering approaches and haplotype based local assembly strategies. As a result, different tools often produce discordant outputs when applied to the same dataset. This lack of agreement becomes even more pronounced in tumor-only NGS analyses, where no matched normal sample is available to facilitate artifact filtering. Consequently, selecting an appropriate variant caller remains a crucial task, while systematic benchmarking efforts are often hindered by the limited availability of datasets with well defined ground truth.A major challenge in variant calling evaluation is the lack of knowledge regarding the actual mutational landscape of biological samples. Although initiatives such as Genome in a Bottle, the ICGC-TCGA DREAM Challenge and SEQC2 have significantly contributed to benchmarking efforts, biological datasets inherently contain uncertainties related to incomplete truth sets, biological heterogeneity and experimental variability. These limitations complicate the direct assessment of variant caller performance and controlled comparisons.To address this challenge, this PhD research focused on the development of synth4bench, a synthetic data based benchmarking framework designed to systematically evaluate tumor-only somatic variant callers under fully controlled conditions. Synth4bench was conceived as a technical benchmarking pipeline that enables reproducible investigation of variant caller behaviour across different NGS conditions, while maintaining explicit knowledge of the underlying ground truth.The framework utilizes the NEAT simulator to generate synthetic NGS datasets from a human reference genome. Synthetic datasets were generated within the TP53 gene locus of the GRCh38 reference genome and systematically varied across two key sequencing parameters: sequencing coverage and read length. Coverage values ranged from 300× to 5000×, while read lengths ranged from 50 bp to 300 bp, creating a structured experimental feature space that allowed the investigation of algorithmic responses to different combinations of NGS features.The central methodological contribution of synth4bench concerns the implementation of a controlled dilution strategy for generating variants with different allele frequencies (AF), including low-frequency variants. Multiple independently simulated datasets were merged to create predefined AF distributions, while preserving complete knowledge of variant origin. Variants present at 100% AF within individual datasets were designated as the True Variant set and subsequently diluted through the merging process, enabling systematic benchmarking of low-frequency variant detection. This design provided control over variant AF and allowed direct evaluation of caller sensitivity under challenging detection scenarios.Five widely used tumor-only variant callers were selected for evaluation, namely Mutect2, FreeBayes, LoFreq, VarDict and VarScan2. The framework was designed as a complete end-to-end benchmarking environment, incorporating data generation, preprocessing, variant calling, normalization, harmonization of variant representations and downstream performance assessment. Particular emphasis was placed on reproducibility through the use of well-defined workflows, controlled simulation settings and comprehensive tracking of all experimental parameters.Three independent analyses were performed within the benchmarking framework. The first focused on predefined True Variants, evaluating the ability of callers to recover known variants after dilution. The second examined background variant signal, referred to as “noise”, allowing investigation of caller behaviour in the presence of variants not belonging to the predefined truth set. The third analysis focused on insertions and deletions (indels), which represent one of the most challenging types of genomic lesions for variant detection algorithms.To improve the assessment of indel performance, an additional harmonization and categorization strategy was developed. Beyond standard indel normalization procedures, mismatches between caller output and ground truth were further classified according to positional discrepancies, reference allele mismatches and alternate allele mismatches. This additional level of granularity enabled a more detailed characterization of indel specific errors and provided insights into the underlying causes of caller disagreement.A total of 29 synthetic datasets were generated and analyzed. The benchmarking process revealed substantial differences among the evaluated caller outputs. Variant caller behaviour was found to vary significantly across the examined NGS parameter space, highlighting the importance of benchmarking under controlled conditions before applying these tools in real world studies.Among the evaluated callers, VarScan2 demonstrated the highest sensitivity for recovering predefined True Variants, consistently yielding the largest number of true positive calls. However, this increased sensitivity was accompanied by a small tendency toward AF underestimation. LoFreq exhibited the most stable AF estimates and maintained high precision across multiple analyses, making it particularly suitable when accurate quantification of variant AF is required. VarDict achieved a favorable balance between sensitivity and indel detection performance, whereas FreeBayes generally adopted a more conservative calling strategy, resulting in lower false positive rates along with substantially reduced sensitivity. Mutect2 showed increased robustness to NGS quality variation, yet exhibited lower overall sensitivity within the evaluated tumor-only setting.Our study also highlighted a persistent difficulty in indel detection. Across all evaluated callers, indels exhibited substantially lower recall compared to single nucleotide variants (SNVs). False negative indels dominated the results, with positional discordance emerging as one of the principal causes of disagreement between caller outputs and the ground truth. Our findings reinforce previous observations that indel detection remains an open challenge in variant calling and requires further methodological development.Overall, our results demonstrate that no single variant caller consistently outperformed all others across every analytical objective. Instead, caller selection should be guided by the specific goals of a study, such as maximizing sensitivity, minimizing false positives, improving AF estimation, or enhancing indel detection. The benchmarking framework therefore not only enables systematic comparison of existing methods, but also provides practical guidance for selecting appropriate tools under different NGS strategies.Altogether, the primary contribution of this PhD research is the development of a reproducible and extensible framework for controlled benchmarking of tumor-only somatic variant callers. Synth4bench provides a transparent methodology for generating synthetic datasets with known ground truth, evaluating caller performance under predefined NGS conditions and dissecting algorithmic behaviour. By enabling systematic assessment of variant detection performance, the framework contributes toward improving benchmarking practices within the field of genomics and offers a foundation for future studies involving additional genomic regions, whole genome analyses, new variant callers and machine learning-based variant interpretation approaches.Future work will focus on extending the framework beyond the current locus-limited implementation, incorporating additional NGS characteristics and quality profiles, while also investigating the relationship between synthetic benchmarking datasets and biological benchmark resources. Such developments will strengthen the role of synthetic data as a complementary approach for methodological evaluation and support the continued advancement of reliable genomic analysis pipelines.
Degree
thesis:*- Grantor dc:publisher
- National and Kapodistrian University of Athens
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Fragkouli, Styliani-Christina
- Φραγκούλη, Στυλιανή-Χριστίνα
Subjects
dc:subject × 18- Βιοπληροφορική
- Βιοπληροφορικοί αλγόριθμοι
- Συνθετικά δεδομένα
- Ανίχνευση παραλλαγών
- Αξιολόγηση αλγορίθμων
- Γονιδιωματική
- Bioinformatics
- Bioinformatics algorithms
- Synthetic data
- Variant calling
- Benchmarking
- Genomics
- Βιολογία
- Φυσικές Επιστήμες
- Βιολογία, διεπιστημονική προσέγγιση
- Biological Sciences
- Natural Sciences
- Biology, interdisciplinary
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/63019