We’re excited to announce RAFT v1.5.1 (our Nextflow-based bioinformatics project manager) and LENS v1.9.0 (our tumor antigen prediction workflow). Together, these releases make LENS substantially easier to install and run while providing a more rigorously reviewed, higher-confidence set of tumor antigen candidates.

For new users, these releases substantially reduce the work required to install RAFT, configure LENS, and review results. For existing users, they add more robust cloud execution, automatic resource scaling, improved QC, and tiered antigen reports designed to distinguish the complete candidate set from the candidates supported by the strongest evidence.

These releases began with a conversation Brian Parker and I had over breakfast in early January. As we reflected on the previous year, I wasn’t satisfied with where RAFT and LENS stood in the immuno-oncology bioinformatics ecosystem. RAFT was not as portable as I wanted it to be, and LENS remained too difficult to run reliably.

RAFT and LENS had a small group of users who provided excellent feedback. However, we also suspected that other scientists had tried the tools, encountered installation or environment-specific problems, and moved on. Expanding the user base would not only make the tools useful to more researchers—it would also expose more edge cases, uncover more issues, and ultimately make both projects stronger.

Our goal for these releases was therefore straightforward: make RAFT and LENS accessible to scientists across a much broader range of bioinformatics experience.

That goal presented a practical challenge. The core development team is small: Brian and I develop RAFT and LENS alongside our other responsibilities in the lab. To accelerate development, we began using agentic coding models to assist with implementing, testing, and debugging selected changes. AI-assisted changes remain subject to human review and the same testing and validation standards as the rest of the codebase.

Below is a bird’s-eye view of the major improvements, followed by a more detailed discussion of their motivation, expected impact, and implementation.

• Web-based manifest setup, workflow configuration, and interactive results review

• Improved cloud support and streamlined commands

• Automated RAFT dependency handling

• Automatic sample-level resource scaling

• Higher-confidence antigen prioritization

• Expanded tumor characterization and QC

RAFT updates

RAFT (Reproducible Analyses Framework and Tools) is our Nextflow wrapper that creates and manages project-level environments for reproducible bioinformatics analyses. More information about RAFT can be found at useraft.io.

User experience enhancements

• RAFT web UI — Users can now use a web-based UI to create sample manifests and configure workflows during raft run, or to review project outputs with raft generate-reports. This also works in headless environments, as long as users can copy and paste data between the headless environment and a system with a web browser.

• Unified raft run — RAFT now has a single entry point that replaces raft run-ots (previously used to create a project using an off-the-shelf workflow, like LENS) and raft run-workflow (previously used to run an already established project).

• Added raft project-status — Users now have access to a project-level summary describing the project, the manifest, the project’s run status, and available outputs, if any.

• Improved reference download visuals — RAFT now provides a cleaner interface during reference file downloading.

• Automated dependency installs and tests — RAFT dependencies (Nextflow, Git, and Singularity) can be automatically installed using raft install-dependencies.

• Minimal profile generation — RAFT now provides users with a default Nextflow profile with sensible resource allocations to use on their local machine whenever RAFT cannot detect a user-defined Nextflow profile.

• Faster module downloads — Project-specific module fetches are now faster thanks to adaptive throttling.

Improved workflow support

• Workflow profiles — Workflows, such as LENS, can now support multiple configuration profiles when using raft run with the --workflow-profile parameter. Available workflow profiles can be discovered using raft available-workflow-profiles.

• Patient filtering — RAFT now filters patients who do not meet a workflow’s minimum input requirements—for example, patients without an RNA-seq sample when one is required by LENS. Users can bypass this filtering with --include-all-samples.

• Resource scaling — Resources are now allocated on a sample-by-sample basis based on input FASTQ size. This reduces failures caused by underallocated resources while avoiding unnecessary overallocation.

Better reference handling

• raft download-refs — Users can now download references for a specific workflow (and version) without first creating a RAFT project.

• Cloud staging — References can now be staged to a specific cloud bucket (for use with --references-bucket) without first creating a RAFT project.

• Improved duplicate-reference errors — Duplicate reference files (files or directories with the same name in different locations under /path/to/raft/references) are now explicitly listed when a duplicate-reference error occurs.

• Safer reference generation — Lock-based reference generation prevents concurrent raft run (or raft download-refs) commands from corrupting downloads.

• Mirrors for references — Several references now have fallback mirrors available in case the primary source goes offline. If mirrors fail, users can provide an alternative URL or local filesystem path (if the file is already available elsewhere).

Support for off-the-shelf immuno-oncology datasets

• Off-the-shelf datasets — RAFT now supports running analyses on publicly available datasets by using --dataset with raft run. raft available-datasets lists available datasets, and raft download-dataset lets users download a dataset (FASTQs, metadata, and manifest) without creating a RAFT project. Currently, only Hugo et al., 2016 is available, though more datasets are planned.

Expanded cloud support

• Formally supported hybrid mode — RAFT’s hybrid mode (local head node with cloud-based processing) is now formally supported. This has been tested extensively in GCP, but we’d like feedback on AWS support.

• Multiple-bucket support — Hybrid mode now supports --references-bucket (for RAFT reference files and aligner indexes), --fastqs-bucket (for input FASTQs), and --project-bucket (for project-specific files). This enables stable, shared buckets across projects, reduces load on the head node, and lets users isolate storage by bucket when desired.

• Resource labels — Hybrid mode emits user-customizable, cloud-provider-aware resource labels to help track costs and resource usage.

• Output syncing — RAFT now copies relevant output files (e.g., QC data and LENS reports) back to the head node upon workflow completion.

• BYOC (EXPERIMENTAL) — We are introducing a bring-your-own-cloud approach that is entirely web UI-driven and does not require installing RAFT on a head node. Note that this mode is under active development and is not yet recommended for production use.

Maintainability and CLI improvements

• Module-based, not monolithic — Earlier versions of RAFT used a monolithic Python script. RAFT v1.5.1 now uses a module-based architecture, which should make debugging easier and help support new feature development.

• Message of the day — RAFT now provides a message of the day (MOTD) when users are running either an out-of-date version of RAFT or an out-of-date workflow version.

• Improved Nextflow support — RAFT now runs using Nextflow 26.04.6 instead of relying on older versions.

Reproducibility and archiving

• Improved project tracking — RAFT now tracks not only the commands used to generate a RAFT project, but also changes made between runs (e.g., parameter changes, Docker image changes, or reference changes).

Improved documentation

• Improved user-focused documentation — useraft.io now guides users through understanding and running RAFT for their purposes.

• Code status — RAFT now provides automatically updated Pass/Fail/In Progress status indicators for workflows, datasets, subworkflows, and tool-level modules.

LENS updates

LENS (Landscape of Effective Neoantigen Software) is our tumor antigen prediction workflow, run through RAFT. LENS v1.9.0 improves execution efficiency, expands tumor characterization and candidate-level QC, and introduces tiered reports that make it easier to identify candidates supported by the strongest evidence.

Workflow improvements

• Additional modular subworkflows — Previously hard-coded subworkflows (such as variant annotation with snpEff) have been replaced with modular subworkflows to improve tool support.

• Simplified pMHC identifiers — LENS now assigns candidate-id as a user-friendly unique identifier across pMHCs from various origins.

• Multiplier-aware modules — Tool-level module configuration files now support RAFT’s resource scaling functionality.

Seamless input handling

• Lane merging — LENS now automatically merges FASTQs generated from the same library for use as workflow inputs.

Faster alignments

• Faster DNA alignments — A new process pipes bwa-mem2 → samblaster → samtools sort, performing duplicate marking and coordinate sorting as part of the alignment stream rather than through separate post-processing steps.

Improved HLA typing

• More robust HLA allele consensus calling — The immuno module now supports more combinations of HLA typing tools during consensus calling.

• Normalization of user-provided HLA alleles — User-provided alleles are now normalized internally to maximize compatibility with downstream tools.

• Optimized razers3 call — The razers3 process has been updated to support parallel alignments instead of running FASTQs in series.

Expanded QC

• Improved BAM QC — qualimap has been added and now runs on all generated BAMs. This helps validate analysis quality and prioritize tumor antigens.

Tumor characterization

• Added LOH, APPM, and spliceosome mutation reporting — LOH, APPM, and spliceosome mutation status are now computed as part of LENS and reported in the project’s outputs/ directory. LOHHLA data has been moved to a separate sidecar output file.

• Improved purity estimates — The sequenza process has been updated to pull tumor purity from the confints file rather than the alternative_solutions file. This should yield nearly identical purity estimates in most cases while following the recommended output from sequenza.

Antigen-specific updates

• Added viral integration detection — LENS can now optionally detect viral integration sites using DNA and RNA alignment data.

• Improved evidence for splice variants — LENS now uses regtools to detect cis-splice-effect variant associations. These associations are used for downstream pMHC filtering and help reduce false positives.

• Mouse splice variant support (EXPERIMENTAL) — Previous splice-variant tools used in LENS, Neosplice and SNAF, do not explicitly support mouse samples. Our port of SNAF, mSNAF, introduces mouse support, but it has not been thoroughly tested.

• Improved cancer/testis antigen gene set — LENS now uses an updated, annotated CTA gene set generated by Dr. Alex Rubinsteyn.

pMHC prioritization

• vaxrank added — vaxrank (https://github.com/openvax/vaxrank) now runs by default. Relevant columns from vaxrank can be found in the LENS reports.

• Alignment-based QC — Candidate peptides are now further scrutinized using alignment-based information such as read support, read quality, and mapping quality. More information can be found in the LENS evidence review specifications.

• Multiple tiered LENS reports — LENS now emits a base LENS report (with all pMHC candidates), a “reviewed evidence” LENS report with deeper QC for each candidate, and a high-confidence LENS report containing pMHCs that pass high-confidence filters. More information on these reports can be found in the documentation covering LENS report interpretation.

Interactive reporting

• Interactive web UI for results review — RAFT’s generate-reports function now provides users with an interactive web-based UI for reviewing sample-level QC and pMHC candidates, comparing candidates across timepoints or treatments, and generating PDF reports.

Improved documentation

• Improved documentation flow — LENS documentation now provides users with a more direct path to installing and using RAFT for running the LENS workflow.

• Interactive LENS flowchart — An interactive flowchart showing the default path through the LENS workflow has been added.

• Code status page — Like RAFT, LENS now provides an automatically updated code status page for modules it uses.

Licensing updates

• RAFT, LENS, and all modules maintained as part of these projects are now licensed under the Apache License 2.0.

We put a great deal of effort into these releases, and we hope that effort is reflected not in the length of the changelog, but in how much easier RAFT and LENS are to use. If you try them, we’d genuinely appreciate hearing what works, what doesn’t, and what would make them more useful for your research.

We would also like to thank Brad Nelson of Pathfinder Oncology for his many contributions to both RAFT v1.5.1 and LENS v1.9.0.

If you have questions, comments, or suggestions, please contact Steven Vensko II at steven_vensko@med.unc.edu or join us on the RAFT Slack server.

Antigen Selection

1. Start with a good open-source antigen prediction pipeline (rather than developing a new pipeline in-house)

Rationale: Predicting tumor antigens from genomics data seems deceptively easy, e.g., just call some variants from whole exome sequencing, look for expression of these in RNA-seq data, predict HLA binding with netMHCpan or MHCFlurry, and rank predictions by binding. That’s what was done in the first papers describing a neoantigen prediction workflow (and is what many groups still do). However, the vast majority of predicted neoantigens are neither presented by HLA nor immunogenic in vivo, so the nuances of which features to capture from the genomics data, filtering heuristics, and ranking heuristics can matter quite a bit. Although there are multiple pipelines reported in the literature, most of these are proprietary and/or underspecified in methods. There are three main open-source neoantigen prediction workflows to choose from, deploy without much friction, and build upon (pVACtools, OpenVax, LENS). The most widely used in clinical trials so far has been pVACtools (GitHub repo, paper), which was developed by Malachi and Obi Griffith’s lab at Washington University in St. Louis and has supported the WashU and Jamie Leandro Foundation vaccine trials. OpenVax (GitHub repo, paper) was developed by Alex Rubinsteyn, Tim O’Donnell, and Julia Kodysh while working in the Hammerbacher lab at Mt. Sinai in New York, supporting the Mt. Sinai trials run through Nina Bhardwaj’s group. LENS (GitHub repo, paper) is our software, developed to allow prediction of tumor antigens from genomic sources beyond small somatic mutations, harmonize RNA expression across genomic sources for antigen prioritization, capture features important for immunogenicity prediction, and be transparent, reproducible, and extensible based on containerization. So, depending on what you want to do, you could leverage the development time, testing, and prior FDA review of any of these pipelines.

Counterpoints: If you have special insight into genomics prediction of tumor antigens in general or for a subclass of antigens, especially if you have novel bioinformatics methods required for antigen prediction, you’d want to incorporate these. In that case, you might still want to build on one of the open-source workflows given they’ve already been reviewed by the FDA and their libraries could speed development time. You may also have collaborator(s) in academia or industry with proprietary pipelines that you’d like to use. If so, I would benchmark these against one or more open-source pipelines first to ensure good performance.

2. Consider including expressional tumor antigens beyond genomically templated sources (e.g. alternative splice variants, endogenous retroviral elements, and cancer/testis antigens) but use only those with minimal expression in medullary thymic epithelial cells

Rationale: Even in tumors with a high mutation rate, only a minority of predicted neoantigens are presented by cancer cells in vivo, and only a minority of those are immunogenic. In some tumor types, somatic mutations are rare. Optimal antigen selection may include expressional antigens as well as antigens derived from somatic mutations. You can use publicly available RNA-seq data from medullary thymic epithelial cells to exclude expressional antigens that are expressed highly in mTECs (which would be less likely to be immunogenic given negative selection in the thymus).

Counterpoint: Using only neoantigens derived from somatic mutations allows confidence in tumor-specificity of the specific antigen peptide(s). Splice variants, CTAs and ERVs may be expressed by normal tissues, potentially causing toxicity if the vaccine elicits potent antigen-specific T cell responses. That said, using publicly available genomics datasets to confirm tumor-specificity may attenuate this risk.

3. Use long-read as well as short-read sequencing if possible to capture larger somatic mutations

Rationale: Long-read whole genome and whole transcriptome sequencing (e.g., Pacific Biosciences or Oxford Nanopore) allow identification of germline and somatic structural variants that cannot be identified by short-read sequencing (e.g., Illumina) alone. Large somatic variants may elicit more immunogenic neoantigens than single-nucleotide variants, as they contain longer stretches of non-self amino acid subsequences (i.e., look more like viruses than self). Our workflow for long read data analysis (Exacto) is here: GitHub repo / blog post.

Counterpoints: Long read sequencing is expensive and requires fresh or frozen tissue for high-molecular weight DNA and RNA extraction. For these reasons, it may be logistically challenging. It may be the case that most large structural variants can be resolved with short read sequencing (unclear as robust benchmarking has not yet been done). No study has yet shown an example of an immunogenic neoantigen that could be detected by long-read sequencing but not short-read sequencing.

4. Validate antigen presentation as best you can using immunopeptidomics studies

Rationale: Only a minority of predicted neoantigens are presented by HLA in vivo. The true positive rate of prediction is probably 2-20% depending on the tumor and genomics/bioinformatics workflow, so antigen selection using genomics alone would yield mostly false positives. Mass spectrometry based immunopeptidomics can be used to confirm HLA presentation of tumor antigen peptides (examples: 1, 2, 3). Another option would be to validate predicted tumor antigens by looking for antigen-specific T cell responses prior to vaccination (presuming they would only be present if the tumor antigens were presented in vivo). The downside of this second option is that you would be measuring the cancer’s immunological history, not necessarily generating evidence that the cancer cells were presenting the antigen(s) at the time of sample collection.

Counterpoints: Like extraction of high-molecular weight DNA and RNA for long-read sequencing, immunoprecipitation of peptide/HLA molecules for immunopeptidomics currently requires fresh or frozen tumor samples as the input material. Sensitivity of shotgun immunopeptidomics studies is lower than you’d like, and targeted immunopeptidomics is prohibitively expensive to screen hundreds of peptides per patient. Even if you integrate immunopeptidomics into your integral biomarker for antigen selection, you will likely need to allow for cases where there are insufficient numbers of validated tumor antigens to choose from. You could use immunopeptidomics for antigen prioritization rather than making it a hard filter on antigen selection.

5. Include validated antigens such that the vaccine antigen set covers all tumor subclones

Rationale: Failure to eliminate all tumor subclones will select for escape variants that are resistant to T cell populations elicited by the vaccine (due to lack of relevant antigen presentation). You can use single cell sequencing studies to define the clonal landscape of the tumor and select an antigen set that covers all subclones. Our software for data analysis for this is here (github, blog post).

Counterpoints: Single cell sequencing requires capturing fresh or viably cryopreserved tumor cells, which may be logistically challenging. Antigen-directed immunotherapies that target a single antigen, or a set of antigens that do not cover all tumor subclones, may still be clinically effective if they induce epitope spreading and/or are part of a combination therapy strategy where other drugs can eliminate the resistant subclones.

6. Select validated antigens where the set covers HLA proteins encoded by alleles on both chromosomes

Rationale: Loss of heterozygosity of HLA (LOHHLA) is somewhat frequent mechanism of resistance to T cell-directed therapies. LOHHLA can happen without much cost in selective fitness of the cancer cells, however given that more extensive loss of HLA can trigger NK cell-mediated cytotoxicity, and a gene coding for a core subunit of RNA polymerase I lies within the HLA Class I loci, loss of both large chromosome segments containing all classical HLA class I loci has a much higher fitness cost. You can hedge against LOHHLA in vaccine design by including antigens presented by HLA proteins encoded by HLA alleles on both chromosomes. The original LOHHLA workflow is here (paper, github), and a variant of this is implemented in LENS.
Counterpoint: The optimal quantitative trade-offs between selecting antigens presented on HLA across both chromosomes and selecting optimal antigens regardless of HLA are unknown.

7. Select as few tumor antigens as possible (subject to the above constraints)

Rationale: Responses to potentially immunogenic antigens may be attenuated by competitive inhibition when multiple antigens are included in the vaccine (i.e., immunodominance amongst the antigen set). Selecting as few antigens as possible may minimize competitive inhibition of immunogenic antigens.

Counterpoints: The optimal quantitative trade-offs between increasing the frequency of immunogenic antigens in the vaccine (by selecting more antigens) and reducing antigen competitive inhibition (by selecting less antigens) are unknown. You could hedge risk here by allowing changes to antigen inclusion over time based on elicitation of antigen-specific T cell responses and tumor coevolution/immune escape.

Vaccine Formulation

8. Use an mRNA/LNP formulation if possible

Rationale: The only randomized clinical trial which has reported a benefit of neoantigen vaccination (in this case given with anti-PD-1 vs anti-PD-1 only) is the Merck/Moderna adjuvant melanoma trial that used an mRNA/LNP vaccine. We have some preclinical data in mice that mRNA/LNP vaccination may be superior to peptide/adjuvant vaccination, with more benchmarking ongoing (although adjuvant data are difficult to interpret translationally due to differences between innate immune signaling in mice vs humans). There are also practical advantages of an mRNA formulation: faster synthesis, no loss of antigens to synthesis failure based on hydrophobicity and cysteine content, and more flexible combination with other therapies (deliverable via LNPs).

Counterpoints: Peptide/adjuvant vaccines have a longer history of clinical trials and may be cheaper and easier to access for now. mRNA and LNP design and manufacturing are encumbered by IP constraints, which may matter if you are looking for investment or commercial returns from your vaccine platform. If you use a peptide vaccine, consider using multiple adjuvants (along with immune checkpoint inhibition) for potentially greater efficacy. DC vaccines may be more potent, but they are expensive in time and money to manufacture and as such are difficult to scale.

Vaccine Dosing

9. Give the vaccine in the context of combination immunotherapy

Rationale: Eliciting tumor antigen-specific T cells in vivo is not enough to cure people. As best you can, you also want to reverse or preempt resistance mechanisms. The best place to start is probably giving the vaccine with PD-1 inhibition (based on results from the Merck/Moderna melanoma study), however additional resistance mechanisms are probably applicable in your tumor of interest, and you may wish to target these as well.

Counterpoints: Increasing complexity of the intervention means increased cost in time and money for the trial. For anti-PD-1 specifically, this may interfere with establishment and maintenance of antigen-specific T cell memory (even as it improves effector function in the short term) (paper).

Correlative Studies

10. Measure everything you can that may be a resistance mechanism to see which ones occur and at what frequency

Rationale: For a therapeutic vaccine to work clinically, you need all of the following biology: (1) Antigen targets that are presented by HLA, (2) Enough peptide/HLA complexes on the cancer cell surface to support effective targeting by T cells, (3) Elicitation of enough T cells specific for target antigen(s) in the body to outcompete the number of tumor cells, (4) Achievement of T cell ingress into tumors, (5) Capacity of elicited T cells to kill cancer cells in the tumor microenvironment (including proper tuning of T cell-intrinsic phenotypes and subversion of intratumoral T cell suppression and evasion), and (6) Capacity to prevent or resolve acquired resistance mechanisms that develop under evolutionary selection pressure induced by your therapy. If a patient does not have a clinical response, then any of the above could’ve failed. It’s important to understand the which failure modes are common, how common they are, and whether they can be intervened upon to improve cure rates. If you have research biopsies pre- and post-therapy along with serial peripheral blood samples, you can evaluate these.

Counterpoints: Correlative studies are expensive, and the space of potential resistance mechanisms is huge, so evaluating them would be exploratory at first. The extent to which active resistance mechanisms are shared among patients is unknown, so the best you may be able to do in a small trial is categorize likely candidates and show they lead to T cell dysfunction and/or evasion in ex vivo studies (rather than causally linking them to lack of clinical response in specific patients). Best would be to embed these studies in a context of iterative refinement of treatment strategies, like the n-of-1 approach described in Sid Sijbrandij’s “Going Founder Mode on Cancer.”

Tumors are composed of complex cell populations consisting of several subpopulations (subclones) with distinct genotypes and transcriptional phenotypes. Strategies targeting tumor antigens with the highest population prevalence (i.e. clonal or near-clonal) intuitively seem ideal, but deciphering the complicated subclonal architecture within a tumor with the generally available bulk DNA and RNA sequencing can prove challenging. Bioinformatic tools exist to aid in estimating somatic variant cellular prevalence (i.e. clonality) using bulk DNA sequencing (e.g. Sequenza and CNVKit), but estimating cell-level transcriptional activity cannot be done reliably from bulk RNA sequencing.

To better understand why, let’s imagine a scenario where we have gene-level count data from a tumor generated using standard bulk RNA sequencing. We peruse the data and notice one of our favorite CTAs has elevated expression relative to our expectation. Unfortunately, with bulk sequencing, each cell’s contribution to the transcriptional profile is conflated into a single observation and one cannot tell whether a singular tumor cell is transcribing a seemingly infinite amount of that CTA or if every tumor cell is expressing your CTA of interest. The truth likely lies somewhere in between these extremes, but bulk RNA sequencing fails to establish where.

Single-cell RNA sequencing (scRNA-seq) provides the resolution required to deconvolute cellular expression patterns, but most scRNA-seq protocols utilize Illumina-based short-read sequencing and only sequence the (5’ or 3’) end of genes. The resulting gene counts provide unprecedented insights into tumor biology on a cellular level, but this limits our ability to interrogate sequence-level variation within gene bodies. This limitation prevents the use of short-read scRNA-seq to estimate the tumor heterogeneity of most tumor antigens (e.g. single nucleotide variants, fusions, insertions, or deletions). A small but growing class of assays is extending the utility of single-cell data through the utilization of long read sequencing. To address this, we created a tool to quickly turn scLRRNA-seq (single-cell long-read RNA-seq) VCFs and gene count matrices into an improved understanding of intratumoral heterogeneity. TAHP can be found at github.com/pirl-unc/tahp.

Installation

TAHP requires both Conda and Singularity/Apptainer for installation.

Users can install and run TAHP by running the following commands:

git clone <https://github.com>:pirl-unc/tahp.git; cd tahp; conda env create -f environment.yml -y; conda activate tahp; singularity exec docker://spvensko/tahp_data:latest sh -c “cp -r /refs .; cp -r /test_data .”

This command will:

• Clone the TAHP repo locally

• Create the tahp Conda environment

• Activate the tahp Conda environment

• Download the relevant references files and test data

After initial installation, users can run TAHP by activating the tahp environment and running the ./tahp command.

Demonstration and Outputs

To demonstrate TAHP’s utility in estimating tumor antigen heterogeneity, we’re running it on a primary melanoma sample from GSE212945 (https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE212945), a dataset generated for Shiau et al., 2023 (https://www.nature.com/articles/s41467-023-39813-7). The original analysis by Shiau et al. focused on better understanding the tumor microenvironment (TME), but we will be focusing on expression patterns of tumor antigens within the tumor cells.

We’ll specifically be using sample GSM6564304 by running the following command (more information regarding inputs and parameters can be found below):

./tahp -s tahp_demo_sample -g test_data/sclrRNA-seq/GSM6564304.gene_exps -h HLA-A*02:01 -v test_data/sclrRNA-seq/GSM6564304.vcf.gz

TAHP will produce both an outputs/ and a plots/ directory. Users will find a TSV in the outputs/ directory named tahp_demo_sample.pmhc_coverage.tsv containing high scoring pMHCs observed within the sample’s tumor cells along with information regarding tumor cell coverage. Here’s an example of that output:

Let’s break down what this file’s telling us. The columns are:

The pmhc column contains a variety of information related to potentially targetable pMHCs. Let’s consider an example:

PRAME/TLAKFSPYL/HLA-A*02:01/Affinity:12nM/Presentation:91.8%

The above /-delimited line is jam packed with information. Let’s split it up into its components and go over them:

PRAME - The target generating the listed pMHC. This could be a SNV, CTA, or ERV.

TLAKFSPYL - The peptide portion of the pMHC

HLA-A*02:01 - The allele portion of the pMHC

Affinity:12nM - The pMHC binding affinity (per MHCflurry)

Presentation:91.8% - The pMHC presentation score (per MHCflurry)

The target identifier varies by antigen source. CTAs have their gene symbol (e.g. PRAME) listed, SNVs entries are described as chr:position:ref:alt (So the pMHC in the table above is a C → A substitution at the 55,726,929th position of the twelfth chromosome. ERVs are described by the chromosome and position of their MetID from their gEVE classification.

Next, let’s consider the contents of the file. Each row represents a pMHC derived from either a CTA, ERV, or SNV. The CTA and ERV entries must be expressed at a pre-defined expression threshold (user adjustable — see section below on inputs) across a pre-defined proportion of tumor cells (also user adjustable — see section below on inputs). SNV entries must have a cellular population allelic frequency < 90% (following methods of Shiau et al., 2023) and must have a genotype in at least 50% of cells. SNVs are not explicitly filtered using a specific expression threshold; however, we make the assumption that a SNV reliably detected using single-cell RNA sequencing must be conditionally sufficiently expressed for detection.

Notably, the order of rows is meaningful! The first row is the pMHC with the highest tumor cell population expression coverage and the subsequent rows are pMHCs that, as determined through a greedy algorithm, cover the highest proportion of tumor cells not already covered by a previously chosen pMHC(s). So, looking at our table above, we see that the SNV chr12.55726929.C.A covers the most tumor cells at 299 (56.1%) of tumor cells. Next, the ERV chr21.27791519 is listed as it increases the number of covered tumor cells by 49 additional cells (9.2%) for a total of 348 tumor cells covered. This algorithm continues until it has included every high scoring pMHC observed in the sample.

The information in this table is visualized in plots/tahp_demo_sample.tumor_het.pmhcs.pdf (Figure 1):

Figure 1. pMHC coverage across tumor cells in GSM6564304.

A network graph of pMHC combinations (Figure 2) is also produced as plots/tahp_demo_sample.tumor_antigen_network.pdf. The nodes in the network graph are functionally pMHC barcodes in which each potential pMHC is presented as either 1 if observed within a cell and 0 if not observed. For example, let’s consider a scenario where only two pMHCs are observed, one from the CTA PRAME and another from CTA MAGEA4. In that case,

• A node of tumor cells expressing both PRAME and MAGEA4 would have a barcode of 11

• A node of tumor cells expressing PRAME but NOT MAGEA4 would have a barcode of 10

• A node of tumor cells not expressing PRAME, but instead expressing MAGEA4 would have a barcode of 01

• A node of tumor cells expressing neither PRAME nor MAGEA4 would have a barcode of 00

The edges connect nodes that are a hamming distance of 1 apart (in other words, nodes that differ by one position between their barcodes). For example, 11 would be connected to 10 and 01, but not 00.

Figure 2. Network plot of pMHC combinations among tumor cells in GSM8330680.

The Nuts and Bolts

Next, let’s go over the technical stuff. We’ll cover the input parameters first, then cover the algorithm being used.

TAHP can accept the following parameters:

TAHP, by default, has two required inputs — a scRNA-seq generated gene count matrix and a single-cell VCF. Generating this matrix and VCF (e.g. filtering criteria, normalization techniques, etc.) is up to the user (and users should be cognizant or how different inputs will influence the resulting outputs). Notably, our demonstration gene matrix and VCF were generated through scNanoGPS‘s reporter_expression.py and reporter_snv.py scripts using a modified GTF file amended with ERV coordinates from the gEVE database (https://pmc.ncbi.nlm.nih.gov/articles/PMC4885607/).

The gene matrix should be a TSV where rows are gene symbols and columns (after the initial descriptive columns) are cell barcodes. An example is available in the TAHP github repository and a minimal example is shown below.

The VCF should resemble a multisample VCF where each sample corresponds to a cell barcode:

The TAHP algorithm can be broken down into a few discrete steps:

Availability

The TAHP repository, including reference files, test data, and a Dockerfile describing dependencies, can be found at github.com/pirl-unc/tahp.

Caveats and Considerations

As mentioned previously, TAHP is a relatively simple tool and, as a result, there are several things users should keep in mind while running it:

TAHP does not sanity check input gene matrices

TAHP will consume gene matrices and VCFs as provided without performing any QC, applying any corrections, or normalizing any data. As a result, it’s possible users may get unexpected results if high quality data are not provided. For example, a gene matrix and VCF from a low read sequencing run will result in severe underestimation of pMHC coverage.

Tumor cell coverage is likely an underestimate

One feature of single-cell RNA sequencing data is the general sparsity observed with respect to gene-level expression. This, along with the old adage “absence of evidence is not evidence of absence”, suggests there are likely many cases where TAHP is reporting no expression of a specific CTA, ERV, or SNV in a specific cell when they may in fact be expressed within that cell, but missed for technical reasons.

pMHC characterizations aren’t perfect

Tools used to estimate a peptide’s binding affinity to a given HLA molecule, such as MHCflurry and NetMHCpan, can provide fantastic insights used to guide wet-lab validation experiments. Still, these estimated binding affinities should only be used as a guide and not as evidence that a peptide can be expected to be presented and immunogenic.

Future Directions

Single-cell RNA sequencing has become increasingly popular since its introduction due to the sheer number of questions it can address where bulk sequencing simply isn’t applicable. We hope that TAHP serves as a springboard for further investigation into tumor heterogeneity.

TAHP is being released as a BASH script which offers both pros (portability, ease of use) and cons (limited error detection and mitigation). It also requires users to generate inputs with scNanoGPS which can be an involved process. We expect TAHP to be implemented as part of an end-to-end RAFT workflow (similar to LENS) in the near future.

Thanks for your interest in TAHP! Feel free to leave any comments or questions you may have.

• My last post described a universal grammar for integrating different types and descriptions of DNA variants (“Occam’s Variant Grammar”).

• In this post, I expand Occam’s Variant Grammar to also represent RNA variants, enabling DNA-RNA integration and amino-acid level traceability essential for telling which parts of a protein are tumor-specific (and can safely be targeted).

• This abstraction is embodied in a new tool called Exacto, which identifies somatic and germline DNA variants, RNA variants, integrates DNA and RNA evidence, and characterizes mutant peptide sequences with each amino acid traced to its RNA and DNA origins. Exacto is available here: https://github.com/pirl-unc/exacto

1. Previously On “A Unifying Grammar for DNA Variants”

In my earlier post, I proposed a new variant grammar that conceptualizes DNA variant types in a variation graph. The Occam’s Variant Grammar unifies representation of major DNA variant types using just two atomic operations: add and connect. It simplifies the growing list of DNA variant types into just two graph operations (upstream or downstream) on a variation graph.

2. Lost in Translation

In the context of neoantigen discovery for personalized immunotherapy development, simply knowing which DNA variants are present is not enough. It is equally critical to identify RNA variants, determine whether they incorporate or arise from DNA alterations, and predict mutant peptide sequences while contextualizing the underlying RNA and DNA changes.

However, we see a fragmentation of computational methods for identifying variants at each level of the central dogma. Developers often focus their new software on identifying a subset of major variant types. Some tools are limited to detecting small variants such as SNVs, insertions, and deletions typically smaller than 30 base-pairs (bp). Others only profile larger (30bp+) structural variants. Layered on top of this is the further divide between tools that identify DNA variants and those that characterize RNA variants. These compartmentalizations typically arise as a by-product of the historical limitations inherent in different sequencing technologies and specialization of lab expertise.

While computational methods for DNA variant detection have matured over the past two decades, RNA variant detection remains relatively underdeveloped. This is largely due to the complexity introduced by splicing and the limitations of short-read sequencing technologies. Compounding the challenge is our incomplete understanding of the splicing process itself. One of the most advanced splicing prediction models to date, SpliceAI, remains a black-box model.

Long-read RNA characterization tools such as FLAIR and SQANTI only focus on identifying RNA variants, especially splicing aberrations and not sequence variants (e.g. SNVs, insertions, deletions, and breakpoints). They also do not integrate RNA variants with DNA variants either. Their outputs and usefulness have functional limitations when you try to apply them in the personalized immunotherapy development setting. A common roadblock in their outputs is, so what are all the underlying characteristics of the variant transcript as they relate to a mutant peptide sequence?

As the conventional wisdom goes, many a times “the whole is more than the sum of its parts.” That is, to know what an elephant is, one ought to see the whole elephant. Then, the functions of the trunk, leg, and ear make sense in the context of the entire system. We want to make it make sense - a simple yet powerful adage that my advisors Ben and Alex have instilled in our lab.

Personalized immunotherapy targets, or neoantigens, are tumor-specific to selectively eliminate cancer cells while minimizing autoimmune responses, thereby optimizing therapeutic efficacy. In the Make it Make Sense framework, this means that each neoantigen sequence should be traceable to a tumor RNA variant and, in turn, back to a somatic tumor DNA variant, with compelling evidence established at every level. It follows that a computational tool that accomplishes this inherently integrates data across the DNA, RNA, and peptide levels.

While tools such as RegTools, NeoDisc, and moPepGen predict mutant proteoform sequences by DNA and RNA variant integration, they were designed for short-read sequencing data. They focus on small DNA variants and do not take advantage of long-read sequencing technologies. Most importantly, they do not annotate how each amino acid in each proteoform was affected by the underlying DNA and RNA variant nucleotides. Moreover, short-read RNA sequencing cannot be reliably used to resolve splicing complexities and is prone to generating false positive peptide sequences.

All in all, fragmentation of variant types in variant calling together with a dependence on short-read data may be leading to loss of clinically significant translational information.

3. Found in Full with Long-read Sequencing

It is important to highlight that our lab’s direction and attempt to increase the sensitivity in mutant peptide sequence prediction for personalized immunotherapy development is fundamentally enabled by long-read sequencing. In particular, the high fidelity full-length isoform sequencing made possible by Pacific Biosciences enables an unprecedented capability to sequence most mammalian messenger RNA (mRNA) molecules end-to-end; I do not have any conflict of interest. Indeed, it is still the case that very long genes such as TTN (over 100,000 nucleotides long) cannot be fully captured at high (over 99%) base accuracy rates. Nevertheless, most mRNAs in the human transcriptome can be profiled end-to-end using the latest long-read isoform sequencing that can generate reads over 15,000-bp in length (Figure 1). This means that we can read the full length of a mutant messenger RNA that originates from one or more somatic DNA variants.

Compared to short-read sequencing, long-read sequencing provides two major advantages - one at the genome level and the other at the transcriptome level.

First, at the genome level, long-read sequencing enables the accurate resolution of both small and structural variants - think of a 500-bp insertion. While earlier studies, such as that by Choo et al. suggested that many large structural variants could be identified using short reads, their evaluation used a generous threshold. If a short-read derived variant call landed within 1,000 bp of the ground truth location, it was considered a match. That may be sufficient to detect the presence of a large structural variant, but personalized immunotherapy requires much more. We need to pinpoint the exact breakpoints, where a variant begins and ends, so that we can infer its consequences at the amino acid level. It’s one to thing to know that something exists. It’s another to know exactly what it is.

Second, at the transcript level, long-read sequencing enables precise assembly of transcript models. Transcriptome assembly using short reads typically generates a complex graph that generates false positive transcripts. Long-read sequencing, on the other hand, gives you readouts of the same isoform multiple times end-to-end. This reduces ambiguity around alternative splicing and enables RNA variant calling to be anchored by a more accurately observed transcript. Furthermore, having an accurate, observed transcript allows for direct comparison against known reference transcripts. This is an essential step for annotating where RNA variants begin and end so that each mutant amino acid can be traced back to each RNA codon as well as to each DNA nucleotide.

All in all, this is what long-read DNA and RNA sequencing enable us to do:

Of note, the central dogma above isn’t unique to immunogenomics. It is a guiding principle across genomics as a whole.

4. Occam’s Variant Grammar for RNA Variants

To make sense of RNA variants in the context of DNA variants, we turn back to the Occam’s Variant Grammar. Here, I expand it to allow for representation of RNA variants:

An RNA variant can be described as:

where:

Variant Node

A variant node can either be an empty sequence (i.e. deletion) or an insertion sequence (i.e. single-nucleotide variant, multi-nucleotide variant, or insertion).

Graph Operations (Edge Orientations and Flag Types)

Similar to the DNA variant, connections between a new RNA variant node and the reference genome backbone can be made by the following orientations:

• Upstream (towards 5’ end in the coding sense of the messenger RNA)

• Downstream (towards 3’ end in the coding sense of the messenger RNA)

Reference positions can be flagged to indicate splice aberrations. A flag can be one of the following types (Figure 2):

• Include
  Include the reference position; intron retention or cryptic exons.

• Mark

  Mark the reference position; non-canonical splice site.

• Skip
  Skip the reference position; exon skipping or truncation.

Note that both Include and Skip flags inherently indicate non-canonical splicing.

Just like a DNA variant, an RNA variant can be encoded as:

rna_variant = {
    chromosome_1:  <reference chromosome>,
    position_1:    <reference position>,
    operation_1:   <downstream, upstream, include, mark, OR skip>,
    chromosome_2:  <reference chromosome>,
    position_2:    <reference position>,
    operation_2:   <downstream, upstream, include, mark, OR skip>,
    sequence:      <variant sequence>
}

where sequence represents the variant node while operation_1 and operation_2 represent the edge types (upstream or downstream) or flag types (include, mark, or skip). The reference positions denote where the edges should be created on the reference genome backbone OR the start and end of flagged reference positions.

Here’s a graphical overview of how this variant grammar allows us to represent the following major RNA variant types:

• Circular RNA

• Cryptic exon

• Deletion

• Duplication

• Exon truncation or skipping

• Fusion gene (two or more known genes)

• Insertion

• Intergenic RNA

• Intron retention

• Inversion

• Single-nucleotide variant

• Multi-nucleotide variant

• Translocation

Note that the representation for deletion, duplication, insertion, inversion, SNV (including A-to-I RNA editing), and translocation are the same as those described in an earlier blog post. Here, we define any combination of the above RNA variant types to be characteristic of a novel, or more accurately, variant transcript.

Here are the RNA variant types represented in the Occam’s Variant Grammar (Figure 3).

5. Exacto Performs DNA and RNA Variant Calling and Integration

Exacto (EXacto Accurate Characterization of Transcriptomes and genOmes) identifies genomic and transcriptomic variants from reference-aligned long-read DNA and RNA sequences.

It performs five main tasks using long-read sequencing data:

1. Identification of germline and somatic (case-specific) DNA variants.

2. Identification of RNA variants.

3. Integration of DNA and RNA variants.

4. Translation of full-length RNA sequences.

5. Tracing of each in silico translated amino acid to its RNA and DNA origins.

It is available here on GitHub: https://github.com/pirl-unc/exacto

DNA Variant Calling

The unified DNA and RNA variant grammar structure, that is the Occam’s Variant Grammar, drives the variant identification algorithm at the genomic and transcriptomic levels. The core parts of Exacto are written in Rust for speed.

Exacto parses the alignment map information such as CIGAR as well as CS and SA (Supplementary Alignment) tags to detect variant records. The variant records are subsequently clustered using union-find to identify variant calls (Figure 4).

Prior to variant calling, Exacto creates an alignment structure for each read. This is an important preprocessing step as I’ve noticed that the following occasionally happens in alignment, leading to missed detection of insertions (Figure 5).

To resolve such cases where there are overlapping alignments or unaligned sequences, Exacto first builds an alignment structure for each long read. That is, Exacto collects all alignment records for each read from the Binary Alignment Map (BAM) file and then creates a map between each nucleotide in the original sequencing read and each nucleotide in the reference genome. This is a crucial preprocessing step as it enables us to identify otherwise hidden insertions between breakpoints - viral integrations for example.

RNA Variant Calling

RNA variant types are more complex than DNA variant types. While DNA variant types can be inferred directly from the alignment data (i.e. SNVs, insertions, deletions, and breakpoints), RNA variant calling requires an additional step of comparing alignments against reference transcript models. This added complexity arises because DNA is transcribed and spliced to produce RNA molecules. A single DNA region can generate multiple distinct transcripts through alternative splicing and introduce variation that does not exist at the DNA level.

It is important to highlight that splicing “aberrations” can only be identified by interpreting a putative transcript in the context of known reference transcripts. For example, suppose we have three reference transcripts from the same gene and one assembled transcript (Figure 6). Applying Occam’s razor (principle of parsimony) we might conclude that the assembled transcript most closely resembles reference transcript 3 with a truncation in its second exon.

It is important to identify a matched reference transcript because just pooling all canonical splicing sites does not lead to an accurate characterization of the variant transcript. Consider the following example (Figure 7):

While the assembled transcript is different from all three reference transcripts, all of its splice sites (A, B, and C) are found among the pooled set of reference splice sites. It is possible that its second, third, or fourth exon may have originated from reference transcript 2, reference transcript 1, or reference transcript 3, respectively. If aberrant splicing identification only relied on splice sites absent in the pooled set of reference splice sites, the assembled transcript would not be detected as a variant transcript.

There are several ways to compare an assembled transcript against a set of reference transcripts. Here are some approaches that I’ve tested:

where:

When I simulated GENCODE v41 (hg38) protein-coding transcripts with RNA degradation towards the 5’ end, here are the benchmark results I got for the above approaches:

Exacto takes as input a FASTA file of assembled transcripts to identify RNA variants. This design choice was made due to variations in RNA alignments that result from RNA degradation - typically towards the 5’ end of the RNA molecule for polyA tail capture kits.

Exacto applies the same the same variant calling algorithm used for DNA variant calling to detect SNVs, MNVs, insertions, duplications, deletions, and inversions for RNA variant calling. Additionally, it performs splice-aware variant calling by comparing alignments against reference transcripts to identify the following splicing aberrations:

• Circular RNA

• Cryptic exon

• Exon truncation or skipping

• Fusion gene

• Intergenic RNA

• Intron retention

By default, Exacto selects the top 3 reference transcripts for each assembled transcript based on cosine similarity scores.

DNA and RNA Variant Integration

Exacto is designed to identify mutant peptides that result from genomically templated somatic variants. Recall that subsequences of mutant peptides could be loaded on major histocompatibility (MHC) class I molecules, making neoantigens that could be personalized immunotherapy targets. An essential step in this task is inferring which RNA variants, if any, were generated as a result of a somatic DNA variant. At first, this problem might appear to be a splicing prediction problem. Given altered and unaltered DNA sequences, predict differences in their splicing behaviors. However, while splicing prediction tools such as SpliceAI could help to increase our confidence in pairing an RNA variant with a somatic DNA variant, they are not robust in oncology as proteins comprising spliceosomes could themselves be mutated and affect splicing patterns.

In the personalized immunotherapy development setting, we often have both the DNA and RNA sequencing data of the tumor. This allows us to computationally observe any splicing aberrations in close proximity to somatic DNA variants. In fact, the Griffith Lab published RegTools in the past to solve this problem with short read sequencing data.

Here, we extend the idea of genomic distance proximity from RegTools for DNA and RNA variant integration and apply it with advantages offered by long-read sequencing, namely the ability to sequence RNA molecules end-to-end and phase all variants in the sequenced transcript.

Recall that Exacto identifies variants in assembled transcripts. This allows us to work with transcript models, most of which are captured in their full lengths.

Transcript Structure Encoding

A major advantage of identifying RNA variants on assembled transcripts is the ability to contextualize the variants within transcript models.

The proposed Transcript Structure encoding represents the following:

1. The encoding is a combination of transcript bases and events between two bases.

2. An assembled transcript is compared against a set of reference transcripts and any variations between the two are encoded.

3. Each transcript structure annotates each base and any event between two bases.

Conceptually, the encoding retains the reference genome-level comparison features of the cs tag and adds reference transcript-aware annotations. This is a necessary extension for accurate downstream in silico translation at single amino-acid resolution.

An important aspect of the proposed encoding is that it is generated by directly comparing the assembled transcript to a set of reference transcripts (in cases of a fusion gene). Pooling splicing and exonic regions from multiple reference transcripts within the alignment scope can create a false impression of canonical splicing due to overlapping features from alternative isoforms. A one-to-one comparison with a specific reference transcript is therefore essential for accurately identifying mutant peptide sequences.

The Transcript Structure is comprised of 2 types of information:

• Transcript model base: a base in an assembled transcript.

• Transcript model event: an event (e.g. breakpoint, deletion, or splicing) between two bases in an assembled transcript.

Transcript Model Base Kind

A transcript model base is one of the following kinds:

Match: A transcript model base is of the match kind if it matches a base in the reference genome.

Mismatch: A transcript model base is of the mismatch kind if it is a different allele from the reference at the aligned position.

Insertion: A transcript model base is of the insertion kind if it is part of an insertion based on alignment to a reference genome. Soft-clipped bases without supplementary alignments are also considered insertion bases.

Unaligned: A transcript model base is of the unaligned kind if it is not aligned to any part of a reference genome.

Transcript Model Base Context

A transcript model base is one of the following contexts:

Exonic: A transcript model base is of the exonic context if it aligns to an exonic base position of a reference transcript. Bases in the untranslated regions (UTR) are also considered exonic bases.

Intronic: A transcript model base is of the intronic context if it aligns to an intronic base position of a reference transcript.

Intergenic: A transcript model base is of the intergenic context if it aligns to a position in the reference genome that falls outside all annotated genic regions.

Transcript Model Event Kind

A transcript model event is one of the following kinds:

Breakpoint: A transcript model event is of the breakpoint kind when a supplementary alignment exists between two transcript model bases. If the two bases are not directly adjacent, transcript model bases between the two are considered insertion bases.

Deletion: A transcript model event is of the deletion kind if a genomic deletion exists between two transcript model bases.

Splicing: A transcript model event is of the splicing kind if two transcript model bases are separated by an intronic region containing a pair of splice sites.

Boundary: A transcript model event is of the boundary kind when it occurs at either end of a transcript model (i.e. the first or last transcript model base). Boundary events capture exon truncation or skipping beyond the 5’ or 3’ ends of the transcript model (i.e. think of omitted exons in the 5’ end of a transcript model with 5’ RNA degradation).

Transcript Model Event Context

A transcript model event is one of the following contexts:

Canonical Splicing: A transcript model event is of the canonical splicing context if both the donor and the acceptor sites match a known pair of splice sites in the corresponding reference transcript.

Non-canonical Splicing: A transcript model event is of the non-canonical splicing context if either the donor or the acceptor site does not match any known pair of splice sites in the corresponding reference transcript.

Backsplicing: A transcript model event is of the backsplicing context if a breakpoint event loops back to a previously traversed reference base, causing the same genomic position to be covered multiple times. To distinguish this from a duplication event, upstream and downstream exons must be skipped or truncated.

Fusion Gene: A transcript model event is of the fusion gene context when the two bases flanking the event, of either the breakpoint or splicing kind, align to genic regions of different genes.

Transcript Structure Examples

Here are some of the RNA variant types represented in the Transcript Structure tag:

Single-nucleotide Variant

Insertion

Deletion

Cryptic Exon

Exon Truncation / Skipping

Intron Retention

Fusion Gene

Primary Structure Encoding

Finally, to predict a neoantigen sequence we would take the window of a mutant peptide sequence that includes the underlying RNA variants. The Transcript Structure information can be translated in silico while keeping track of any RNA variants. This allows us to encode a primary structure with each amino acid traceable to an RNA variant.

For example, the SNV (ATGCTTGGCCCATTTCCAGACGA) example above would be translated as:

amino_acid|codon|start:end|indices
M|ATG|0:2|0
L|CTT|3:5|0
event=1         <-- canonical splicing
G|GGC|6:8|2
P|CCA|9:11|2,3  <-- SNV
F|TTT|12:14|4
P|CCA|15:17|4
event=5
D|GAC|18:20|6

The exon truncation (ATGCTTGGCCCCTTTGACGA) example above would be translated as:

amino_acid|start:end|base_indices
M|ATG|0:2|0
L|CTT|3:5|0
event=1        <-- canonical splicing
G|GGC|6:8|2
P|CCC|9:11|2
F|TTT|12:14|2
event=3        <-- non-canonical splicing
D|GAC|15:17|4

6. Closing

In the next blog post, I will present simulated benchmark study results on DNA and RNA variant calling as well as predicted mutant peptide sequences in real-world samples found by Exacto and other methods.

7. Acknowledgement

This work was made possible through the generous support of the Jaime Leandro Foundation, a non-profit foundation committed to brining therapeutic cancer vaccines to patients.

I’m a big fan of the movie Tenet, especially the mind-bending concept of temporal pincers. Without giving too much away, let’s just say the plot goes forwards and backwards - kind of like a cancer genome when a somatic inversion rewrites the script.

In the world of bioinformatics, I occasionally come across a temporal pincer in a cancer genome - an inverted DNA fragment. Recently, I rehashed my understanding of head-to-head (5’ to 5’) and tail-to-tail (3’ to 3’) inversion breakpoints. Here, I share my notes on what they are and how to spot them in sequencing alignments. Below are some online resources I’ve checked out to learn about inversions:

• Biology Stack Exchange

• Bioinformatics Stack Exchange

• Wally

2. What is Inversion?

An inversion is a structural DNA variant whereby a DNA fragment is flipped in the opposite direction within the genome (Figure 1).

Notably, the inverted DNA is inserted back into the genome in the reverse-complement direction. This is to preserve the 5’ and 3’ orientation in the DNA double helix structure, as shown above.

Inversions are important to understand because, in cancer, they can be early structural alterations that initiate tumorigenesis.

3. Head-to-Head and Tail-to-Tail Breakpoints

An inversion can be long, long enough that sequencing reads cannot capture the whole event end-to-end. But, what’s happened, happened. Sequencing reads should still be able to observe parts of the inversion. This is where the head-to-head and tail-to-tail breakpoints come in (Figure 2). As shown below, a read that captures the 5’ ends (5’ to 5’) of the inversion generates a head-to-head breakpoint upon alignment to the reference genome while a read that captures the 3’ ends (3’ to 3’) leads to a tail-to-tail breakpoint.

4. Inversion by Sequencing Technologies

Let’s take a look at the inversion alignment patterns between short-read (paired-end) and long-read sequencing data.

Here, I simulated a somatic inversion (chr17:7,672,400-7,682,500) in the hg38 genomic region that encodes the tumor-suppressor gene TP53, and generated both short-read (151-bp) and long-read (~10,000-bp) DNA sequencing data for the same event.

Paired-end (Short-read) Sequencing

Owing to the short read length, short-read paired-end sequencing cannot capture most inversion events end-to-end. As we saw above, this results in observing 2 kinds of read alignments: head-to-head (5’ to 5’) alignment (red) and tail-to-tail (3’ to 3’) alignment (blue) (Figure 3).

Long-read Sequencing

If a long read captures an inversion event end-to-end, the read will have 2 supplementary alignments (soft-clipped) and have 3 alignment records (Figure 4).

5. Inversion Representation

In my previous post, I introduced the Occam’s Variant Grammar. If we identified variants in both the short-read and long-read data, we would have found the following variants encoded in the Grammar:

// 5' to 5' (head-to-head) variant
dna_variant_1 = {
    chromosome_1:  'chr17',
    position_1:    7672399,
    strand_1:      '+',
    operation_1:   'D',      // downstream (towards 3')
    chromosome_2:  'chr17',
    position_2:    7682500,
    strand_2:      '-',
    operation_2:   'D',      // downstream (towards 3')
    sequence:      ''
}

// 3' to 3' (tail-to-tail) variant
dna_variant_2 = {
    chromosome_1:  'chr17',
    position_1:    7672400,
    strand_1:      '-',
    operation_1:   'U',      // upstream (towards 5')
    chromosome_2:  'chr17',
    position_2:    7682501,
    strand_2:      '+',
    operation_2:   'U',      // upstream (towards 5')
    sequence:      ''
} 

To detect the inversion, a variant caller that uses the Occam’s Variant Grammar would look for the above pair of events and classify them as evidence supporting an inversion.

Stay tuned for more on how structural variants are implicated at the RNA and peptide levels.

6. Closing

Haven’t you guessed by now? This whole blog post is a temporal pincer. See if you can spot famous lines from Tenet in this post and let us know in the comments section below! I gave you one to begin with.

“I’ll see you at the beginning, friend.”

7. Acknowledgement

This work was made possible through the generous support of the Jaime Leandro Foundation, a non-profit foundation committed to brining therapeutic cancer vaccines to patients.

PIRL has been working hard to continuously improve our publicly available tumor antigen prediction workflow, Landscape of Effective Neoantigen Software (or LENS). We’ve just released LENS version 1.7, so we wanted to highlight both the more substantial upgrades and the reasoning behind the update. We conclude this blog post with a sneak peek at what else we’re working on.

A quick peek at LENS v1.7’s change log (https://uselens.io/en/lens-v1.7/changelog.html) reveals the following changes (note that the bolded and italicized points are discussed below):

Version 1.7                                                                                         
-----------                                                                                         
• Added HLApollo support
• Added pepsickle support  
• Improved references download script to improve compatibility among LENS versions     
• Fixed input tuples and process blocks for netCTLpan, antigen.garnish, deephlapan, and abra2 (thanks @kevinpryan)
• Improved OptiType process block for speed improvements (thanks @kevinpryan)                                                                           
• Updated Jacquard to 1.1.5                                                                         
• Updated sample delimiters from '*' to '+' to prevent edge case of substring run identifiers                    
  
• Fixed gEVE-related references in download script to circumvent third party server downtime (with permission from Dr. So Nakagawa, PhD.)                                                                                
• Fixed 'grep awk' bug in snaf.nf                                                                   
• Fixed processing of ambiguous amino acids by MHCflurry                                                                                            
• Updated LENSTools to fix lenstools_make_indel_peptides_context bugs                               
• Added preliminary intra-patient inter-group analysis                                              
• Added multipass pMHC characterization to better handle long running tools
• Modified workflow to natively support RNA only patients (no SNV or InDels pMHCs will be reported) 
• Reimplemented support for intermediate file cleaning of trimmed FASTQs and BAMs (using raft.py run-workflow ... --clean-intermediates) -- more information available in documentation and https://tinyurl.com/trick-nf-caching
• Reduced HLA allele consensus support requirement to single tool such that all alleles by all tools will be processed. Users can filter alleles (including by support) in the LENS report.
• Added vcf2maf is run as part of workflow with more stringent variant filtering                    
• Added automatic checking of user-provided metadata files for raft.py run-ots and raft.py run-workflow modes
• Fixed bug preventing intersect variant combining method from working                         
• Fixed LENStools use of bcftools Docker image                                                      
• Added documentation for LENS report columns previously missing from documentation 

This blog post will go into some technical information regarding each point, but we’ve separated each point, so readers can focus on the updates most relevant to them.

Added HLApollo support

We built LENS to be as modular as possible with the expectation of new tools being incorporated. We are continuing our commitment to supporting LENS through inclusion of additional functionality by adding HLApollo. HLApollo is a pMHC presentation predictor produced by Genentech. We were motivated to include HLApollo after a discussion on neoantigen predictors on Reddit (https://www.reddit.com/r/bioinformatics/comments/1kclvc1/neoantigen_prediction_pipelines/) and reviewing the tool’s associated manuscript (https://www.nature.com/articles/s41467-024-54887-7). HLApollo runs by default within version 1.7 and columns should automatically be included in the LENS report.

Specifically, the HLApollo outputs for each pMHC can be found in the columns:

• hlapollo_train_allele

• hlapollo_mhc_pred_0

• hlapollo_mhc_pred_0_rank

Note that HLApollo does not use a versioned release system, so a tool version is not indicated in the columns.

If you are aware of a tool that may be worthy of including in LENS then please do not hesitate to reach out and let us know. We cannot promise to support every suggested tool, but are definitely interested in our users’ thoughts.

Added pepsickle support

MHCflurry provides pMHC-level binding affinity estimates and both MHCflurry and the recently added HLApollo provide pMHC-level presentation scores. These metrics, among others, are relevant in prioritizing candidate peptide targets. One metric that was previously missing from LENS was peptide cleavage. Peptide cleavage is relevant to how a peptide is processed by the proteasome and (potentially) presented on the tumor cell surface. Of course, a peptide that has a predicted high probability cleavage site within it should be deprioritized as its internal cleavage would prevent its presentation. On the other hand, a peptide with low cleavage probabilities among its amino acids and with a high cleavage probability for the amino acid immediately downstream of the peptide would make a better target. With this in mind, we’ve generated a score using the pepsickle outputs to aid in prioritizing pMHCs. The score is defined as:

Pr(cleavage of downstream AA) * (1 - Max(Pr(cleavage within peptide)))

This score is the probability of cleavage between the peptide of interest and the neighboring downstream amino acid multiplied by 1 minus the maximum probability of cleavage among amino acids within the peptide. An ideal case involves a high probability of cleavage between the peptide and the neighboring downstream amino acid with minimal probability of cleavage among amino acids within the peptide of interest. This scenario should result in a score near 1.

This score, and other relevant pepsickle information can be found in the following columns from the LENS report:

• pepsickle-0.2.1_max_pep_aa_cleav_score

• pepsickle-0.2.1_plus1_aa_cleav_score

• pepsickle-0.2.1_score

Improved references download script to improve compatibility among LENS versions

We want LENS to be as painless to run as possible. This mentality is what drove the run-demo and run-ots RAFT modes which allow users to, theoretically, go from manifest to LENS report without any further user input.

An important step in the run-demo and run-ots modes is checking for the required reference files in the local RAFT installation and downloading any missing reference files. This system saves the user from manually fetching and generating the required references.

Earlier iterations of this system made assumptions regarding the relationships among files. For example, if RAFT detects File_A and RAFT knows that File_A and File_B are downloaded concurrently, then it assumes File_B is also available in the RAFT references/ directory. This system has historically worked well, but its flaws are becoming apparent as new LENS versions are released. Let’s consider an example — let’s say a new, shiny version of LENS has been released. The new version requires File_A, File_B, and a new file, File_C. If you have run the previous version of LENS (which required only File_A and File_B) then the downloading of File_C will be skipped due to RAFT detecting File_A. This will, of course, result in LENS failing with an error about a missing reference file.

We have revamped the reference download script to prevent the above issue going forward. Effectively, every required reference file will be individually checked (and downloaded if needed) which should prevent these errors going forward.

Fixed gEVE-related references in download script to circumvent third party downtime (with permission from Dr. So Nakagawa, PhD.)

One of the several potential tumor antigen sources that LENS can predict are endogenous retroviruses (ERVs). ERV detection is based heavily upon the computationally predicted ERV coordinates made available through the gEVE database (http://geve.med.u-tokai.ac.jp/). Readers who clicked on the gEVE link in the previous sentence should receive a 404 error (at least, as of June of 2025). This downtime is apparently due to a cyberattack targeted at Tokai University. Unfortunately, these files are essential for running LENS and their unavailability was preventing new users from running LENS. We requested permission from Dr. So Nakagawa (gEVE PI) to independently host the relevant files to allow users to continue using LENS. Luckily, we were granted permission and are temporarily hosting the files on Gitlab.com.

This occurrence reminds me of a conversation I had with Alex Rubinsteyn many years ago in which he expressed concern about external, third-party resources disappearing from lack of relevant funding or a variety of other reasons. I didn’t feel as strongly about the topic as Alex did, but, with hindsight, perhaps I should have. I think this lesson is relevant to all bioinformaticians out there — consider locally caching third-party references that your workflows rely upon.

Fixed input tuples and process blocks for netCTLpan, antigen.garnish, deephlapan, and abra2 (thanks @kevinpryan)

We want to thank Kevin P. Ryan for fixing some of the broken input tuples for netctlpan, antigen.garnish, deephlapan, and abra2. Specifically, Kevin fixed cardinality issues with the input, output, and script blocks.

Updated sample delimiters from * to + to prevent edge case of substring run identifiers

A major aspect of RAFT (and by extension, LENS) beyond reproducibility is output file sample provenance tracking. For example, if a user encounters a VCF file of expression-filtered somatic variants (as in, the VCF only contains a subset of variants that are encoded within transcripts that exceed some arbitrary transcript abundance threshold), then a key question is “Which RNA sample was used for filtering?”. To prevent these types of confusion, one of the design principles in LENS is to include the relevant sample-level information in the file name.

Our method for incorporating sample identifiers into file names is based on the template ${DATASET}-${PATIENT_NAME}-${RUN_NAME(s)} where RUN_NAME(s) contains the sample(s) information. DATASET and PATIENT_NAME are constant for a given patient, but the RUN_NAME(s) can vary by file (see https://uselens.io/en/lens-v1.7/manifest.html#sample-patient-and-dataset-hierarchy). For example, a salmon generated quant.sf file will need to indicate which RNA sample was utilized while a VCF containing somatic variants will need to indicate which normal DNA sample and which tumor DNA sample were used. The former scenario is straightforward while the latter is more complicated — what’s the best way to indicate that multiple samples were used to generate the file and how should they be represented in the file name? While - seems like a suitable delimiter between the run identifiers (RUN_NAME from the LENS manifest), it is not appropriate due to - already being a required character in run identifiers. Sure, - may technically work, but they are difficult to read and could lead to failures in some scenarios. With these factors in mind, we opted to use * as the delimiter between samples in output file names (e.g. MyDataset-Pt01-ad-123*nd-234). The * delimiter makes it very clear that there are two samples and it neatly partitions the sample identifiers.

The * delimiter has been used in LENS without issue until the current release, version 1.7. The change was prompted by a case where a patient has multiple Group names (see https://uselens.io/en/lens-v1.6/manifest.html#complex-sample-sets). Group names allow users to group (see where we got the name?) samples from a single patient together into sets for analyses. For example, if patient Pt01 has both a pre-treatment group of samples and an on-treatment group of samples then it’s reasonable to have a pre group and a on group (with some samples, like the normal DNA, being shared between groups). Generally, we use a RUN_NAME pattern of consistent length where the first characters describe the sample type (ad, ar, nd, or nr), the third character is a - delimiter and the remaining characters are a unique identifier of fixed length (e.g. the last 5 characters of the sample’s SRA identifier). One thing to keep in mind is that * functions as a wildcard when interacting with filesystems. This wildcard behavior has never resulted in unexpected behavior and even relaxing the RUN_NAME template constraints (e.g. having variable length suffixes) will, in most scenarios, not result in an undesirable or unexpected outcome.

This wildcard behavior can break LENS when the following conditions are met:

• A patient has multiple groups

• A RUN_NAME in one group has a substring of a RUN_NAME in a separate group and both RUN_NAMEs are associated with the same sample type (e.g. ad-123 and ad-1234)

• A process loads files related to the affected groups described above

Prior to version 1.7, there was only a single LENS process that met these conditions; however, the introduction of inter-group analyses in version 1.7 will make these scenarios more likely. Additionally, it’s probably best not to tempt fate with wildcards. We chose + as the new delimiter since it seems like the most natural option as it describes the adding of multiple samples to create an output.

Updated LENSTools to fix lenstools_make_indel_peptides_context_bugs

lenstools is a Python3 tool that handles a lot of the legwork within LENS, including generation of mutant proteins for peptide prediction. Despite the lenstools package’s unit tests, not all cases are covered. We recently encountered two cases, a misannotated variant and an unhandled exception resulting from neighboring phased frameshift InDel, that were resulting in process failures.

The first case involved a variant within a transcript’s 3’ UTR. Typically, these are well-annotated by snpEff and are handled appropriately within LENS. In this scenario, snpEff was labeling the variant as a disruptive in-frame insertion (within the last exon). We performed some sanity checks and realized that snpEff had misannotated the variant. lenstools was updated to better handle this situation moving forward.

The second case involved two neighboring phased variants in which the upstream variant was a frameshift InDel. The actual error was a result of an internal function not returning a correctly translated peptide sequence (in fact, it didn’t return anything, hence the error).

The lenstools_make_indel_peptides_context function within lenstools was updated to better handle these scenarios.

Added preliminary intra-patient inter-group analysis

The subjoining functionality within LENS allows users to run LENS on multiple sample sets per patient (e.g. various time points, before and on treatment, etc.) with minimal processing overhead. Prior to LENS v1.7, it was up to the user to then perform any ad-hoc analysis or comparisons between these Groups to find interesting patterns. For example, how many and which pMHCs are shared between the Groups? Which are unique to one group or another? What antigen sources make up the pMHCs in each category?

LENS v1.7 introduces the lenstools_multigroup_analysis process which serves as a template for automatically performing the analyses described above. The analysis is currently quite minimal (basically performing the analysis described above), but we expect to expand it to provide more meaningful information as to why pMHCs may not be shared. For example, in the case of a Group-specific SNV-derived pMHC, we can ask: Was the variant gained (or lost) in one Group relative to another? If not, then did the transcript abundance of the transcript(s) harboring that variant change drastically between Groups? In other words, we want to provide not just which pMHCs are shared or unique, but also why they may be unique.

Added multipass pMHC characterization to better handle long running tools

LENS has the ability to generate a lot of predicted pMHCs. These pMHCs are not uniformly distributed among tumor antigen sources. For example, a SNV may result in a handful of 8-11mers to be tested, an in-frame InDel will produce a similar count while a frameshift InDel has the potential to produce more. These all pale in comparison to the number of 8-11mers produced by CTAs and ERVs. We consider the entirety of CTA and ERV coding sequences to be non-self for peptide prediction purposes, and utilize a sliding window method to capture 8-11mers of interest. These 8-11mers are then paired with each of the patient’s HLA alleles. The resulting pMHCs are passed to relevant pMHC characterization tools (e.g. mhcflurry, netmhcpan, etc.). Unsurprisingly, the number of pMHCs considered can grow quite large (over 210,000 pMHCs in a random sample I just checked) and many tools struggle to handle this many pMHCs. In some cases, such as with netmhcpan and mhcflurry, we have optimized how the tools are run to make the best use of the compute resources available.

It may not be practical to optimize each pMHC characterization tool since it’s a hefty time investment and it’s unclear which tools will be relevant to pMHC prioritization, so we have developed a tiered characterization system instead. Specifically, the entire set of potential pMHCs will be processed in the first pass by whatever tool(s) the user specifies. By default, this is mhcflurry. After processing, the pMHCs that pass the binding affinity filter will then be processed through additional pMHC tools. For the sample described above, the original 210,000 pMHCs were reduced to 3,500 pMHCs — a much less burdensome task for most pMHC tools which will result in reduced run times. Users have the ability to determine what tools they want to run in the first pass and which they’d prefer to run on the reduced pMHC set in the second pass.

Modified workflow to natively support RNA only patients (no SNV or InDels pMHCs will be reported)

The LENS documentation has previously stated that patients require three different sample types — a normal DNA sample, a tumor DNA sample, and a tumor sample — for LENS to work properly. Unfortunately, it’s not uncommon for patients to, for a variety of reasons, not have all of these samples available. In fact, we’ve encountered this scenario when analyzing data within PIRL, but it has been a rare enough occurrence that no official support was implemented, and circumventing the sample limitation was an ad-hoc fix.

Recently, a third party reached out to us and requested this functionality to support their efforts, so we’ve updated LENS to directly support RNA-only patients in version 1.7.

The workflow modification involves disentangling the generation of RNA-derived pMHCs for RNA-only patients (specifically CTAs and fusions) from any dependency on the DNA samples. In general, LENS will internally call germline variants from the normal DNA sample and then use this information to update the coding sequence of CTA and fusion transcripts. This approach better captures the sequence variation within the patient and should generate higher fidelity pMHCs from these sources. Of course, incorporation of germline variants from the normal DNA sample requires that a normal DNA sample be available. Previously, LENS would encounter errors when attempting to incorporate the germline variants, which would result in workflow failure. LENS version 1.7 allows bypassing germline variant integration for RNA-only patients, such that pMHCs (except for SNV- and InDel-derived pMHCs) are still predicted and reported.

Reimplemented support for intermediate file cleaning of trimmed FASTQs and BAMs (using raft.py run-workflow ... --clean-intermediates)

Managing storage utilization at PIRL is, as with many groups, a constant struggle. Given infinite storage, we will still find a way to require infinite plus one. Alleviating that storage crunch motivated our original implementation of RAFT’s --clean-intermediates mode. The --clean-intermediates mode deletes many of the large intermediate files (e.g. BAMs) once LENS finishes running, but does so in a way that does not break Nextflow’s caching system (more information available here). This feature was temporarily removed as we worked to support cloud platforms, but has been reintroduced. It is highly recommended that users running LENS on a local or networked file system (i.e. not cloud buckets) run this option after LENS has completed.

Added automatic checking of user-provided metadata files for raft.py run-ots and raft.py run-workflow modes

LENS manifests can be very complicated, especially for complex datasets containing patient samples across multiple timepoints or treatments (see https://uselens.io/en/lens-v1.7/manifest.html). Generally, we at PIRL have avoided manifest-related LENS issues by thoroughly QCing the manifest prior to running LENS.

Despite this, manifest issues still occur. Previously, LENS would run with a malformed manifest until it hit an error. In the best-case scenario, this could be due to a mislabeled RUN_NAME and would result in some steps of the workflow being skipped. This would require some investigation and was generally easy to fix. On the other hand, scenarios where multiple samples of the same type (e.g. two tumor DNA samples, hereafter referred to as tumor DNA sample A and tumor DNA sample B) exist for the same patient and the same Group (see https://uselens.io/en/lens-v1.7/manifest.html#complex-sample-sets) cause more serious issues. LENS will run seemingly normally until the LENS project is later run again. At that point, it’d be possible that many of the processes involving the tumor DNA samples would re-run. This is a side effect of how Nextflow handles joining when multiple options are available (i.e. the normal DNA sample could be paired with tumor DNA sample A or it could be paired with DNA sample B). Specifically, Nextflow’s join operator randomly chooses one of the options each time Nextflow is run. That means the resulting pairing each time LENS is run is nondeterministic. This outcome generally takes more investigation to resolve as the initial behavior appears to be a Nextflow caching error.

RAFT already has a check-manifest mode, but we suspect many users do not currently run that utility before starting LENS. To address this, RAFT’s check-manifest functionality has been incorporated into the utilities.nf module and is now part of the LENS workflow. This means LENS will stop running if the manifest fails the validation and will alert the user, rather than continuing. This feature still needs some work to handle the more complex datasets (as it can sometimes provide overly cautious warnings), but we hope this helps users generate (and run) their manifests.

In the long term, we plan to overhaul how users interface with LENS by developing a more user-friendly manifest format, so stay tuned.

So, what’s next?

We plan to immediately start development on version 1.8-dev. There are a variety of smaller upgrades that will be immediately introduced in 1.8-dev such as the ability to filter pMHCs on metrics beyond binding affinity (e.g. presentation score, stability, etc.) and the ability to formulate your own pMHC prioritization score (instead of using the current hard-coded approach). Furthermore, we plan to revisit some of the LENSTools and RAFT module unit tests to ensure we have coverage for the additional cases we’ve seen popping up. Finally, we plan to improve documentation and automate documentation generation where possible (to avoid cases where tool features and documentation become unsynced).

We also plan to work further on our long-read analog to LENS, Tumor Antigens Through the Applications of Long Reads, or TATTALR. TATTALR will have a very similar internal infrastructure to LENS, but with added features such as pMHC prediction from larger structural variants.

Thanks for taking the time to read our LENS updates! We’re already excited to discuss LENS with those curious to learn more. Please do not hesitate to reach out to Steven Vensko II at steven_vensko@med.unc.edu, or stop by the RAFT Slack server: https://pirl-raft.slack.com.

This work was made possible through the generous support of the Jaime Leandro Foundation, a non-profit foundation committed to brining therapeutic cancer vaccines to patients.

• Current genomic variant representations are inconsistent across different variant callers and limit neoantigen discovery.

• New unified variant grammar addresses the fundamental issue.

• New tool VSTOL lets you standardize DNA variants from 14 different variant callers.

1. Short-Sighted in a Long Genome

We know that CD8+ killer T cells have the remarkable ability to selectively kill cancer cells in many cancer types. Immune checkpoint blockade therapies such as PD-1 inhibitors (e.g. Nivolumab and Pembrolizumab) and PD-L1 inhibitors (e.g. Atezolizumab and Avelumab) demonstrate this ability either by releasing the “brakes” on the killer T cells or by exposing the “disguise” of tumors cells to killer T cells. At the same time, we often don’t know which tumor-specific targets the killer T cells recognize to distinguish tumor cells (non-self) from normal cells (self). Occasionally, we discover tumor-specific peptide-major histocompatibility complex (pMHC) molecules that are recurrently presented across multiple patients with the same cancer type. For example, the cancer-testis antigen (CTA) NY-ESO-1 is expressed in over 70% of synovial sarcomas while PRAME, another CTA, is found in more than 80% of melanomas. These recurrently expressed CTAs have been the targets of effective adoptive T-cell receptor (TCR) therapies, as demonstrated by D’Angelo et al. and Wermke et al.

However, we also know that most cancers are characterized by both inter- and intra-patient heterogeneity rather than homogeneity. As a result, TCR therapeutics developed for shared targets remain limited to only a handful of cancer types. This challenge has driven significant efforts towards the development of personalized immunotherapies that individualize target selection for each cancer patient. Despite the promising scientific narrative behind this precision oncology approach, most personalized immunotherapies have been unsuccessful. Neon Therapeutics, Gritstone bio, and PACT Pharma are just some of the companies that have attempted but failed to deliver successful personalized immunotherapies.

There have been a few notable successes in personalized immunotherapy development. These successes are characterized by either hedging across many different target predictions or by directly using TCRs from antigen-experienced tumor-infiltrating lymphocytes.

So why does personalized immunotherapy usually fail? At PIRL, we think that at least one major deficiency is in the first few steps of the target selection process.

Target selection for personalized immunotherapy development relies on identification of neoantigens. Neoantigens are short (8-11 amino acids in length) mutant peptides presented by MHC class I molecules on tumor cells. The neoantigen identification process typically involves sequencing the tumor DNA and RNA as well as the matched normal DNA. The process begins by detecting somatic DNA variants, which are exclusively found in the genomes of tumor cells. This step ensures that personalized immunotherapies target tumor-specific molecular features and minimize the risk of autoimmune reactions against healthy and normal cells.

Over the past two decades, personalized immunotherapy development has suffered from the streetlight effect of looking at a specific type of somatic DNA variants: single-nucleotide variants (SNVs) that result in single-amino acid substitutions. As the name implies, these changes are subtle - so subtle that they might be indistinguishable from self-antigens on normal cells to killer T cells. If I wrote, “I hope you have a greit day,” you would still know what I meant - unless you speak Norwegian and know that greit means okay.

SNVs are typically detected in formalin-fixed paraffin-embedded (FFPE) tumor samples using short-read whole-exome sequencing - an assay that profiles only ~1% of the human genome (30 million out of the 3,000 million base pairs in the genome). By design, this target discovery setup overlooks the remaining 99% of the genome.

Could the most clinically actionable mutations lie in regions where we haven’t examined carefully? Could there be sources of therapeutic targets qualitatively better, less self-like, and more immunogenic? Could the answers we’ve been searching hidden in plain sight - just beyond the boundaries we’ve chosen to search?

As a field, we might have been short-sighted in a long genome.

2. Bigger Mutations, Bigger Impact?

So the natural question that arises is, “Are there bigger changes in tumor genomes?” The answer is yes. The Pan-cancer Whole Genomes Analysis Group (PCAWG) has revealed somatic structural variants (commonly defined in the literature as 30-50 base pairs or larger) in genomes of over 2,600 cancer patients representing 33 tumor types. There are 5 major types of structural variants: insertions, deletions, duplications, inversions, and translocations. If somatic structural variants affect the open-reading frames (ORFs) of protein-coding transcripts or otherwise give rise to novel ORFs, structural variants could represent a large pool of previously untapped neoantigens potentially highly divergent from the landscape of self-antigens. Given their inherently large sizes, somatic structural variants have the capacity to generate a bigger number of candidate mutant sequences subject to neoantigen prediction compared to SNVs (Figure 1). Also, tumors previously thought to have a low mutational burden (a metric typically based on the number of SNVs) might have targetable somatic structural variants. Moreover, the growing adoption of long-read sequencing means that we are better powered to identify and resolve somatic structural variants in cancer genomes.

In fact, a recent cancer vaccine trial has shown that frameshift mutations (insertions and deletions) elicited the highest in vitro (ELISPOT) immune responses in 4 out of the 9 patients that were enrolled in the study, and showed comparable responses to SNVs in 2 additional patients (see the publication’s Extended Data Figure 2b).

However, as you’ll see, targeting such mutations is difficult given the existing toolkits and ways of thinking about variants. So my PhD thesis work is focused on addressing this challenge and discovering neoantigens that originate from somatic structural variants in tumors.

3. One Variant, Multiple Representations

What is a DNA variant? Many of you reading this blog post will be familiar with SNVs. An SNV is a type of DNA mutation that affects just one base-pair in the genome. For example, a given position in the reference genome might have a common allele of Cytosine (C). When that same position changes to Thymine (T), we call this mutation an SNV.

Here is an Integrative Genomics Viewer (IGV) screenshot of an SNV in a tumor-suppressor gene TP53:

However, when it comes to structural variants, many more than 1 base-pair is mutated. Here is an IGV screenshot of a 30 base-pair deletion in TP53:

In football, basketball, and baseball, every player keeps their eyes on a single ball - there is just one game piece that drives the actions. As you will see, in bioinformatics however, we often find ourselves chasing many “balls” at once.

If you are a bioinformatician reading this post, you are familiar with the Variant Call Format (VCF) - the widely used standard for representing genomic variants. While the VCF is efficient for storage, comparing a putative variant across different variant callers can reveal just how much room there is for tool-specific quirks in representation. This issue becomes more pronounced with complex events like structural variants. Let’s take a closer look at an example.

Here, I have simulated 10 long sequencing reads capturing a somatic deletion (chr17:4,341,674 - chr17:7,677,491) between TP53 and UBE2G1. Additional long reads supporting germline DNA variants (SNVs) were generated for the same genomic positions for both the simulated tumor and matched normal samples. No sequencing errors were introduced to the reads:

I ran the following long-read structural variant callers using the tumor and normal samples:

• Savana

• SVIM

• Severus

• Sniffles2

• CuteSV

• NanomonSV

• SVision-pro

• PBSV

• Dysgu

Here are summarized VCF results from the above tools (Table 1):

The information highlighted in blue represents details related to the variant type. While Savana and SVIM reported breakpoints, Severus, Sniffles2, CuteSV, and NanomonSV classified the same event as a deletion. Note that in SVIM outputs, the mate breakpoint must be parsed from the ALT column. These differences are important to underscore because they expose the limitations of using outputs from existing variant callers to trace variants through the central dogma - connecting genomic changes to their transcriptomic and ultimately proteomic consequences.

The information highlighted in red represents the number of supporting reads, necessary for downstream calculation of cancer cell fraction and clonality. These are important data because personalized immunotherapy may benefit from targeting both clonal and subclonal events. Notice how this information is encoded across three columns (INFO, FORMAT, and TUMOR) in five different ways across the six tools:

1. TUMOR_READ_SUPPORT

2. SUPPORT

3. DV

4. RE

5. VR

This is not to mention that the lack of standardization in VCF outputs is further obscured by a growing list of complex rearrangement types such as pyros, rigma and various inversion subtypes.

The problem with the heterogeneity in variant representation is that owing to variations in precision and sensitivity across bioinformatics tools, we often want to identify a union (high sensitivity) set or an intersecting (high precision) set of variants from different methods.

4. One Variant, One Representation

The large variability in variant representation demonstrates a need for variant representation standardization. Before you yell inside, ‘So this is yet another standard?’ please allow me to explain.

Occam’s Variant Grammar

We know that the broader genomics field is moving towards pangenome graph representations. At the core of this graph approach is the fundamental idea that the string representation of the genome can be edited in or out of a reference genome. Building on this idea, I propose Occam’s Variant Grammar, a framework in which major DNA variant types are represented in a variation graph using the following set of graph operations:

where:

Variant Node

A variant node can either be an empty sequence (i.e. deletion) or an insertion sequence (i.e. SNV or insertion).

Edge Orientations

An edge between a variant node and the reference genome backbone can be created in one of the following two orientations:

• Upstream (towards 5’ end in the coding sense of the messenger RNA)

• Downstream (towards 3’ end in the coding sense of the messenger RNA)

A DNA variant (i.e. variant node and edge orientations) can be encoded as the following in the Occam’s Variant Grammar:

dna_variant = {
    chromosome_1:  <reference chromosome>,
    position_1:    <reference position>,
    strand_1:      <reference strand>,
    operation_1:   <upstream OR downstream>,
    chromosome_2:  <reference chromosome>,
    position_2:    <reference position>,
    strand_2:      <reference strand>,
    operation_2:   <upstream OR downstream>,
    sequence:      <variant sequence>
}

where sequence represents the variant node while operation_1 and operation_2 represent the edge orientations (upstream or downstream). The reference positions denote where the edges should be created on the reference genome backbone.

Here’s a graphical overview of how the Occam’s Variant Grammar allows us to represent the following major DNA variant types found in cancer:

• Circular DNA

• Deletion

• Duplication

• Insertion

• Inversion

• Single-nucleotide variant

• Translocation

• Viral integration

Duplication can be thought of as an insertion and in cases where the event is big, it can be captured with breakpoints. The same can be said of circular DNAs.

Under the Occam’s Variant Grammar, the aforementioned deletion will be represented as:

dna_variant = {
    chromosome_1:  'chr17',
    position_1:    4341673,
    strand_1:      '*',      // if reads support both strands
    operation_1:   'D',      // downstream (towards 3')
    chromosome_2:  'chr17',
    position_2:    7677492,
    strand_2:      '*',      // if reads support both strands
    operation_2:   'U',      // upstream (towards 5')
    sequence:      ''
}

Note that the denoted reference positions are 4,341,673 and 7,677,492 because the actual (simulated) deletion spans from 4,341,674 (inclusive) to 7,677,491 (inclusive).

Notably, the proposed grammar improves the variant representation by accurately capturing any hidden or otherwise unaligned insertion sequences between breakpoints. A good example of this is viral integration. A large proportion of ovarian, head, and neck tumors are caused by human papillomavirus (HPV), where after prolonged viral infection, the viral DNA integrates into the human host genome. When the host genome is shattered upon viral DNA integration, random nucleotides have been found to be inserted as part of the integration. However, based on our benchmark study, existing variant callers miss such variant types (data not shown). Given that HPV-positive head and neck tumors may exhibit a low mutational burden, this potentially highlights the low sensitivity in the current therapeutic target discovery processes.

A tumor genome represented as a graph genome resembles a multiverse where there are many versions of the genome reflecting different combinations of DNA variants (barring sensitive and precise variant phasing). Although we are moving towards a pangenome representation as a field, I think a linear coordinate system (e.g. hg38) that collapses the variant representation will remain valuable for interpretability of variants from the genome to the proteome for therapeutic development purposes. In this regard, the Occam’s Variant Grammar serves as a bridge between the graph-based and linear paradigms.

Advantages Over VCF 4.4

Before we go further, I would like to credit and acknowledge the authors of GRIDSS for being the first to separate the building blocks of a complex rearrangement event and the interpretation of the building blocks. As the authors noted in their documentation, this design rationale was in anticipation of the VCF 4.4 update. Compared to VCF 4.4, the Occam’s Variant Grammar provides three advantages for DNA variant representation:

1. Unification of major DNA variant type representation into just two atomic operations: add and connect.

2. DNA variants are readily compatible for construction of a personalized graph genome.

3. Human-readable yet machine-friendly encoding of orientation and insertion sequences between breakpoints. Knowing this information can be important in making sense of RNA variants in relation to their underlying DNA alterations. While the VCF 4.4 allows for encoding of these attributes, the long documentation (46 pages long) makes it difficult for developers to practice every detail.

Most importantly, as you’ll see in the next blog post in this series, the real power and value of the Occam’s Variant Grammar is in its ability to simultaneously represent DNA and RNA variants.

5. Automated Variant Standardization

To automate the variant grammar standardization effort, I developed VSTOL (Variant Standardization, Tabulation, and Operations Library; pronounced vee-stawl), a Python package with command-line interfaces. VSTOL supports standardization into the Occam’s Variant Grammar schema for 14 variant callers across both the short- and long-read worlds.

In the broader data science community, standard formats like CSV (Comma Separated Values) and TSV (Tab Separated Values) are the norm for data analysis because data stored in these formats can be loaded onto memory with a few lines of code. So why not align with what is already familiar and widely supported? VSTOL takes VCF files as inputs and outputs in the TSV format with the following headers:

chromosome_1:  <reference chromosome>,
position_1:    <reference position>,
strand_1:      <forward "+", reverse "-", both "*", unknown "">,
operation_1:   <upstream "U", downstream "D", unknown "">,
chromosome_2:  <reference chromosome>,
position_2:    <reference position>,
strand_2:      <forward "+", reverse "-", both "*", unknown "">,
operation_2:   <upstream "U", downstream "D", unknown "">,
sequence:      <variant sequence>
read_ids:      <read IDs separated by comma>

Additional annotations such as gene annotations, number of reads supporting reference alleles, and variant allele fractions can be appended as extra columns. Although the column names will vary across tools, the TSV format provides an easier data access.

6. In Closing

Back in college, I was a part of a racing team. We built our own race car. Before we could realize the designed speed and energy efficiency, we had to set up the tool board with the right instruments. How we equipped it shaped everything that followed. Bioinformatics is no different. For personalized immunotherapy to succeed, how we prepare our “tool board” of data representations, computational methods, and workflows matters deeply.

While the variant representation described here is motivated by cancer immunogenomics, its relevance extends much further. Structural variants are implicated in germline traits and other illnesses such as HIV/AIDS and cardiovascular diseases.

Stay tuned - there’s more to come on building our tool board.

7. Acknowledgement

This work was made possible through the generous support of the Jaime Leandro Foundation, a non-profit foundation committed to brining therapeutic cancer vaccines to patients.

We are interested in the performance of vaccine formulations as therapeutic tumor antigen vaccines. To this end, we repeated a subset of our prior experiment adding in peptide + polyIC-LC +/- montanide ISA 51 vaccination as treatment groups (Figure 1).

As in our prior experiment, we gave a priming vaccine dose on day 0, a single boost dose on day 14, and then analyzed splenocytes on day 21 for antigen-specific reactivity by interferon gamma ELISPOT. We chose the polyIC-LC dose based on published work, where 50 µg has been routinely administered intramuscularly (Belnoue et al., 2016; Derouazi et al., 2015; Sultan et al., 2020; Tran et al., 2020; Weng et al., 2023; Zhu et al., 2007). We administered polyIC through two injections per time point in the current experiment (as opposed to a single injection per time point in previous experiments) to enable a direct comparison with polyIC-LC, which must be administered through two injections per time point due to the concentration at which this adjuvant is provided and limits on volumes administered intramuscularly in mice.

We also determined the montanide (ISA 51) dose based on published work. Although identification of actual ISA 51 doses used in prior peptide vaccination studies is challenging, a dose of 25-100 µL of ISA 51 mixed 1:1 (v/v) with some other diluent (PBS, peptide suspension, or other adjuvant solution) seems to be common in mice, with 50 µL ISA 51 (prior to 1:1 mixing) being the most frequently used dose (Beebe et al., 2008; Chawla et al., 2019; Chu et al., 2022; Gabri et al., 2006; Ghorbani et al., 2005; Rosenthal et al., 2017; Varypataki et al., 2016). In this experiment, we delivered a total of 50 µL ISA 51 (prior to diluting 1:1) at each vaccine time point. In both humans and mice, this adjuvant is commonly administered subcutaneously or intramuscularly (van Doorn et al., 2015). To be consistent with recent clinical trials (Slingluff et al., 2021), we administered ISA 151 subcutaneously after mixing it 1:1 with peptide suspended in poly-ICLC as per manufacturer recommendations. The preparation of this adjuvant is critical to its success, with the syringe-extrusion/ two-syringe technique recommended over vortexing to mix (Koh et al., 2006; van Doorn et al., 2015). In the current study, we used the syringe-extrusion/two-syringe mixing protocol that was obtained from Seppic, the manufacturer.

In this experiment, peptide + CpG and mRNA vaccination gave similar results (Figure 2). Although polyIC-LC may have boosted responses relative to polyIC, there was high variance in the polyIC-LC group, and the difference between polyIC and polyIC-LC was not statistically significant in this small study.

Going forward, we will test more vaccine formulations (more adjuvants and DC vaccines next, hopefully DNA vaccines after that), dose titrations of key formulations, different vaccine schedules (varying prime and boost timing and number of injections), and importantly, we will evaluate vaccination against a set of validated neoantigens (rather than the M57 antigen alone) in tumor-bearing mice. Should these results bear out in future studies, we will pursue peptide + CpG and/or mRNA/LNP vaccination for clinical translation.

Many thanks to the Jamie Leandro Foundation (a non-profit foundation dedicated to making therapeutic cancer vaccines available to patients) as well as Natera for funding this work and supporting making the results available on this platform.

References:

• Arribillaga, L., Echeverria, I., Belsue, V., Gomez, T., Lozano, T., Casares, N., Villanueva, L., Domingos-Pereira, S., Romero, P. J., Nardelli-Haefliger, D., Hervás-Stubbs, S., Sarobe, P., Rodriguez, M. J., Carrascosa, J. L., Zürcher, T., & Lasarte, J. J. (2020). Bivalent therapeutic vaccine against HPV16/18 genotypes consisting of a fusion protein between the extra domain A from human fibronectin and HPV16/18 E7 viral antigens. Journal for ImmunoTherapy of Cancer, 8(1), e000704. https://doi.org/10.1136/jitc-2020-000704

• Barrios, K., & Celis, E. (2012). TriVax-HPV: An improved peptide-based therapeutic vaccination strategy against human papillomavirus-induced cancers. Cancer Immunology, Immunotherapy: CII, 61(8), 1307–1317. https://doi.org/10.1007/s00262-012-1259-8

• Beebe, M., Qin, M., Moi, M., Wu, S., Heiati, H., Walker, L., Newman, M., Fikes, J., & Ishioka, G. Y. (2008). Formulation and characterization of a ten-peptide single-vial vaccine EP-2101, designed to induce cytotoxic T-lymphocyte responses for cancer immunotherapy. Human Vaccines, 4(3), 210–218. https://doi.org/10.4161/hv.4.3.5291

• Belnoue, E., Di Berardino-Besson, W., Gaertner, H., Carboni, S., Dunand-Sauthier, I., Cerini, F., Suso-Inderberg, E.-M., Wälchli, S., König, S., Salazar, A. M., Hartley, O., Dietrich, P.-Y., Walker, P. R., & Derouazi, M. (2016). Enhancing Antitumor Immune Responses by Optimized Combinations of Cell-penetrating Peptide-based Vaccines and Adjuvants. Molecular Therapy, 24(9), 1675–1685. https://doi.org/10.1038/mt.2016.134

• Cafri, G., Gartner, J. J., Zaks, T., Hopson, K., Levin, N., Paria, B. C., Parkhurst, M. R., Yossef, R., Lowery, F. J., Jafferji, M. S., Prickett, T. D., Goff, S. L., McGowan, C. T., Seitter, S., Shindorf, M. L., Parikh, A., Chatani, P. D., Robbins, P. F., & Rosenberg, S. A. (2020). mRNA vaccine-induced neoantigen-specific T cell immunity in patients with gastrointestinal cancer. The Journal of Clinical Investigation, 130(11), 5976–5988. https://doi.org/10.1172/JCI134915

• Castro Eiro, M. D., Hioki, K., Li, L., Wilmsen, M. E. P., Kiernan, C. H., Brouwers-Haspels, I., van Meurs, M., Zhao, M., de Wit, H., Grashof, D. G. B., van de Werken, H. J. G., Mueller, Y. M., Schliehe, C., Temizoz, B., Kobiyama, K., Ishii, K. J., & Katsikis, P. D. (2023). TLR9 plus STING Agonist Adjuvant Combination Induces Potent Neopeptide T Cell Immunity and Improves Immune Checkpoint Blockade Efficacy in a Tumor Model. The Journal of Immunology, 212(3), 455–465. https://doi.org/10.4049/jimmunol.2300038

• Chawla, B., Mahajan, B., Oakley, M., Majam, V. F., Belmonte, A., Sedegah, M., Shimp, R. L., Kaslow, D. C., & Kumar, S. (2019). Antibody-Dependent, Gamma Interferon-Independent Sterilizing Immunity Induced by a Subunit Malaria Vaccine. Infection and Immunity, 87(10), e00236-19. https://doi.org/10.1128/IAI.00236-19

• Chen, J., Ye, Z., Huang, C., Qiu, M., Song, D., Li, Y., & Xu, Q. (2022). Lipid nanoparticle-mediated lymph node–targeting delivery of mRNA cancer vaccine elicits robust CD8+ T cell response. Proceedings of the National Academy of Sciences of the United States of America, 119(34), e2207841119. https://doi.org/10.1073/pnas.2207841119

• Cho, H.-I., Barrios, K., Lee, Y.-R., Linowski, A. K., & Celis, E. (2013). BiVax: A peptide/poly-IC subunit vaccine that mimics an acute infection elicits vast and effective anti-tumor CD8 T-cell responses. Cancer Immunology, Immunotherapy: CII, 62(4), 787–799. https://doi.org/10.1007/s00262-012-1382-6

• Chu, Y., Qian, L., Ke, Y., Feng, X., Chen, X., Liu, F., Yu, L., Zhang, L., Tao, Y., Xu, R., Wei, J., Liu, B., & Liu, Q. (2022). Lymph node-targeted neoantigen nanovaccines potentiate anti-tumor immune responses of post-surgical melanoma. Journal of Nanobiotechnology, 20(1), 190. https://doi.org/10.1186/s12951-022-01397-7

• Derouazi, M., Di Berardino-Besson, W., Belnoue, E., Hoepner, S., Walther, R., Benkhoucha, M., Teta, P., Dufour, Y., Yacoub Maroun, C., Salazar, A. M., Martinvalet, D., Dietrich, P.-Y., & Walker, P. R. (2015). Novel Cell-Penetrating Peptide-Based Vaccine Induces Robust CD4+ and CD8+ T Cell–Mediated Antitumor Immunity. Cancer Research, 75(15), 3020–3031. https://doi.org/10.1158/0008-5472.CAN-14-3017

• Gabri, M. R., Mazorra, Z., Ripoll, G. V., Mesa, C., Fernandez, L. E., Gomez, D. E., & Alonso, D. F. (2006). Complete Antitumor Protection by Perioperative Immunization with GM3/VSSP Vaccine in a Preclinical Mouse Melanoma Model. Clinical Cancer Research, 12(23), 7092–7098. https://doi.org/10.1158/1078-0432.CCR-06-1075

• Ghorbani, M., Nass, T., Azizi, A., Soare, C., Aucoin, S., Giulivi, A., Anderson, D. E., & Diaz-Mitoma, F. (2005). Comparison of Antibody- and Cell-Mediated Immune Responses After Intramuscular Hepatitis C Immunizations of BALB/c Mice. Viral Immunology, 18(4), 637–648. https://doi.org/10.1089/vim.2005.18.637

• Kim, K. Q., Burgute, B. D., Tzeng, S.-C., Jing, C., Jungers, C., Zhang, J., Yan, L. L., Vierstra, R. D., Djuranovic, S., Evans, B. S., & Zaher, H. S. (2022). N1-methylpseudouridine found within COVID-19 mRNA vaccines produces faithful protein products. Cell Reports, 40(9), 111300. https://doi.org/10.1016/j.celrep.2022.111300

• Koh, Y. T., Higgins, S. A., Weber, J. S., & Kast, W. M. (2006). Immunological consequences of using three different clinical/laboratory techniques of emulsifying peptide-based vaccines in incomplete Freund’s adjuvant. Journal of Translational Medicine, 4, 42. https://doi.org/10.1186/1479-5876-4-42

• Liu, J., Lu, X., Li, X., Huang, W., Fang, E., Li, W., Liu, X., Liu, M., Li, J., Li, M., Zhang, Z., Song, H., Ying, B., & Li, Y. (2023). Construction and immunogenicity of an mRNA vaccine against chikungunya virus. Frontiers in Immunology, 14, 1129118. https://doi.org/10.3389/fimmu.2023.1129118

• Melssen, M. M., Petroni, G. R., Chianese-Bullock, K. A., Wages, N. A., Grosh, W. W., Varhegyi, N., Smolkin, M. E., Smith, K. T., Galeassi, N. V., Deacon, D. H., Gaughan, E. M., & Slingluff, C. L. (2019). A multipeptide vaccine plus toll-like receptor agonists LPS or polyICLC in combination with incomplete Freund’s adjuvant in melanoma patients. Journal for Immunotherapy of Cancer, 7, 163. https://doi.org/10.1186/s40425-019-0625-x

• Panagioti, E., Redeker, A., van Duikeren, S., Franken, K. L., Drijfhout, J. W., van der Burg, S. H., & Arens, R. (2016). The Breadth of Synthetic Long Peptide Vaccine-Induced CD8+ T Cell Responses Determines the Efficacy against Mouse Cytomegalovirus Infection. PLoS Pathogens, 12(9), e1005895. https://doi.org/10.1371/journal.ppat.1005895

• Pavlick, A., Blazquez, A. B., Meseck, M., Lattanzi, M., Ott, P. A., Marron, T. U., Holman, R. M., Mandeli, J., Salazar, A. M., McClain, C. B., Gimenez, G., Balan, S., Gnjatic, S., Sabado, R. L., & Bhardwaj, N. (2020). Combined Vaccination with NY-ESO-1 Protein, Poly-ICLC, and Montanide Improves Humoral and Cellular Immune Responses in Patients with High-Risk Melanoma. Cancer Immunology Research, 8(1), 70–80. https://doi.org/10.1158/2326-6066.CIR-19-0545

• Ramos da Silva, J., Bitencourt Rodrigues, K., Formoso Pelegrin, G., Silva Sales, N., Muramatsu, H., de Oliveira Silva, M., Porchia, B. F. M. M., Moreno, A. C. R., Aps, L. R. M. M., Venceslau-Carvalho, A. A., Tombácz, I., Fotoran, W. L., Karikó, K., Lin, P. J. C., Tam, Y. K., de Oliveira Diniz, M., Pardi, N., & de Souza Ferreira, L. C. (2023). Single immunizations of self-amplifying or non-replicating mRNA-LNP vaccines control HPV-associated tumors in mice. Science Translational Medicine, 15(686), eabn3464. https://doi.org/10.1126/scitranslmed.abn3464

• Rosenthal, K. S., Stone, S., Koski, G., & Zimmerman, D. H. (2017). LEAPS Vaccine Incorporating HER-2/neu Epitope Elicits Protection That Prevents and Limits Tumor Growth and Spread of Breast Cancer in a Mouse Model. Journal of Immunology Research, 2017, 3613505. https://doi.org/10.1155/2017/3613505

• Sabbatini, P., Tsuji, T., Ferran, L., Ritter, E., Sedrak, C., Tuballes, K., Jungbluth, A. A., Ritter, G., Aghajanian, C., Bell-McGuinn, K., Hensley, M. L., Konner, J., Tew, W., Spriggs, D. R., Hoffman, E. W., Venhaus, R., Pan, L., Salazar, A. M., Diefenbach, C. M., … Gnjatic, S. (2012). Phase I trial of overlapping long peptides from a tumor self-antigen and poly-ICLC shows rapid induction of integrated immune response in ovarian cancer patients. Clinical Cancer Research: An Official Journal of the American Association for Cancer Research, 18(23), 6497–6508. https://doi.org/10.1158/1078-0432.CCR-12-2189

• Schoenmaker, L., Witzigmann, D., Kulkarni, J. A., Verbeke, R., Kersten, G., Jiskoot, W., & Crommelin, D. J. A. (2021). mRNA-lipid nanoparticle COVID-19 vaccines: Structure and stability. International Journal of Pharmaceutics, 601, 120586. https://doi.org/10.1016/j.ijpharm.2021.120586

• Slingluff, C. L., Zarour ,Hassane M., Tawbi ,Hussein Abdul-Hassan, Kirkwood ,John M., Postow ,Michael A., Friedlander ,Philip, Devoe ,Craig E., Gaughan ,Elizabeth M., Mauldin ,Ileana S., Olson ,Walter C., Smith ,Kelly T., Macri ,Mary J., Ricciardi ,Toni, Ryan ,Aileen, Venhaus ,Ralph, & and Wolchok, J. D. (2021). A phase 1 study of NY-ESO-1 vaccine + anti-CTLA4 antibody Ipilimumab (IPI) in patients with unresectable or metastatic melanoma. OncoImmunology, 10(1), 1898105. https://doi.org/10.1080/2162402X.2021.1898105

• Sultan, H., Kumai, T., Fesenkova, V. I., Fan, A. E., Wu, J., Cho, H.-I., Kobayashi, H., Harabuchi, Y., & Celis, E. (2018). Sustained Persistence of IL2 Signaling Enhances the Antitumor Effect of Peptide Vaccines through T-cell Expansion and Preventing PD-1 Inhibition. Cancer Immunology Research, 6(5), 617–627. https://doi.org/10.1158/2326-6066.CIR-17-0549

• Sultan, H., Wu, J., Fesenkova, V. I., Fan, A. E., Addis, D., Salazar, A. M., & Celis, E. (2020). Poly-IC enhances the effectiveness of cancer immunotherapy by promoting T cell tumor infiltration. Journal for ImmunoTherapy of Cancer, 8(2), e001224. https://doi.org/10.1136/jitc-2020-001224

• Tahtinen, S., Tong, A.-J., Himmels, P., Oh, J., Paler-Martinez, A., Kim, L., Wichner, S., Oei, Y., McCarron, M. J., Freund, E. C., Amir, Z. A., de la Cruz, C. C., Haley, B., Blanchette, C., Schartner, J. M., Ye, W., Yadav, M., Sahin, U., Delamarre, L., & Mellman, I. (2022). IL-1 and IL-1ra are key regulators of the inflammatory response to RNA vaccines. Nature Immunology, 23(4), Article 4. https://doi.org/10.1038/s41590-022-01160-y

• Temizoz, B., Kuroda, E., Ohata, K., Jounai, N., Ozasa, K., Kobiyama, K., Aoshi, T., & Ishii, K. J. (2015). TLR9 and STING agonists synergistically induce innate and adaptive type-II IFN. European Journal of Immunology, 45(4), 1159–1169. https://doi.org/10.1002/eji.201445132

• Tran, T.-A.-T., Kim, Y.-H., Duong, T.-H.-O., Jung, S., Kim, I.-Y., Moon, K.-S., Jang, W.-Y., Lee, H.-J., Lee, J.-J., & Jung, T.-Y. (2020). Peptide Vaccine Combined Adjuvants Modulate Anti-tumor Effects of Radiation in Glioblastoma Mouse Model. Frontiers in Immunology, 11. https://doi.org/10.3389/fimmu.2020.01165

• van Doorn, E., Liu, H., Huckriede, A., & Hak, E. (2015). Safety and tolerability evaluation of the use of Montanide ISA^TM51 as vaccine adjuvant: A systematic review. Human Vaccines & Immunotherapeutics, 12(1), 159–169. https://doi.org/10.1080/21645515.2015.1071455

• Varypataki, E. M., Silva, A. L., Barnier-Quer, C., Collin, N., Ossendorp, F., & Jiskoot, W. (2016). Synthetic long peptide-based vaccine formulations for induction of cell mediated immunity: A comparative study of cationic liposomes and PLGA nanoparticles. Journal of Controlled Release, 226, 98–106. https://doi.org/10.1016/j.jconrel.2016.02.018

• Weng, M.-T., Yang, S.-F., Liu, S.-Y., Hsu, Y.-C., Wu, M.-C., Chou, H.-C., Chiou, L.-L., Liang, J.-D., Wang, L.-F., Lee, H.-S., & Sheu, J.-C. (2023). In situ vaccination followed by intramuscular poly-ICLC injections for the treatment of hepatocellular carcinoma in mouse models. Pharmacological Research, 188, 106646. https://doi.org/10.1016/j.phrs.2023.106646

• Xu, K., Lei, W., Kang, B., Yang, H., Wang, Y., Lu, Y., Lv, L., Sun, Y., Zhang, J., Wang, X., Yang, M., Dan, M., & Wu, G. (2023). A novel mRNA vaccine, SYS6006, against SARS-CoV-2. Frontiers in Immunology, 13. https://www.frontiersin.org/articles/10.3389/fimmu.2022.1051576

• Yang, J.-X., Tseng, J.-C., Tien, C.-F., Lee, C.-Y., Liu, Y.-L., Lin, J.-J., Tsai, P.-J., Liao, H.-C., Liu, S.-J., Su, Y.-W., Hsu, L.-C., Chen, J.-K., Huang, M.-H., Yu, G.-Y., & Chuang, T.-H. (2023). TLR9 and STING agonists cooperatively boost the immune response to SARS-CoV-2 RBD vaccine through an increased germinal center B cell response and reshaped T helper responses. International Journal of Biological Sciences, 19(9), 2897–2913. https://doi.org/10.7150/ijbs.81210

• Zhu, X., Nishimura, F., Sasaki, K., Fujita, M., Dusak, J. E., Eguchi, J., Fellows-Mayle, W., Storkus, W. J., Walker, P. R., Salazar, A. M., & Okada, H. (2007). Toll like receptor-3 ligand poly-ICLC promotes the efficacy of peripheral vaccinations with tumor antigen-derived peptide epitopes in murine CNS tumor models. Journal of Translational Medicine, 5(1), 10. https://doi.org/10.1186/1479-5876-5-10

For the setup, I ran 1 ug of mRNA per lane in a 1% agarose gel containing SYBR Safe stain. Lane 1 contained a DNA ladder (not the best control since double-stranded DNA is expected to move more slowly through the gel than single-stranded mRNA, but hey, sometimes we just use what we have). The mRNA was either free (lanes 2-3) or encapsulated within LNPs (lanes 4-12). The concentration of the free mRNA had previously been determined using a Qubit assay while the concentration of the encapsulated mRNA had previously been determined using a RiboGreen assay. The encapsulated mRNA was either run alone (lanes 4-6) or after combining with Triton X-100 (lanes 7-9) or Tween 20 (lanes 10-12). Final concentration of either surfactant was 0.5%. Nuclease-free water was used for dilutions, and of course all samples were combined with a loading dye prior to loading in the gel. I ran the gel at 120 V and imaged it under UV illumination at both 20 and 40 min, after which it was begging to be put out of its misery and basically disintegrated. Here is how it looked at both time points:

Figure 1 – Gel after running for 20 min.

Figure 2 – Gel after running for 40 min.

From the images, we can gather a couple pieces of useful information. First, as a crude sanity check, the loading appears similar across the wells, suggesting concordance across methods of ascertaining mRNA concentration (RiboGreen vs. Qubit assay), which is always a good sign. We can see that the brightest band for mRNA1 appears slightly below the band for mRNA2, which matches what we know about the sizes of each of these constructs: mRNA1 should be 324 nucleotides while mRNA2 should be slightly larger at 399 nucleotides, slowing its movement through the gel just a tad. As expected, the free mRNA quickly migrated down the gel, while the encapsulated mRNA was mostly retained near its entry point close to the loading wells. Some smearing is visible in lanes 4-6, indicating that not all mRNA was retained within the nanoparticles (for reference, the encapsulation efficiency of the three tested batches of nanoparticles by RiboGreen assay was 94-98% immediately following formulation), but there is an obvious decrease in retention with the addition of either Triton X-100 or Tween 20. Although I was rooting for Tween 20, Triton X-100 was clearly better at releasing mRNA from the nanoparticles than its counterpart. Finally, just to address the obvious, I’m not sure what’s up with the ladder, but it’s still possible to make out the expected bands (although, as I described above, this doesn’t really matter for our purposes).

This was a crude but cheap and simple way to visualize our mRNA LNPs and the effects of two different surfactants. Along with confirming the presence of encapsulated mRNA, the main takeaway is that, at least for the time being, I will continue to use Triton X-100 to disrupt mRNA LNPs when needed instead of switching to Tween 20. Additionally, some of the batches of LNPs tested here were over two months old, suggesting that they may remain intact longer than we previously suspected, although this likely won’t change our current approach of using particles that are as “fresh” as possible.

Subscribe now

One challenge in optimizing cancer vaccines is our incomplete understanding of comparative vaccine potency. To address this, our lab is on a somewhat quixotic journey to benchmark cancer vaccine formulations and dosing schedules using functional readouts of T cell and B cell responses. Amongst a sea of competing formulations in clinical trials and preclinical development (many variations of peptide + adjuvant vaccines, DNA vaccines, mRNA vaccines, dendritic cell vaccines, viral component vaccines, etc.), we want to know which formulations are most immunogenic and why. In an early stab at this, we set up an experiment to vaccinate C57BL/6 mice with a known immunogenic MCMV epitope (M57) formulated as peptide + adjuvant and mRNA vaccines as below:

Here we tested poly(I:C), CpG, and montanide (all relatively common vaccine adjuvants), a STING agonist (which we can’t specify for now given constraints in our MTA with the supplier), and two mRNA vaccine formulations. For each of the mRNA formulations, N1-Methylpseudouridine was substituted for uridine. mRNA 2 included a segment coding for an MHC class I signal peptide as well as the vaccine epitope, whereas mRNA 1 did not. mRNA was carried by lipid nanoparticles that were manufactured in our lab using the Moderna mRNA-1273 lipid mix. All vaccine formulations were given intramuscularly, except the STING agonist which was given subcutaneously. The vaccine schedule was a simple prime and boost design, with a single priming dose given on day 0, a boost dose given on day 14, and splenocytes analyzed for M57-specific T cell responses by interferon gamma ELISpot on day 21. Experiments were done by Maria Sambade and Jess Alley, who wrote about generating the mRNA/LNP vaccines here. Results:

CpG performed best out of the adjuvants tested for peptide vaccines, however mRNA/LNP vaccines may have performed best overall. We will need to repeat this experiment with more animals per group and test other functional readouts along with interferon gamma secretion to be sure. Notably, poly(I:C) and montanide both performed rather poorly here. Most human clinical trials that use poly(I:C) actually use poly(I:C)-LC (Hiltonol), a poly(I:C) variant that is augmented by addition of polylysine. We need to test poly(I:C)-LC in future comparisons for translational relevance. We tested the STING agonist only with CpG here because (1) we had a limited amount of it, and (2) we had seen in prior experiments that CpG was a better adjuvant than poly(I:C) in this system.

Going forward, we plan to test other vaccine formulations (more adjuvants and DC vaccines next, hopefully DNA vaccines after that), different vaccine schedules (varying prime and boost timing and number of injections), and a set of validated neoantigens (rather than the M57 antigen alone) in tumor-bearing mice. We will continue to post results of these experiments here, and we hope this will be useful to the cancer vaccine community as we all try to figure out what are the best formulations to take to clinical trials.

A critical variable not addressed here is how to best do tumor antigen selection. Vaccine immunogenicity is irrelevant if the vaccine antigens are not presented by the tumor cells (just like antigen selection is irrelevant if vaccine immunogenicity is poor). More to come on that in future posts…

Many thanks to Natera for providing funding for this work along with Jamie Leandro Foundation (JLF), a non-profit foundation dedicated to making therapeutic cancer vaccines available for patients who could benefit from them. Thanks to both Natera and JLF for supporting publication of these results on Substack rather waiting for a “complete story” to emerge and be published as a preprint or peer-reviewed article.

NC was first discovered in 2003 by Dr. Christopher French, who has since led much of the research in this small but active field. NC is what's known as an epigenetically modulated fusion-driven cancer. That’s a lot of scientific jargon, so let’s break it down.

First, NC arises from a chromosomal translocation: a swap between parts of two chromosomes that fuses two genes together. The 5' partner is always NUTM1, a gene normally silent in most cells. The 3' partner is most often BRD4, but sometimes BRD3, NSD3, or other rare partners. Interestingly, BRD4 and BRD3 are nearly identical, and NSD3 has been shown to work through BRD4 — suggesting that, no matter the fusion partner, the BRD4::NUTM1 pathway drives oncogenesis.

That’s the "fusion" part. What about epigenetic modulation?

Epigenetics refers to molecular changes that alter gene expression without requiring modifications to the DNA sequence. One major contributor driving epigenetic changes is transcription factors, proteins that bind DNA and regulate gene transcription. In NC, the fusion protein BRD4::NUTM1 acts as a transcription factor. Normally, BRD4 binds acetylated DNA without causing trouble. However, in the case of NC, it brings along an unwanted partner: NUTM1. NUTM1 recruits p300, a histone acetyltransferase, leading to extensive acetylation through a feed forward loop. This creates huge stretches of hyperacetylated, open chromatin, driving uncontrolled gene activation and ultimately, cancer.

Luo, J., Bishop, J.A., DuBois, S.G. et al. Nat Rev Clin Oncol 22, 292–306 (2025)

Recent work has revealed an important nuance in the field of NC research – not all NUTM1 fusions are considered NC, as some give rise to other cancers. While it might sound like academic hairsplitting, distinguishing between NUTM1-rearranged NC and other NUTM1-rearranged cancers has real clinical implications, particularly for diagnosis and treatment. In the same way breast cancer treatments are tailored based on molecular markers like ER/PR and HER2 status, a biology-first approach in NC could lead to better outcomes.

In summary so far:

• NC is primarily driven by BRD4::NUTM1 fusion, causing massive transcriptional dysregulation.

• Not all NUTM1 fusions are NC.

• Molecular characterization could improve treatment strategies.

This is where our lab steps in. As an immunotherapy-focused group, we're on the hunt for cancer associated antigens — targets for therapies that can direct the immune system to selectively attack tumors.

Given the transcriptional dysregulation caused by BRD4::NUTM1, there are likely to be upregulated genes that result in therapeutics targets. An ideal target would be highly tumor-specific and broadly expressed across patients.

We’ve taken a particular interest in Cancer-Testis Antigens (CTAs). These are proteins normally restricted to germline tissues like the testis, fetal ovary, or placenta, and are usually absent from adult somatic tissues. Their expression in cancer is epigenetically regulated, mainly through DNA hypomethylation and histone modification. In short, CTAs fit our profile: tumor associated and likely to be broadly expressed since CTA epitopes can be presented by various HLA alleles.

This brings us to our latest preprint:

We systematically characterized CTA expression in five NC cell lines and a patient-derived xenograft (PDX) model, aiming to find consistently expressed CTAs that could be viable immunotherapy targets.

Using bulk RNA-seq, we found that out of 164 known CTAs, 14 were expressed in one or more samples. Some, like CT83 (an early diagnostic marker candidate in gastric cancers) and HORMAD1 (linked to lung cancer progression), were expressed in just 1–2 samples. More promisingly, PRAME was highly expressed in 4 out of 5 NC cell lines. (One cell line, TC-797, showed no PRAME expression across multiple replicates.) We don’t yet understand why one cell line lacks PRAME expression, but we have plans to follow up on this in future work.

BUT, and this is a big but in my humble opinion, RNA expression is just part of the story. For an immunotherapy to work, a protein must not only be expressed but also processed and presented on the cell surface via the Major Histocompatibility Complex (MHC).

Here’s a simplified overview of antigen presentation written out as steps:

1. Genes are transcribed into mRNA.

2. mRNAs are translated into proteins.

3. Proteins are degraded into small peptides (8–11 amino acids).

4. Peptides are loaded onto MHC molecules and transported to the cell surface.

5. T cells recognize these peptides, distinguishing normal cells from tumor cells.

And that same process turned into a visualization for my visual thinkers/learners.

Note: Figure made with bioRender

To see if our CTAs, especially PRAME, made it to the cell surface, we turned to immunopeptidomics — isolating MHC-bound peptides and analyzing them via LC-MS/MS.

We performed immunopeptidomics on the same five NC cell lines and PDX model. Excitingly, we found abundant PRAME-derived peptides — but only in PRAME-expressing cell lines. (No PRAME peptides in TC-797, matching RNA-seq results.)

Among the peptides we detected was SLLQHLIGL, a PRAME epitope currently targeted in clinical trials!

An interesting aside on this result is the MAGE-A1 epitope found on 3 of the samples. This is both confusing and interesting because we see no expression of MAGE-A1 in any of the samples at the RNA level, yet, we see it is the most recurrent peptide. MAGE-A1 is a promising target because it is implicated in a host of other cancers, and there is active research toward developing vaccines/therapies. Got thoughts on this? Feel free to share them!

Building on this, our collaborators in the Kuhlman lab at UNC have developed a bispecific T cell engager (BiTE) targeting the PRAME epitope SLLQHLIGL. BiTEs are engineered proteins with one arm binding T cells and the other binding tumor antigens, helping T cells kill tumor cells by placing them in prolonged proximity.

When we co-cultured T cells with PRAME+ NC cell lines (10-15 and 14169) and added the PRAME BiTE, we saw tumor cell killing at a 5:1 effector:target (E:T) ratio. In contrast, the PRAME-null (TC-797) and an HLA-mismatched line (H460) showed no response.

We have shown that in 2D cell culture, we can detect an expressed and presented CTA, PRAME, and, using a BiTE, effectively kill PRAME-positive tumor cells.

The next step? Expanding to a patient cohort to validate PRAME expression.

To do this, we partnered with Tempus AI to access CTA expression data from a cohort of 165 patient samples containing a NUTM1 fusion. In line with our earlier discussion on NUTM1-rearranged NC and other NUTM1-rearranged cancers, we stratified our cohort accordingly.

To our excitement, we observed that PRAME is abundantly and highly expressed across the cohort. Samples with high PRAME expression were statistically enriched in BRD4::NUTM1 fusions (fisher exact, p<0.001). Samples with BRD3::NUTM1 and NSD3::NUTM1 fusions—also canonical NC fusions—also exhibited higher PRAME levels compared to non-NC NUTM1 samples.

The presence of high PRAME expression in some non-NC samples is not surprising. PRAME has been implicated in multiple other cancers, and it is possible that in some cases, although a NUTM1 fusion is present, it is not the dominant driver of the cancer, or that the cancer itself intrinsically upregulates PRAME.

Collaboration with Dr. Chris French enabled us to confirm PRAME protein expression via immunohistochemistry (IHC) staining on a tissue microarray (TMA) (details of the microarray can be found in a previous publication here) comprised of 77 patient samples. Encouragingly, about half of the patients expressed PRAME in at least 25% of their tumor cells, giving us more confidence that PRAME epitopes are presented broadly in clinical samples.

From this blog post, we hope we have shown that PRAME is highly and recurrently expressed in both cell lines and patient samples, opening the door for NC patients to be considered for ongoing PRAME-targeted therapeutic clinical trials.

If you’d like to learn more about PRAME in NC, check out our preprint at bioRxiv or look at our analysis via our GitHub. Also, are you a researcher working on NUT Carcinoma? We’d love to connect and/or collaborate!

We would also like to thank the Brown family, the Victor Family Foundation, and the Max Vincze Foundation for their encouragement and support. Though a small field, we are mighty, and we are supported by incredible families and foundations. We could not do what we do without them, so thank you! Additionally, if you are a patient or a family member of a someone affected by NC, please visit the International NUT Carcinoma Registry for more information on the disease and clinical trial listings.

Unfortunately, we as a field are bad at most if not all elements of the personalized cancer vaccine workflow. Most predicted tumor antigens are not actually presented by tumor cells, most antigens included in personalized tumor vaccines are minimally immunogenic in vivo, no immunogenicity models work very well, (most?) microenvironment-based resistance mechanisms are unknown, and for the ones that are known, we don’t know much about how to target them therapeutically.

Still, the idea of a personalized therapeutic cancer vaccine is almost mythologically compelling: To turn the thing that makes a cancer cell cancerous (genetic changes that allow it to survive and divide aberrantly) into targets for its destruction seems like poetic justice. Adaptive immunity is the most elegant system in nature for self/non-self discrimination, and immunotherapy that generically boosts adaptive immunity can work in highly immunogenic tumors, for example yielding better than 50% long term survival in metastatic melanoma (compared to <10% with chemotherapy). Smarter immunotherapy should work even better.

There are three axes of innovation for companies (and academic labs) working on improving personalized cancer vaccines: (1) antigen selection, where improvement means increased likelihood of including vaccine antigens that are presented by a patient’s cancer cells and recognizable by T cells (and/or antibodies); (2) formulation potency, where improvement means increased likelihood of eliciting antigen-specific T cell clonal expansion and anti-tumor cytotoxicity against given antigens, and (3) manufacturing constraints, where improvement means increased likelihood of successful manufacturing for the set of included antigens and/or decreased cost in time or resources to manufacture the vaccine. To achieve clinical efficacy, a vaccine needs to be manufacturable and target good antigens with a potent formulation (i.e., succeed on all three axes), however a company with a major advantage in a single axis could be a good investment if its technology were woven into a vaccine strategy (or strategies) that cover the other two. For example, a company with a great adjuvant but poor antigen selection and a company with great antigen selection but poor adjuvant may both fail individually, but they could be very valuable in combination if their best aspects were integrated into an improved product. Perhaps there are other ways to gain competitive advantage in the cancer vaccine space, the most obvious being to go to clinic first for a disease indication where no other vaccine is approved (while having good enough efficacy to discourage competition for that niche); however personally I wouldn’t be excited about investing in a vaccine company without a demonstrably better vaccine.

Eventually we will be able to measure or predict vaccine efficacy without animal models, but we are not there yet. In the meantime, I look for the following features in evaluating the potency of vaccines in preclinical development in immunocompetent murine model systems:

Treatment of established tumors with well-formed tumor microenvironments. Many neoantigen vaccine studies are done with the vaccine given before injection of tumor cells or development of endogenous tumors. This is more likely to yield an efficacy signal but does not reflect human tumor biology (where the vaccine would be given after months-to-years of tumor development and coevolution with the patient’s immune system). I would prefer to see treatment started after tumors are relatively large, as well as to see evidence that the tumor immune microenvironment includes a distribution of cell types similar to analogous human tumors.

Use of multiple tumor models with concordant results. Murine tumor models are highly heterogeneous, different amongst themselves and different from human tumors (in some ways we know and many ways we don’t know). Even so, murine models have been instrumental to understanding immunobiology, including breakthroughs in immune checkpoint inhibition and cellular therapy that have been translated to FDA-approved immunotherapies for cancer. A vaccine strategy that is successful in multiple murine tumor models, coupled with evidence that the vaccine is inducing productive antigen-specific T cell responses through a similar mechanism in each model, is more likely to be generalizable to humans than a vaccine strategy that is tested in a single model only.

Vaccination using endogenous tumor-specific antigens. Many cancer vaccine studies use vaccine targets derived from xeno-antigens, the most common being antigens derived from the chicken protein ovalbumin, tested in murine tumor models engineered to express these xeno-antigens. While xeno-antigens are useful targets for immunological proof-of-principle experiments, they are highly non-self and more immunogenic than naturally occurring tumor antigens. As such, they present an unrealistically low bar for evidence of vaccine efficacy.

Evidence that the vaccine elicits T cell responses specific for antigens included in the vaccine, over and above those elicited in the absence of vaccination and with adjuvant alone, with effect sizes that would be clinically significant if replicated in humans. Tumor antigen-specific T cell responses may arise endogenously during tumor growth and/or as a result of therapy with vaccine adjuvant alone. To assess the biological efficacy of a cancer vaccine, I would like to see either an increased breadth of antigen specificities recognized by T cells after vaccination or increased number and/or potency of pre-existing antigen-specific T cell populations after vaccination. These effects should be greater with vaccination than without, and they should be substantial enough to provide clinical benefit if they were to occur in humans.

Evidence that vaccine treatment is associated with tumor shrinkage and improved survival of tumor-bearing animals, over and above that of vaccine adjuvant alone, with effect size that would be clinically significant if replicated in humans. This would be evaluated by measuring response rate, depth of responses, survival, and resistance to rechallenge with tumor cells (testing immunological memory).

Performance comparisons against other basic vaccine formulations. There are multiple competing vaccine formulations in the neoantigen vaccine space. To show improvement in vaccine efficacy, I would like to see comparisons against at least the leading three vaccine formulations: peptide + adjuvant, mRNA LNP, and dendritic cell vaccines. Ideally, testing should be done against multiple parameterizations of vaccine formulation for each formulation type. At minimum, testing should be done against vaccine formulations of each type that mirror those in current clinical trials.

We are pursuing experiments with all of these features in PIRL, however as an academic lab we can’t do all we would like to do. I hope that more companies developing in this space will take the time (and expense) to comprehensively test their vaccine products before deploying them in clinical trials. It will be hard to beat mRNA/LNP vaccines for manufacturability, however it’s still possible to innovate there by offering point of care manufacturing. Huge opportunities to improve antigen selection and vaccine potency remain untapped and will be needed to drive more cures for otherwise incurable cancers.

A key challenge in bioinformatics is ensuring the reproducibility of results. Many factors can complicate this, including differences in tool versions, discrepancies in reference files, missing or misplaced scripts, and inadequately described methods. These issues often lead to a "works on my machine" scenario—where the original developer can reproduce results, but others cannot.

Still skeptical? Try reproducing the analysis from the Hugo et al., 2016 manuscript using only the methods section. To address this, PIRL is proud to introduce RAFT, our Nextflow-based workflow manager designed to support reproducible bioinformatics pipelines. We use RAFT to power our neoantigen prediction workflow, LENS.

To help users get started, we've created a series of short tutorial videos introducing the fundamentals of both RAFT and LENS, available below. While these videos cover the basics, more in-depth information is available on their respective websites: useraft.io and uselens.io.

First, an introduction to RAFT:

Next, a tutorial on installing RAFT on your local machine:

Now that you have RAFT installed in your machine, let’s run a demonstration workflow. In this scenario, we’ll be running LENS v1.6 on Pt01 from the Hugo dataset. The Hugo dataset is available from this manuscript.

Once RAFT has completed, we can review the outputs/ directory. Don’t worry if you can’t or haven’t run the demonstration workflow, as an example of the output is available on PIRL's GitHub.

Now that we have a better understanding of RAFT and what it offers, let’s use RAFT to run LENS. First, an introduction to LENS:

Next, let’s briefly go over what’s available in the LENS report:

We hope that this tutorial series was informative and is helpful in getting you started with RAFT and LENS. Please do not hesitate to reach out to the head developer, Steven Vensko II, at steven_vensko@med.unc.edu with any questions or comments.

For example, we know some vaccine adjuvants are better at eliciting T-cell responses while other more commonly used ones seem weak in comparison. We know that vanishingly few predictions of tumor-specific antigens are actually presented by tumor cells. We know culturing T-cells with long peptides gets us effectively only helper T-cells and have a few different "fixes" to better isolate antigen-specific killer T-cell clones. We have written lots of computational tools for e.g. analyzing expressed structural variants from long read cancer sequencing data and: no one knows about them and thus no one uses them.

Preprints could theoretically fix our hoarding problem but this would take a little bit of cultural pioneering around what a preprint means and how early in the research process you could submit partial results to bioRxiv. I think concatenative preprinting across years of a research project would be vastly preferable to preprinting right before journal submission but students/post-docs/fellows correctly sense that the latter is a cultural norm in biomedical sciences.

So, we're adding a less polished faster lane to our lab's scientific communication: Substack posts. Every PIRL member can now post here without need for review, editing, or approval from anyone else in the lab. I'm hoping that we significantly accelerate our dissemination of important but incomplete results and start cross-pollinating conversations with other researchers who might also be hoarding their own results in adjacent research topics.

Is Substack actually the right venue for detailed scientific blogging? I'm not sure — it does seem to reward cultural commentary over detailed empirical communication. However, it also seems like there's a small but engageable scientific audience on this platform who may be curious about our adventures on the frontiers of machine learning, genomics, and immuno-oncology and might give us enough feedback to motivate a high cadence of scientific communication. Like all uncertain questions of reality, we'll run the experiment and see what happens.

This type of cancer was originally discovered by Dr. Chris French in 2003, whose lab has since steadfastly worked to unravel the mysteries of what drives NUT carcinoma’s rapid proliferation and identify potential treatments. Other research groups have since joined the effort and there is an increasing awareness that NUT carcinoma is both under-diagnosed and part of a larger spectrum of rare cancers involving NUTM1. Unfortunately, good outcomes for patients with NUT carcinoma remain mostly limited to those who are able to completely remove it surgically and we don’t yet know a successful treatment regimen for metastatic disease.

Over the past year and a half, PIRL has been working on several different therapeutic ideas for NUT carcinoma which I’ll describe in future blog posts. For now, I wanted to take stock of the current landscape of options for patients with incomplete surgical resections and make sense of when existing therapies do in fact work for a small number of patients.

Important to note: I’m not a medical doctor but rather a Computer Scientist who went for a long professional sojourn into immuno-oncology. Please don’t take this blog post as medical advice when it’s actually an incomplete literature search by a non-MD researcher.

Impressions

With that caveat out of the way, I have pasted a reading list below and will attempt to summarize my impressions:

• The presentation, response to treatment, and overall prognosis of NUT1 rearranged cancers depends heavily on the 5’ partner gene. Certain 5’ fusion partner genes (e.g. BRD4, BRD3, NSD3, YAP1, &c) are characteristic of cancers which look and act like NUT carcinoma, whereas others (e.g. RUNX1) only present in blood cancers and others (e.g. CIC) only in cancers which are categorized as sarcomas.

• Among the diversity of malignancies involving NUTM1 rearrangements, it does seem that cancers driven by BRD4:NUTM1 fusions are particularly aggressive and unresponsive to chemotherapy. There are many case studies of surprisingly good response to various therapies and almost all of them involve 5’ fusion partners other than BRD4.

• There was a lot of hope that the growth of NUT carcinoma could be significantly slowed by inhibiting the BRD4 portion of the drive oncogene using BET inhibitors such as molibresib and birabresib. Unfortunately, these drugs inhibit a broadly expressed family of proteins (BRD2, BRD3, BRD4, BRDT) and the resultant toxicity means existing BET inhibitors can’t be safely used at a dose level sufficient to fully arrest cancer growth. They do achieve a partial tumor shrinkage in a minority of patients, with some noteworthy case studies among patients with non-BRD4 fusion oncogenes.

• There’s a new BET inhibitor, ABBV-744, which is more selective and designed to limit off-target toxicity. It does not, alone, seem very therapeutically exciting even in mouse models of NUT carcinoma. However, ABBV-744 does seem to synergize with the EZH2 inhibitor tazemetostat and, while not entirely eradicating NUT carcinoma growth in mice, it does at least delay progression for months. Whether this translates to humans remains to be seen.

• For the most part, metastatic NUT carcinoma shrugs off most chemotherapy regimens and simply keeps growing, though there is modest evidence in retrospective studies that ifosfamide may be a better choice for chemotherapy.

• The most remarkable thing I found in this literature search is several case studies of complete responses to the Scandinavian protocol for Ewing Sarcoma in NUT carcinomas with BRD4 as the 5’ fusion partner. The treatment consists of “two courses of VAI (vincristine, doxorubicin, ifosfamide) alternating with one course of PAI (cisplatin, doxorubicin, ifosfamide) at 3-weekly intervals”. That’s a very aggressive chemotherapy regimen and I’m way out of my expertise in trying to judge whether it’s generally applicable to NUT carcinoma patients (likely only pediatric patients would survive it). I will note that “I” in this VAI/PAI protocol stands for ifosfamide, so maybe that’s another clue that ifosfamide may have a role to play in treating NUT carcinoma. In any case, I hope someone elevates these anecdotal successes into some kind of prospective clinical trial.

Reading List

Background & Overviews:

• Report of the First International Symposium on NUT Carcinoma

• Supercharging BRD4 with NUT in carcinoma

• MYC, a downstream target of BRD-NUT, is necessary and sufficient for the blockade of differentiation in NUT midline carcinoma

• The BRD4–NUT Fusion Alone Drives Malignant Transformation of NUT Carcinoma

• Ectopic protein interactions within BRD4–chromatin complexes drive oncogenic megadomain formation in NUT midline carcinoma

• The expanding universe of NUTM1 fusions in pediatric cancer

• Emerging entities in NUTM1-rearranged neoplasms

Retrospective Patient Cohorts:

• Initial Chemotherapy for Locally Advanced and Metastatic NUT Carcinoma

• Clinicopathologic Features and Long-term Outcomes of NUT Midline Carcinoma

• NUT carcinoma in pediatric patients: Characteristics, therapeutic regimens, and outcomes of 11 cases registered with the German Registry for Rare Pediatric Tumors (STEP)

• Clinical management of NUT carcinoma (NC) in Germany: Analysis of survival, therapy response, tumor markers and tumor genome sequencing in 35 adult patients

• Therapeutic approaches to sinonasal NUT carcinoma: a systematic review

• NUTM1-rearranged neoplasia: a multi-institution experience yields novel fusion partners and expands the histologic spectrum

• An Anatomical Site and Genetic-Based Prognostic Model for Patients With Nuclear Protein in Testis (NUT) Midline Carcinoma: Analysis of 124 Patients

• Patterns of care and impact of prognostic factors in the outcome of NUT midline carcinoma: a systematic review and individual patient data analysis of 119 cases

Unsuccessful Therapies

• How Much Can We Bet on Activity of BET Inhibitors Beyond NUT–Midline Carcinoma?

• Chemical Screen Identifies Diverse and Novel Histone Deacetylase Inhibitors as Repressors of NUT Function: Implications for NUT Carcinoma Pathogenesis and Treatment

• Combined Targeting of the BRD4–NUT–p300 Axis in NUT Midline Carcinoma by Dual Selective Bromodomain Inhibitor, NEO2734

Promising Murine Studies:

• EZH2 Cooperates with BRD4-NUT to Drive NUT Carcinoma Growth by Silencing Key Tumor Suppressor Genes

Durable Responses in Patients with BRD3 and other atypical 5' fusion partners:

• Exceptional Response to Bromodomain and Extraterminal Domain Inhibitor Therapy With BMS-986158 in BRD4-NUTM1 NUT Carcinoma Harboring a BRD4 Splice Site Mutation

• Novel BRD2::NUTM1 Fusion in NUT Carcinoma With Exceptional Response to Chemotherapy: A Case Report

• Sustained Clinical Response to Immunotherapy Followed by BET Inhibitor in a Patient with Unresectable Sinonasal NUT Carcinoma

• Case report: Immunovirotherapy as a novel add-on treatment in a patient with thoracic NUT carcinoma

• A case of metastatic NUT carcinoma with prolonged response on gemcitabine and nab-paclitaxel

Durable Responses in Patients with Typical BRD4:NUTM1 fusions:

• Complete Response Induced by Concurrent Chemoradiotherapy in a Patient with NUT Carcinoma

• Pediatric NUT-midline carcinoma: Therapeutic success employing a sarcoma based multimodal approach

• NUT carcinoma of the nasal cavity that responded to a chemotherapy regimen for Ewing’s sarcoma family of tumors: a case report

• NUT Carcinoma Without Upfront Surgical Resection: A Case Report

This problem is not specific to me and my colleagues. A Nextflow GitHub issues thread regarding this topic recently celebrated its fifth birthday and Ben Sherman (@bentsherman) has, as of the writing of this post, started laying the ground work to allow Nextflow to utilize temporary intermediate files.

Our lab has been tackling storage issues since we started using Nextflow DSL2 way back in November 2019. We were unable to "trick" Nextflow into caching near-zero-sized files despite our best efforts. Luckily, Stephen Ficklin and colleagues, authors of GEMmaker, have developed a clever solution to this problem.

This blog post is to show examples explaining the utility of this approach, provide a syntactical tutorial showing implementation examples, and describe the limitations and pitfalls I encountered while implementing it. Before we discuss the technical aspects of intermediate file cleaning in Nextflow workflows, let's further elaborate on the need for this functionality.

Why?

Consider the following workflow and its associated files:

In our case, we are only required to retain the first (raw FASTQs) and last (fully processed BAMs) files for the purposes of reproducibility and to satisfy downstream workflow inputs, respectively. Nevertheless, Nextflow in its current form requires the four intermediate files to be available on the filesystem for workflow caching purposes.

Now consider the following workflow and its associated files:

Deleting intermediate results in a 75% reduction in storage requirements for the same eventual workflow outcome. This allows for more samples to be processed using the same storage with less downtime resulting from storage maintenance. This optimization can be taken to the extreme by having the fully processed BAM (the star shaped element in our diagram) deleted once your workflow generates its final outputs (e.g. transcript counts, VCFs, etc.). There are, of course, trade-offs to this strategy which we'll discuss in the Limitations and Pitfalls section.

How?

In a nutshell, the GEMmaker strategy works by creating a very small file that resembles the original file closely enough that Nextflow cannot tell the difference. The bash script which performs this action is available in the GEMmaker Github repositorty. Briefly, the script stats the file, records the relevant statistics, creates a sparse file and modifies the sparse file's logical size, modification time, and access time to reflect those of the original file. The result is an indistinguishable (by Nextflow, anyways) file to serve as a placeholder for your previously large intermediate file.

When?

Hopefully one can appreciate the value of the sparse file strategy when applied to running large scale Nextflow workflows. One may be tempted to use this hammer to all of the "nails" in their workflows. Unfortunately, every rose has its thorn. Specifically, the timing of intermediate file cleaning is relatively delicate. Any downstream processes or workflows that utilize the cleanable intermediate files must be completed prior to cleaning or your workflow will fail due to the prematurely deleted intermediate file. User will then be stuck re-running a portion of your workflow after debugging the cause.

The next aspect of "when?" is more philosophical -- is it better to delete as soon as possible or at the end of the workflow when all of your endpoint files are available? Should deletion occur within your top-level workflow (as in GEMmaker) or sub-workflows? This is a topic I've wrestled with while implementing intermediate file deletion in our neoantigen workflow LENS so I have thoughts on it that I can share in a future blog post assuming there's interest.

A Syntax Example

The GEMmaker implementation has been suggested as a potential workaround to commenters in Nextflow's GitHub Issue #452. I attempted to implement this strategy after strong (but understandable) insistence by our sysadmins that I use less storage. I initially found the logic a bit hard to follow (the code is clear, but I am a mere geneticist, not a software engineer!), so I wanted to create a verbose yet minimal example to explain the core logic. The code (trick_nextflow_cache.nf, clean_work_dirs.sh, and utilities.nf) described below can be found on GitHub: https://github.com/pirl-unc/tricking_nextflow_cache.

First, we'll demonstrate the functionality and then walk through the script.

We begin with an empty work directory:

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ du -hscL work/
4.0K    work
4.0K    total

Next, we run our Nextflow script:

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ nextflow trick_nextflow_cache.nf -resume
N E X T F L O W  ~  version 21.10.6
Launching `trick_nextflow_cache.nf` [romantic_fourier] - revision: 22b984aca2
executor >  slurm (2)
[22/0f9755] process > make_a_large_file  [100%] 1 of 1 ✔
[b1/d1cc88] process > inspect_large_file [100%] 1 of 1 ✔

Checking the work directories sizes reveals a ~1 Gb intermediate file generated by make_a_large_file():

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ du -hscL work/
1.1G    work/
1.1G    total

Now we run Nextflow script again and indicate that we'd like intermediate files to be deleted:

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ nextflow trick_nextflow_cache.nf -resume --delete_intermediates True
N E X T F L O W  ~  version 21.10.6
Launching `trick_nextflow_cache.nf` [exotic_jennings] - revision: 22b984aca2
executor >  slurm (1)
[22/0f9755] process > make_a_large_file    [100%] 1 of 1, cached: 1 ✔
[b1/d1cc88] process > inspect_large_file   [100%] 1 of 1, cached: 1 ✔
[16/f99612] process > clean_work_files (1) [100%] 1 of 1 ✔

Note the process clean_work_files() ran this time.

Checking the work directories again revealed a reduction from 1.1G to 60K! 🚀

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ du -hscL work/*
24K     work/16
20K     work/22
16K     work/b1
60K     total

This in itself is not too impressive - anyone can delete or replace a file in a work directory. The magical part happens when we re-run the workflow:

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ nextflow trick_nextflow_cache.nf -resume --delete_intermediates True
N E X T F L O W  ~  version 21.10.6
Launching `trick_nextflow_cache.nf` [mighty_albattani] - revision: 22b984aca2
[22/0f9755] process > make_a_large_file    [100%] 1 of 1, cached: 1 ✔
[b1/d1cc88] process > inspect_large_file   [100%] 1 of 1, cached: 1 ✔
[16/f99612] process > clean_work_files (1) [100%] 1 of 1, cached: 1 ✔

Nextflow pulled the make_a_large_file() process from cache despite the file emitted (make_a_large_file.out.a_large_file) being a 20K sparse file!

Next, let's go file-by-file and explain what's happening behind the scenes.

utilities.nf

#!/usr/bin/env nextflow
process clean_work_dirs {
  input:
  tuple val(directory)

  output:
  val(1), emit: IS_CLEAN

  script:
  """
  for dir in ${directory}; do
  if [ -e \$dir ]; then
    echo "Cleaning: \$dir"
    files=`find \$dir -type f `
    echo "Files to delete: \$files"
    clean_work_files.sh "\$files" "null"
  fi
  done
  """
}

process clean_work_files {

  cache 'lenient'

  input:
  val(file)

  output:
  val(1), emit: IS_CLEAN

  script:
  """
    clean_work_files.sh "${file}"
  """
}

utilities.nf consists of two process - clean_work_dirs and clean_work_files. clean_work_dirs appears to delete the contents of a work directory while clean_work_files targets specific files for deletion. Personally, I have only been using clean_work_files within my workflows. I have included these processes into their own module to allow easy aliasing when including the process into main.nf. For example, consider our current example in which we're running a single cleaning process. Defining the cleaning process within main.nf wouldn’t be the worst idea in the world. However, once you start cleaning multiple outputs files from multiple processes, having multiple cleaning processes defined in main.nf (as opposed to using aliasing while making include statements - see below) get messy fast.

include { clean_work_files as clean_trimmed_fastqs } from '../utilities/utilities.nf'
include { clean_work_files as clean_sorted_bams } from '../utilities/utilities.nf'
...
include { clean_work_files as clean_files_n} from '../utilities/utilities.nf'

clean_work_files.sh

#!/bin/bash
# https://raw.githubusercontent.com/SystemsGenetics/GEMmaker/master/bin/clean_work_files.sh
# This script is meant for cleaning any file in a Nextflow work directory.
# The $files_list variable is set within the Nextflow process and should
# contain the list of files that need cleaning. This can be done by creating
# a channel in a process that creates files, and merging that channel with
# a signal from another process indicating the files are ready for cleaning.
#
# The cleaning process empties the file, converts it to a sparse file so it
# has an acutal size of zero but appears as the original size, the access
# and modify times are kept the same.
files_list="$1"

for file in ${files_list}; do
  # Remove cruff added by Nextflow
  file=`echo $file | perl -p -e 's/[\\[,\\]]//g'`
  if [ -e $file ]; then
    # Log some info about the file for debugging purposes
    echo "cleaning $file"
    stat $file
    # Get file info: size, access and modify times
    size=`stat --printf="%s" $file`
    atime=`stat --printf="%X" $file`
    mtime=`stat --printf="%Y" $file`

    # Make the file size 0 and set as a sparse file
    > $file
    truncate -s $size $file
    # Reset the timestamps on the file
    touch -a -d @$atime $file
    touch -m -d @$mtime $file
  fi
done

clean_work_files.sh is self-explanitory thanks to its great inline documentation.

trick_nextflow_cache.nf

nextflow.enable.dsl=2

include { clean_work_files } from './utilities.nf'

params.delete_intermediates = ''

process make_a_large_file {

  cache 'lenient'

  output:
  tuple val("foo"), path("1G_file"), emit: a_large_file

  script:
  """
  dd if=/dev/zero of=1G_file bs=1G count=1
  """
}

process inspect_large_file {

  cache 'lenient'

  input:
  tuple val(samp), path(required_input_file)

  output:
  tuple val(samp), path("file_stats"), emit: file_stats

  script:
  """
  ls -ldhrt ${required_input_file} > file_stats
  """
}

workflow {
  make_a_large_file()

  inspect_large_file(
    make_a_large_file.out.a_large_file)

  make_a_large_file.out.a_large_file                                                                
    .join(inspect_large_file.out.file_stats, by: [0])                                               
    .flatten()                                                                                      
    .filter{ it =~ /_file$/}                                                                        
    .set{ large_file_done_signal }                                                                  

  if( params.delete_intermediates ) {
    clean_work_files(
      large_file_done_signal)
  }
}

trick_nextflow_cache.nf begins with a shebang, a standard inclusion statement (to import the cleaning process), and a parameter definition used for triggering intermediate file deletion. Next are two separate processes, the latter dependent on output from the former. The first process, make_a_large_file(), generates a one gigabyte sized file which is emitted through the a_large_file channel. The second process, inspect_large_file(), takes the emittied channel and simply ls -lhdrts it. In this example, you can imagine make_a_large_file() being an aligner process (e.g. bwa) and inspect_large_file() as being a variant caller (e.g. stelka2).

Once the inspect_large_file() process has completed, the actual intermediate large file itself is no longer needed. That is where the following code block comes into play:

  make_a_large_file.out.a_large_file                                                                
    .join(inspect_large_file.out.file_stats, by: [0])                                               
    .flatten()                                                                                      
    .filter{ it =~ /_file$/}                                                                        
    .set{ large_file_done_signal }  
  ...
    if( params.delete_intermediates ) {                                                               
    clean_work_files(                                                                               
      large_file_done_signal)                                                                       
  }

In this block, we are taking the channel containing the intermediate file we want to delete (the 1 Gb file), joining it to the channel containing the eventual endpoint file (file_stats). Note we are joining them on the 0th element which is the samp variable. We want to ensure that the intermediate file we're deleting and the corresponding output file are linked by a sample-level identifier. Once we have the joined tuple, we flatten() it and filter{} each element for the intermediate file suffix we are seeking to delete (_file$ in this case). We then pass this new tuple of deletable intermediates to clean_work_files() assuming params.delete_intermediates is true.

Note: Be as specific as possible with your filter{} regex string!!!

Limitations and Pitfalls

I encountered several hurdles when implementing this strategy into our workflows. I've described them below so others can avoid them. Please let me know of any other examples the community encounters and I'll include them.

Limits workflow expansion

This intermediate file deletion strategy, in its current form, should really only be applied to workflows that are well-established end-to-end. Deletion of intermediate files will result in prevention of workflow expansion. For example, consider a scenario in which you have alignment files and are performing variant calling with subsequent BAM cleanup:

process align {
...
}

process variant_caller_x {
...
}

workflow {
  main:
    align()
    variant_caller_x(align.out.alns)
    align.out.alns
      .join(variant_caller_x.out.vcfs, by: [0])
      .flatten()
      .filter{ it =~ /.bam$/ }
      .set{ cleanable_bams }
    clean_work_files(
      cleanable_bams)
}

After completing this workflow, you have your variant files and your storage hungry intermediate BAM files will be eliminated as expected.

Now, let's assume a colleague comes along and suggests also running the latest and greatest variant_caller_y (in addition to your original variant_caller_x). You include the variant_caller_y() process call to workflow (see below) and re-run the Nextflow:

process align {
...
}

process variant_caller_x {
...
}

process variant_caller_y {
...
}

workflow {
  main:
    align()
    variant_caller_x(align.out.alns)
    variant_caller_y(align.out.alns)
    align.out.alns
      .join(variant_caller_x.out.vcfs, by: [0])
      .join(variant_caller_y.out.vcfs, by: [0])
      .flatten()
      .filter{ it =~ /.bam$/ }
      .set{ cleanable_bams }
    clean_work_files(
      cleanable_bams)
}

However, upon running the above code you will be greeted with a curious error. What gives?! The deletion of the intermediate BAM file from the initial run means you can no longer use those BAMS to call variants. Nextflow is unaware that the BAM files are effectively useless since, from its perspective, they are perfectly fine cached files. At this point, you realize you must delete the work directory associated with align() and re-run that portion of the workflow.

Checking for every downstream output

Another consideration is that one must ensure all downstream processes have performed and completed their work prior to deletion of the intermediate file. In other words, the workflow developer must perform all process and channel accounting before allowing clean_work_files to execute. This will likely result in some rather beastly join statements that will clutter the code. Luckily, Ben Sherman (@bentsherman) is working on updating Nextflow to do this accounting internally and removing this burden from the owrkflow developer.

Nanoseconds count!

Astute readers may note my excessive use of the cache directive in process definitions. Inclusion of this directive is required to overcome the next pitfall I encountered. Specifically, it appears the Access and Modify time modications performed on the sparse file by clean_work_files.sh are only to the nearest second, but my cluster's filesystem seems to track Access and Modify times down to sub-second resolution. As a result, the sparse file generated did not pass Nextflow's caching sniff test and results in the process being re-ran (with sparse files as input). The cache directive (when paired with the 'lenient' option) bypasses this issue by removing the Access and Modify times from the Nextflow caching equation.

Original stat output prior to cleaning (from .command.log):

[spvensko@c6145-2-9 dd915decbfc0f0992acd0454f7e14f]$ cat .command.log 
cleaning /PATH/TO/work/71/8fb6ae67536c55ac80815da9c4a231/a_fastq_2.trimmed.fq.gz
  File: ‘/PATH/TO/work/71/8fb6ae67536c55ac80815da9c4a231/a_fastq_2.trimmed.fq.gz’
  Size: 4891699623      Blocks: 9591736    IO Block: 65536  regular file
Device: 31h/49d Inode: 9248463003887304838  Links: 1
Access: (0644/-rw-r--r--)  Uid: (321499/spvensko)   Gid: ( 3150/vincent_lab)
Context: system_u:object_r:nfs_t:s0
Access: 2022-12-14 23:49:37.616634000 -0500
Modify: 2022-12-15 00:02:27.847675000 -0500
Change: 2022-12-15 00:02:27.847675000 -0500
 Birth: -

stat output of sparse file which replaced the original file:

[spvensko@c6145-2-9 dd915decbfc0f0992acd0454f7e14f]$ stat /PATH/TO/work/71/8fb6ae67536c55ac80815da9c4a231/a_fastq_2.trimmed.fq.gz      
 File: ‘/PATH/TO/work/71/8fb6ae67536c55ac80815da9c4a231/a_fastq_2.trimmed.fq.gz’
 Size: 4891699623      Blocks: 0          IO Block: 65536  regular file
Device: 31h/49d Inode: 9248463003887304838  Links: 1
Access: (0644/-rw-r--r--)  Uid: (321499/spvensko)   Gid: ( 3150/vincent_lab)
Context: system_u:object_r:nfs_t:s0
Access: 2022-12-14 23:49:37.000000000 -0500
Modify: 2022-12-15 00:02:27.000000000 -0500
Change: 2022-12-15 01:21:02.845392000 -0500
Birth: -

Note that the Access time from the original file (2022-12-14 23:49:37.616634000 -0500 ) is not the same as the Access time from the sparse file created by clean_work_files.sh (2022-12-14 23:49:37.000000000 -0500).

Target the file, not the file's symlink

The final hurdle I had to overcome was one of misdirection. Nextflow relies heavily upon symlinks when operating on local cluster filesystems to prevent excessive copying of files among work directories. As a result, it's possible for a channel element containing what appears to be a file's path to instead be a symlink pointing to that original file. Consider the following example:

nextflow.enable.dsl=2

process make_a_large_file {

  cache 'lenient'

  output:
  tuple val("foo"), path("1G_file"), emit: a_large_file

  script:
  """
  dd if=/dev/zero of=1G_file bs=1G count=1
  """
}

process inspect_large_file {

  cache 'lenient'

  input:
  tuple val(samp), path(required_input_file)

  output:
  tuple val(samp), path(required_input_file), path("file_stats"), emit: file_stats

  script:
  """
  ls -ldhrt ${required_input_file} > file_stats
  """
}

workflow {
  make_a_large_file()
  make_a_large_file.out.a_large_file.view()
  inspect_large_file(
    make_a_large_file.out.a_large_file)
  inspect_large_file.out.file_stats.view()
}

Running this script results in the following output:

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ nextflow real_vs_symlink.nf -resume
N E X T F L O W  ~  version 21.10.6
Launching `real_vs_symlink.nf` [pedantic_wright] - revision: fffa5484af
[22/0f9755] process > make_a_large_file  [100%] 1 of 1, cached: 1 ✔
[b1/d1cc88] process > inspect_large_file [100%] 1 of 1, cached: 1 ✔
[foo, /home/spvensko/tricking_nextflow_cache_example/work/22/0f975597019f5e81e7561400e3d7b0/1G_file]
[foo, /home/spvensko/tricking_nextflow_cache_example/work/b1/d1cc88cf2ba4b16eb770d90b09f4aa/1G_file, /home/spvensko/tricking_nextflow_cache_example/work/b1/d1cc88cf2ba4b16eb770d90b09f4aa/file_stats]

Notice that both make_a_large_file()'s and inspect_large_files()'s emitted channels contain a path to 1G_file. However, upon closer inspection, we realize one path is the actual file while the other is simply a symlink to the original file:

[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ file /home/spvensko/tricking_nextflow_cache_example/work/22/0f975597019f5e81e7561400e3d7b0/1G_file                                                                                      
/home/spvensko/tricking_nextflow_cache_example/work/22/0f975597019f5e81e7561400e3d7b0/1G_file: data
[spvensko@c6145-2-9 tricking_nextflow_cache_example]$ file /home/spvensko/tricking_nextflow_cache_example/work/b1/d1cc88cf2ba4b16eb770d90b09f4aa/1G_file
/home/spvensko/tricking_nextflow_cache_example/work/b1/d1cc88cf2ba4b16eb770d90b09f4aa/1G_file: symbolic link to `/home/spvensko/tricking_nextflow_cache_example/work/22/0f975597019f5e81e7561400e3d7b0/1G_file'

In this case 1G_file was effectively passed through inspect_large_file(), but a symlink to the original file was emitted rather than the original file itself.

So, what does this matter?

If the workflow developer is not careful and deletes the symlink to 1G_file (../work/b1/d1cc88cf2ba4b16eb770d90b09f4aa/1G_file) rather than the 1G_file itself (../work/22/0f975597019f5e81e7561400e3d7b0/1G_file) then they will not save any space and instead disrupt the caching of their workflow.

The moral of this story is to ensure the channel and element you are targeting with clean_work_files is the actual file you want to delete and not a symlink to that file.

Conclusion

I hope this post helps in explaining why users may want to delete intermediate files in Nextflow, how to perform intermediate deletion, and some of the limitations and pitfalls associated with this approach.

Please do not hesitate to contact me with any thoughts or comments at steven_vensko@med.unc.edu or @spvensko on the nextflow.slack.com.

I would like to first thank Dr. Michael Lazear and Dr. Rob Patro for their wonderful feedback on my recent blog post. Dr. Lazear has generously shown that there is a better implementation that can deliver a blazingly fast compute time by using “map-reduce” with worker pool in the rayon package of Rust. In swift response to Dr. Lazear’s suggestion, Dr. Patro initially confirmed that the suggested implementation delivers a substantial improvement over the Python baseline (also thank you for teaching me about the ‘—release’ parameter for ‘maturin develop’). This approach indeed leads to a 10x improvement in Rust+Python over the pure Python implementation.

1. Updated Implementation (Rust+Python)

Here is the updated Rust+Python (multithreaded) implementation that uses map-reduce in the rayon package (the credit goes to Dr. Lazear for the original source code):

lib.rs

#[pyfunction]
fn count_kmers_multithread_fx_hashmap_improved(sequences: Vec<String>, k: usize, num_threads: usize) -> Py<PyAny> {
    let pool = rayon::ThreadPoolBuilder::new()
        .num_threads(num_threads)
        .build()
        .unwrap();
    let hm = pool.install(|| {
        let hm = sequences
            .par_iter()
            .map(|sequence| {
                let end = sequence.len() - k + 1;
                let mut map: FxHashMap<_, i32> = FxHashMap::default();
                for i in 0..end {
                    *map.entry(&sequence[i..i+k]).or_insert(0) += 1;
                }
                map
            })
            .reduce(
                || FxHashMap::default(),
                |mut acc, x| {
                    for (k, v) in x {
                        *acc.entry(k).or_insert(0) += v;
                    }
                    acc
                },
            );
        return hm;
    });

    Python::with_gil(|py| hm.to_object(py))
}

#[pymodule]
fn scripts(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(count_kmers_multithread_fx_hashmap_improved, m)?)?;
    Ok(())
}

count_kmers_rust_multithread_fx_hashmap_improved.py

import pysam
import time
import pandas as pd
import scripts


def count_kmers_rust_multithread_fx_hashmap_improved(
        fasta_file: str,
        num_processes: int,
        sequence_chunk: int,
        k: int,
        chromosomes: list) -> pd.DataFrame:
    """
    Counts all possible k-mers in a FASTA file.

    Parameters
    ----------
    fasta_file      :   FASTA file.
    num_processes   :   Number of processes to multiprocess FASTA file.
    sequence_chunk  :   Sequence chunk.
    k               :   k-mer k.
    chromosomes     :   Chromosomes to count.

    Returns
    -------
    df_kmers        :   DataFrame of k-mers with the following columns:
                        'k_mer', 'count'
    duration_1      :   Duration of step 1.
    duration_2      :   Duration of step 2.
    duration_3      :   Duration of step 3.
    duration_total  :   Total duration.
    """
    # Step 1. Create tasks
    start_time_1 = time.time()
    fasta = pysam.FastaFile(fasta_file)
    sequences = []
    for curr_chromosome in chromosomes:
        curr_chromosome_len = fasta.get_reference_length(curr_chromosome)
        for i in range(0, curr_chromosome_len, sequence_chunk):
            curr_start = i
            curr_end = curr_start + sequence_chunk
            if curr_end > curr_chromosome_len:
                curr_end = curr_chromosome_len
            curr_chromosome_seq = fasta.fetch(reference=curr_chromosome,
                                              start=curr_start,
                                              end=curr_end)
            sequences.append(curr_chromosome_seq.upper())
    end_time_1 = time.time()

    # Step 2. Run
    start_time_2 = time.time()
    results = scripts.count_kmers_multithread_fx_hashmap_improved(sequences, k, num_processes)
    end_time_2 = time.time()

    # Step 3. Merge dictionaries into one
    start_time_3 = time.time()
    df_kmers = pd.DataFrame(results.items())
    df_kmers.columns = ['k_mer', 'count']
    df_kmers.sort_values(['count'], ascending=False, inplace=True)
    end_time_3 = time.time()
    return df_kmers, \
           end_time_1 - start_time_1, \
           end_time_2 - start_time_2, \
           end_time_3 - start_time_3, \
           end_time_3 - start_time_1

2. Updated Benchmark Test Results

The benchmark test configuration was kept the same as the original post (9-mer counting of chromosome 1 in hg38 with 24 cores with sequences of 1 million nucleotides in length). The Rust implementation using map-reduce led to a 9.6x improvement from the pure Python equivalent (Figures 1 and 2). This was also a substantial improvement compared to the former Rust implementation that relied on a multithreading pool. Note that the benchmark tests were run again on a different node with less powerful cores, which is why you see a different range of durations compared to the original post. Nevertheless, the comparative speed improvement from Python to Rust is conserved from the previous test to the current test.

3. Conclusions

I have learned a great deal from receiving feedback from the much active scientific/bioinformatics community on Twitter at large. So thank you, Dr. Lazear and Dr. Patro, for your help and time. I would like to also thank my co-advisor and principal investigator, Dr. Alex Rubinsteyn, for sharing the post on Twitter in the first place. You were right, Chief!

The experience of writing a blog post about a topic that I am just beginning to learn and then tweeting it has inadvertently become a manifestation of Cunningham’s Law. I suspect that Alex knew about it all along. In any case, I have learned a great deal about using Rust and human behavior. I promise to use these newly acquired skill-sets sparingly and responsibly.

Dynamic scripting languages such as Python and R are convenient for implementing bioinformatics algorithms. However, these languages incur a significant overhead on every operation, which makes processing read-level data in large quantities slow. One way to speed up a slow algorithm is to move the bottleneck out of Python into a more efficient compiled language such as Rust. In this post I explore nucleotide k-mer counting as an example algorithm and evaluate the performance improvement of a few different implementations. Ultimately, my fastest Rust+Python implementation is only 2.5x faster than a pure Python equivalent.

1. Background and Motivation

In our lab, we often process hundreds of thousands, if not millions, of sequencing reads generated from a malignant tumor sample. These reads are footprints of the mutational landscape underlying the tumor and clinically actionable insights are grounded in understanding these footprints. To view the landscape, we usually piece together sequencing reads to solve a puzzle - a large puzzle in fact. The original picture we are after is typically composed of around 3 billions letters and we have millions of pieces to solve. This is where bioinformatics softwares come to our aid.

Many bread-and-butter bioinformatics softwares are written in “low-level” languages such as C and C++ to deliver fast computation times. These languages provide instructions more native to a computer machine’s hardwares compared to “high-level” languages such as Python. While low-level languages guarantee unparalleled computation times, the development process can prove to be slow and tedious, and the eventual codebase can become quite verbose. This makes the codebase hard to maintain and to deploy over iterations. This is why a versatile and succinct high-level language like Python remains one of the most popularly used programming languages. However, the sheer versatility and succinctness of Python are also sources of bottlenecks of its less-than-ideal execution time. For many clinical settings where speed is crucial to generate timely clinical decisions, Python quickly hits a practical limit. For example, k-mer counting is a common computation in bioinformatics that involves updating a hash table and this simple task written in C++ is 25 times faster than Python.

About a decade ago, a new low-level programming language called Rust was released. In recent years, we have seen an uptick in the number of bioinformatics tools developed in Rust. I have recently started learning Rust and have tried to wrap it in Python so that we can integrate the best of both worlds - the comprehensive suite of statistical and optimization libraries offered in Python and the swift computation time of Rust. This post recounts the story of my recent experience in attempting a number of different technical implementations wrapping Rust in Python and how I landed the one that I like (for now). I also share all the different implementations that I have tried in hopes of saving another bioinformatician’s (or developer’s) time.

To compare different implementations, I chose to work with the k-mer counting problem. For example, given the following 3 sequences “CATGATCCAA”, “AGGATAAGCC”, “GGATCACGAC”, we want to count the occurrence of all 3-mers in these sequences over sliding windows (e.g. "GAT” appears 3 times in the three sequences; “CATGATCCAA”, “AGGATAAGCC”, “GGATCACGAC”).

2. Implementations

Each implementation is composed of three parts:

1. Generation of tasks (chunks of sequences).

2. K-mer counting by task workers.

3. Merging hash tables into one.

For full implementations, please check out here.

2-1. Multiprocessing in Python (Baseline)

To establish the baseline, I first implemented k-mer counting in Python3 as a multi-processing program with the function worker_py defined as the task worker:

count_kmers_python_multiprocess.py

import multiprocessing
import pysam
import time
import pandas as pd
from functools import partial
from collections import Counter


def worker_py(sequence: str, k: int) -> Counter:
    """
    Fetches k-mers in a given sequence.

    Parameters
    ----------
    sequence        :   Sequence.
    k               :   k-mer k.
    shared_list     :   Shared list.
    """
    counter = Counter()
    for i in range(0, len(sequence) - k + 1):
        counter[sequence[i:i+k]] += 1
    return counter


def count_kmers_python_multiprocess(fasta_file: str,
                                    num_processes: int,
                                    sequence_chunk: int,
                                    k: int,
                                    chromosomes: list) -> pd.DataFrame:
    """
    Counts all possible k-mers in a FASTA file.

    Parameters
    ----------
    fasta_file      :   FASTA file.
    num_processes   :   Number of processes.
    sequence_chunk  :   Sequence chunk.
    k               :   k-mer k.
    chromosomes     :   Chromosomes to count.

    Returns
    -------
    df_kmers        :   DataFrame of k-mers with the following columns:
                        'k_mer', 'count'
    work_duration   :   Duration of multiprocessing compute
    """
    # Step 1. Create tasks
    fasta = pysam.FastaFile(fasta_file)
    sequences = []
    for chrom in chromosomes:
        chrom_len = fasta.get_reference_length(chrom)
        for i in range(0, chrom_len, sequence_chunk):
            curr_start = i
            curr_end = curr_start + sequence_chunk
            if curr_end > chrom_len:
                curr_end = chrom_len
            curr_chromosome_seq = fasta.fetch(
                reference=chrom,
                start=curr_start,
                end=curr_end
            )
            sequences.append(curr_chromosome_seq.upper())

    # Step 2. Run
    pool = multiprocessing.Pool(num_processes)
    start_time = time.time()
    results = pool.map(partial(worker_py, k=k), sequences)
    end_time = time.time()

    # Step 3. Merge dictionaries into one
    outputs = results[0]
    for curr_result in results[1:]:
        outputs.update(curr_result)
    df_kmers = pd.DataFrame.from_dict(outputs.items())
    df_kmers.columns = ['k_mer', 'count']
    df_kmers.sort_values(['count'], ascending=False, inplace=True)
    return df_kmers, end_time - start_time

Terminal output with the following parameters:

• hg38.fa

• 6 processes

• 1,000,000 nucleotide chunks

• 9-mers (k = 9)

• chr21

Python implementation of k-mer counting
k_mer        count
NNNNNNNNN  6620936
TTTTTTTTT    44007
AAAAAAAAA    41832
TGTGTGTGT    10454
ACACACACA    10170
Program took 14.662775 seconds in total

2-2. Rust Wrapped in Python

To wrap Rust in Python, I used the maturin package in Python. To start writing a Rust function, first initialize by executing the following in the terminal:

maturin init

This generates a lib.rs file in src/ folder and a Cargo.toml file. We will be writing the Rust codes in the lib.rs file and specify any dependencies in the Cargo.toml file. To compile Rust code using maturin, run:

maturin develop

Let’s add compiler optimization to the Cargo.toml file:

[profile.dev]
opt-level = 3

[profile.release]
debug = 0

For more on maturin, refer to the official maturin user guide.

2-2-1. Python with Multi-threaded Rust (STL HashMap)

For starters, to take advantage of Rust’s native multi-threading capabilities, I first implemented a multi-threaded k-mer counting function:

lib.rs

use pyo3::prelude::*;
use rayon::prelude::*;
use std::collections::HashMap;

#[pyfunction]
fn count_kmers_multithread_stl_hashmap(sequences: Vec<String>, k: usize, num_threads: usize) -> Py<PyAny> {
    let pool = rayon::ThreadPoolBuilder::new()
    .num_threads(num_threads)
    .build()
    .unwrap();
    let (tx, rx) = std::sync::mpsc::channel();
    for seq in &sequences {
        let tx = tx.clone();
        let seq = seq.to_owned();
        pool.spawn(move || {
            let mut hm : HashMap<String, i32> = HashMap::new();
            let end = seq.chars().count() - k + 1;
            for i in 0..end {
                *hm.entry(seq[i..i+k].to_owned()).or_insert(0) += 1;
            }
            tx.send(hm).unwrap();
        });
    }

    drop(tx); // close all senders
    let results: Vec<HashMap<String, i32>> = rx.into_iter().collect();

    // Merge HashMaps
    let mut hm : HashMap<&str, i32> = HashMap::new();
    for curr_hm in &results {
        for (key, value) in curr_hm.into_iter() {
            *hm.entry(&key).or_insert(0) += value;
        }
    }

    return Python::with_gil(|py| {
        hm.to_object(py)
    });
}

#[pymodule]
fn scripts(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(count_kmers_multithread_stl_hashmap, m)?)?;
    Ok(())
}

The above code makes the Rust function count_kmers_multithread_stl_hashmap available as part of a package (named script in my code).

Make sure to append the following line to Cargo.toml file under [dependencies]:

rayon = "1.5.3"

count_kmers_rust_multithread_stl_hashmap.py

import pysam
import time
import pandas as pd
import scripts


def count_kmers_rust_multithread_stl_hashmap(fasta_file: str,
                                             num_processes: int,
                                             sequence_chunk: int,
                                             k: int,
                                             chromosomes: list) -> pd.DataFrame:
    """
    Counts all possible k-mers in a FASTA file.

    Parameters
    ----------
    fasta_file      :   FASTA file.
    num_processes   :   Number of processes to multiprocess FASTA file.
    sequence_chunk  :   Sequence chunk.
    k               :   k-mer k.
    chromosomes     :   Chromosomes to count.

    Returns
    -------
    df_kmers        :   DataFrame of k-mers with the following columns:
                        'k_mer', 'count'
    work_duration   :   Duration of multiprocessing compute
    """
    # Step 1. Create tasks
    fasta = pysam.FastaFile(fasta_file)
    sequences = []
    for curr_chromosome in chromosomes:
        curr_chromosome_len = fasta.get_reference_length(curr_chromosome)
        for i in range(0, curr_chromosome_len, sequence_chunk):
            curr_start = i
            curr_end = curr_start + sequence_chunk
            if curr_end > curr_chromosome_len:
                curr_end = curr_chromosome_len
            curr_chromosome_seq = fasta.fetch(reference=curr_chromosome,
                                              start=curr_start,
                                              end=curr_end)
            sequences.append(curr_chromosome_seq.upper())

    # Step 2. Run
    start_time = time.time()
    results = scripts.count_kmers_multithread_stl_hashmap(sequences, k, num_processes)
    end_time = time.time()

    # Step 3. Merge dictionaries into one
    df_kmers = pd.DataFrame(results.items())
    df_kmers.columns = ['k_mer', 'count']
    df_kmers.sort_values(['count'], ascending=False, inplace=True)
    return df_kmers, end_time - start_time

2-2-2. Python with Multi-threaded Rust (FX HashMap)

The above multi-threaded Rust implementation resulted in about 55% reduction (i.e. 2.2x improvement) in execution time from the Python implementation. As I looked to further improve the speed of the Rust code, I stumbled upon this page that explains how the HashMap included in the standard library (STL) of Rust implements cryptographically safe hashing. The natural move was to replace the STL HashMap with another HashMap that relies on a faster hashing function, such as rustc-hash (henceforth referred to as FX HashMap). So all of the HashMap declarations in the above Rust code was replaced with the following:

let mut hm : FxHashMap<String, i32> = FxHashMap::default();
let results: Vec<FxHashMap<String, i32>> = rx.into_iter().collect();
let mut hm : FxHashMap<&str, i32> = FxHashMap::default();

Make sure to append the following line to Cargo.toml file under [dependencies]:

rustc-hash = "1.1.0"

Replacing the HashMap implementation from the STL function to the FX version indeed resulted in a noticeable improvement in speed (data not shown).

2-2-3. Python with Multi-processed Rust (STL or FX HashMap)

Then it occurred to me that using pointers in Rust might lead to another boost in speed. After tinkering around for some time, I realized that implementing a pointer-based access to a HashMap in a multi-threaded Rust program is not so straight forward and elegant. Instead, I opted to multiprocessing a Rust function but allowing pointer-based access and update to the HashMap. Shown below is the Rust code for the task worker function:

#[pyfunction]
fn count_kmers_fx_hashmap_pointer(sequence: String, k: usize) -> Py<PyAny> {
    let mut hm : FxHashMap<&str, i32> = FxHashMap::default();
    let end = sequence.chars().count() - k + 1;
    for j in 0..end {
        *hm.entry(&sequence[j..j+k]).or_insert(0) += 1;
    }
    return Python::with_gil(|py| {
        hm.to_object(py)
    });
}

Here is a simple update to the Python implementation to include the rust worker function:

results = pool.map(partial(scripts.count_kmers_fx_hashmap_pointer, k=k), sequences)

3. Benchmark Test Results

To systematically benchmark the various implementations, I set up a test to count all 9-mers in chromsome 1 of hg38. I ran the above implementations along with some other variations for comparison purposes. In short, of the 7 implementations, the multi-threaded Rust with FX HashMap resulted in the fastest compute time of 16.828 seconds on average. In contrast, Python took 42.321 seconds on average (Figure 1).

It should be noted that the two multi-threaded Rust implementations depend on Rust to merge the HashMaps into one. Computation of task workers (step 2) is a subset of all computations accounted for in the total duration. When we just capture the duration of task workers in multi-threaded and multi-processed implementations, we see there are relatively smaller improvements in speed (Figure 2). Based on the duration of task workers in either multi-threaded or multi-processed implementations, we see that HashMap merging (step 3) accounts for an overwhelming fraction of the time saved (Figure 2). In other words, the multi-threaded implementation comes ahead of the multi-processed implementation because of the heavy-lifting performed by Rust in merging HashMaps.

4. Conclusions

In this post, we saw how to wrap Rust in Python. In conclusion, multi-threaded Rust (with FX HashMap) wrapped in Python yielded the fastest implementation with approximately 2.5x improvement in speed over the baseline Python implementation. In the tech world, people often talk about delivering a 10x improvement for a new technology to tip the scale and be widely adopted. Although the speed improvement is clear, I am not convinced that the additional overhead of keeping Rust codebase along with the Python codebase justifies for mere 2.5x improvement in speed. Stay tuned for more on this topic!

Special thanks to Dhuvi Karthikeyan for his invaluable feedback on my post and for suggesting the title of this blog post.

On the flip side of this, the ionizable lipid’s neutral charge at physiological pH is also critical as it makes ionizable lipids less toxic than previously used, permanently charged cationic lipids. Cholesterol and the helper lipid stabilize and support the nanoparticles, while the PEG lipid prevents particles from aggregating. It’s important to note here that all of these components play additional roles in vivo that can affect where these particles go and how long they persist, so this is a rather simplified overview for the sake of brevity. (If you aren’t claustrophobic and want to go further down the rabbit hole, a nice detailed review can be found here.)

Besides the aforementioned lipids, our nanoparticles also contain nucleoside-modified mRNA, which has structural changes to render it less immunogenic. This may sound counterintuitive when the ultimate goal of administering these particles is to stimulate an immune response, but detection of foreign nucleic acids can trigger cellular changes that prematurely halt protein synthesis, preventing the desired outcome.

To formulate the LNPs, we used a benchtop microfluidic device to rapidly mix the lipid and mRNA solutions then dialyzed the mixture overnight, concentrated it, and sterile-filtered for use in cell cultures. Following formulation, we determined the mRNA concentration and encapsulation efficiency along with the particle size and polydispersity index (PDI). All batches produced had an average diameter of between 75 and 150 nm with a PDI of 0.300 or lower and an encapsulation efficiency of at least 95%.

At this point the reader is probably dying to know what these particles would be used for or, if you are like my cat, has fallen asleep. The purpose of the planned experiments was two-fold: demonstrate the ability of antigen presenting cells to take up the mRNA-containing particles and translate the protein, and confirm that antigen presenting cells treated with said particles could stimulate an antigen-specific T cell response. Specifically, we planned to treat dendritic cells, which are potent professional antigen presenting cells, with LNPs containing equal amounts of luciferase- and ovalbumin-encoding mRNA. Protein synthesis would be determined by measuring luminescence following administration of the luciferase substrate, and T cell response would be derived from CFSE dilution of labeled OT-1 cells in co-culture, indicating proliferation upon recognition of their cognate peptide, the ovalbumin epitope SIINFEKL.

Before diving into the results of this set of experiments, I should highlight some of the lessons learned from my first attempts to work with our LNPs in vitro. First, serum concentration matters. In an early experiment working with a cancer cell line, raising the final serum media concentration present in the wells during nanoparticle administration from five to ten percent, which is commonly used in cell culture, reduced luminescence seven-fold, nearly to the level of untreated cells.

Second, in our hands, it was difficult to work with spleen-derived dendritic cells. Besides the yield constraints (only ~0.5-2% of splenocytes are eluted as dendritic cells when using commercially available column-based negative selection kits), I was unable to convincingly transfect these cells. This was regardless of serum concentration, cell number, and the presence or absence of GM-CSF, and it was not due to decreased viability.

Fortunately, bone marrow-derived dendritic cells (BMDCs) provided an alternative antigen presenting cell type that was better suited for these proof-of-concept experiments. In addition to much greater yield per mouse (> 20 million immature BMDCs on the fifth day of culture), BMDCs could be successfully transfected at various time points during differentiation and could be cryopreserved before or after differentiation and then transfected. Additionally, with our specific LNP formulation, serum-free RPMI worked better than PBS or Opti-MEM as diluent for administering treatments 1:1 to BMDCs in serum-containing media.

Finally, I noticed that as particles grew older, there was a corresponding decrease in luminescence. This could not be explained by changes in mRNA concentration or encapsulation efficiency over time or by the formation of large particle aggregates, which were absent when the LNPs were reassessed. However, an easy remedy to this predicament was to use the particles as soon as possible after formulation.

Post-troubleshooting, the experimental design to evaluate protein synthesis was as follows: Murine bone marrow was collected and cultured with GM-CSF in a low-binding six-well plate for five days to differentiate BMDCs. Cells were then plated in a 96-well plate (200,000 cells per well), rested overnight, and treated with LNPs ~12 hours later. Luminescence was measured 24 hours after adding the LNPs. Results, shown in Figure 1, demonstrated that the luciferase protein was present following treatment with any of the LNP doses tested. Though not directly measured, we can be fairly confident that the steps preceding translation also occurred: our LNPs were taken up by the BMDCs, then mRNA was released into the cytoplasm, where it was translated into protein. The next step was to determine whether or not these proteins were processed and presented adequately to stimulate a T cell response.

To address this question, I proceeded to the portion of the project where the ovalbumin mRNA gets the spotlight. OT-1 cells, which are transgenic CD8+ T cells bearing receptors that recognize the SIINFEKL peptide (derived from the ovalbumin protein) when bound to the H-2Kb MHC class I molecule, were a useful immunological tool for these experiments. The design was such that OT-1 cells would be isolated from splenocytes, labeled with CFSE, and then cultured with LNP-pulsed BMDCs. Whether the T cells responded to antigen presentation by the BMDCs would be determined using flow cytometry, where a dilution of CFSE indicates proliferation. Similar to those above, these experiments also required some troubleshooting, mainly to determine an adequate number of each cell type (BMDCs and OT-1 cells) to include per well. Initial attempts revealed that 200,000 of each per well was too many and 50,000 BMDCs plus 75,000 OT-1 cells was not enough.

To incorporate these findings, the following experimental setup was used: Cryopreserved bone marrow was thawed and cultured with GM-CSF in a low-binding six-well plate for five days to differentiate BMDCs. These cells were then plated in a 96-well plate (100,000 cells per well), rested overnight, and treated with LNPs ~15 hours later. Six hours after adding the LNPs, cryopreserved isolated CD8+ OT-1 cells were thawed, labeled with CFSE, and added to the culture (110,000-126,000 cells per well). The co-culture incubated for three days, with half of the media replenished on the second day, and was then stained and acquired on a flow cytometer. Figure 2 shows the flow cytometry gating strategy of a representative sample, and the results of two experiments (each using T cells from a different mouse) are below in Figure 3. These results demonstrate a dose-response relationship between mRNA LNP dose and T cell proliferation, wherein higher concentrations of mRNA led to greater proliferation, indicating that ovalbumin is not only being translated within the antigen presenting cells but is also being processed internally and presented to T cells on major histocompatibility complex molecules.

This brings us to the end of this set of experiments, although work on this project is far from over. Stay tuned to the PIRL blog for future updates, which we anticipate will involve testing the efficacy of mRNA LNP vaccines in vivo.

Note: There are different types of immunogenicity depending on the type of immune response elicited such as humoral immunogenicity (B-cell) and cell-mediated immunogenicity (T-cell). For brevity, all further references of immunogenicity will be referring to cell-mediated immunogenicity, a concept I break down here.

Background & Motivations

T-cell immune responses require the synergistic operation of a number of sub-cellular events by both the antigen presenting cell as well as the responding T-cell. Canonically, these events can broadly be categorized as falling within: antigen presentation, and TCR recognition (shown above). However, there is a third category of events which, for lack of a consensus term, I call promotion/attenuation that also plays a key part in determining immune response. This category encompasses the expanse of immune checkpoint + co-stimulatory interactions with surface receptors on T-cells to either promote or attenuate that cell’s effector function. Predicting immunogenicity requires accurately predicting all three events.

In a perfect world, with unlimited access to information, we could predict immunogenicity of a peptide in an individual by first looking at the different surface MHC molecules and predicting how stable the peptide will bind MHC and stay presented. Then we could look at the individual’s entire TCR repertoire and predict how many TCRs would react and bind the peptide-MHC (pMHC) conjugate. Last, we would need to check the activation/senescence cell states of the reactive TCRs and count how many clones will initiate effector function. Immunogenicity could thus be found as some weighted score involving the presentation, recognition, and activation propensities. In the real world however, we have nowhere near this level of data available, and must go about immunogenicity prediction in more roundabout ways.

Current Landscape of Immunogenicity Prediction

Because data on attenuators and promotors vary on a clonotype level within an individual and across individuals (affected by prior/concurrent infections), along with other variables, much focus has been placed on first predicting presentation and recognition. Tools like NetMHCpan and MHCFlurry have achieved high accuracy in predicting peptide presentation, whereas no tools have emerged that perform on a similar caliber at predicting TCR recognition. Buckley et al benchmarked a number of immunogenicity predictors and reported that immunogenicity prediction (IP) of arbitrary peptides fails to outperform simple predictors of peptide presentation, indicating an apparent need to better predict TCR recognition. I suspect this is due in large part to the fact that training data used for IP is almost always binarized. Thus, to explicitly train models to learn and predict TCR recognition (which can vary wildly), perhaps a more meaningful metric of quantifying TCR recognition propensities needs to be used as training data.

Current Landscape of Available Immunogenicity Data

Immunogencity data as a whole is siloed by assay of origin. To measure the presentation of a specific peptide there MHC presentation assays. Common assays are for measuring T-cell response include: ELISA/ELISpot, multimer, intracellular stains, proliferation assays, and degranulation assays. These assays are optimal for answering various scientific questions. For example, to check for reactive TCRs: a multimer assay would tell if certain TCRs bind a specific multimer. However, to check if TCR recognition leads to an immune response these multimer+ cells would need to be assayed via intracellular stain or other functional assays. To perfectly check for the TCR recognition propensity of a pMHC, we’d need an exhaustive set of the entire TCR space and a mechanism to check what proportion of this set reacts with a given pMHC. However, because the available TCR space is ~10^15 and each TCR has the ability to bind multiple pMHCs, there is a need to collect assay data from many individuals, in hopes of sampling from the set of TCRs. The IEDB is a comprehensive database with such a dataset that looks at studies where the immunogenicity of specific pMHCs were tested across multiple individuals.

Defining TCR Recognition Propensity with IEDB

From the perspective of PIRL, one of our goals is to create personalized cancer vaccines. So one of the reasons we work on IP is to be able to rank putative neoantigens such that the highest ranked peptides are those that selected for inclusion in a vaccine. In this exploration which is available here as an iPython notebook, I took a look at subset of the available IEDB data that was filtered on MHC-I data and was generated from multimer assay data. We specifically chose this assay as a high confidence assay where we could claim that there were reactive TCRs to the given multimer (pMHC). This reduces immunogenicity from presentation and recognition to just recognition because we condition on the event that the peptide has been presented by the MHC on the multimer. A snapshot of the dataset is shown below.

When looking at the data from the IEDB, we run into the issue that for a given peptide MHC, there are a number of different studies reporting the immunogenicity of that pMHC. Each quantify the immunogenicity of the pMHC in slightly different ways, and in some instances, the studies do not agree with each other. Some include quantitative measurements, and some include qualitative measurements with labels spanning (Immunogenic, Immunogenic-strong, Immunogenic-weak, and Non-immunogenic). Thus, a standardized quantification of TCR recognition needs to be crafted that makes use of the variability of the multimers’ ability to bind TCRs and be classified as positive staining. I decided to look at the number of subjects responded over the number of subjects tested as a standardized method of quantifying TCR recognition across a number of TCR repertoires. Like from the second thought experiment scenario, the idea was that when we look across various individuals, the pMHCs with a higher propensity for TCR recognition will stimulate a representative sample of TCRs with a higher frequency.

Once I chose this as my given metric, I needed to perform data imputation for the cases where the number of individuals tested was not given. In this case, I looked through a few of the papers that didn’t cite the number of individuals tested and found from those papers, that only one individual was tested. So I chose n_tested = 1 for those cases. Furthermore, in order to include some degree of uncertainty for the ratios where only a few number of individuals was tested I decided to use Laplace smoothing to add two dummy individuals to the each case: responded as well as didn’t respond. This smoothing made it such that those pMHCs where a large number of individuals were tested and a majority either responded or didn’t respond, the change to the ratio would be minimal. However if only one individual was tested and either responded or didn’t respond the smoothing pulled the ratio down from 1.00 or up from 0.00.

From here I needed to collate various experiments together that looked at the same pMHC. There are a number of options of how to handle the variability among the results we get combining across experiments. We can treat them as one giant experiment and sum over all the total responded and total tested. However, I decided against this because each experiment was performed separately and I felt that batch effects may somehow confound this. I entertained drawing a sample of outcomes with sampling probabilities equal to the empirical distribution, bootstrapping, and even taking the geometric average. For a first pass approach, I ended up choosing a modified geometric mean that is weighted by the number of individuals tested (personal choice that behaved well with the example above and when class switched).

Below we can see what went from a list of disparate metrics of measuring immunogenicity projected onto a standard axis with a biologically relevant meaning. In future work I will be comparing the performance of models trained on this dataset versus models trained on the binarized version of this dataset. This work is still in progress and I aim to also test out other imputation schemes as well as other methods of combining across experiments, but I do anticipate better performance on downstream predictors from this data. The results and pipeline have been made readily available for the community if anyone is working on IP and likewise thinks that there must be something better than binarized data to train on.