Transforming basecalling in genomic sequencing

Every Oxford Nanopore run starts the same way: a nucleotide strand passes through a pore, the ionic current shifts, and a stream of raw signal begins to flow.

Basecalling — how that signal becomes sequence — shapes everything you do downstream, from variant calling to gene expression analysis. So, when we improve basecalling, we are strengthening the foundation of your research.

Through our continuous iterative improvement, our super accuracy (SUP) basecalling model now gives you more accurate results at the same compute budget, meaning you can push your research further with confidence.

Basecalling fundamentals

Translating the raw, electrical signals captured by our sequencing devices is no mean feat — the scale is considerable.

Take our PromethION 24 sequencer as an example: 24 flow cells, each with up to 3,000 channels active at any one time, and every channel is sampling at 5,000 Hz — that’s a maximum output of 360 million signal samples per second!

Transforming that volume of raw signal into accurate sequence, in real time, within realistic compute limits, is the challenge our basecallers are designed to meet.

From LSTMs to transformers

Since 2016, our basecallers have been powered by recurrent neural networks, most recently long short-term memory (LSTM)-based networks. LSTMs excel in environments where historical data points significantly influence future predictions.

However, they process data sequentially, which can limit training and inference speed. The requirement to compress temporal information into a fixed-sized state vector can also limit scalability. This can be a critical factor when aiming to achieve the highest basecalling accuracy within a compute-constrained inference environment.

Transformers take a different approach. Their architecture allows for parallel processing and direct communication between the positions, which in practical terms, means richer contextual modelling with greater efficiency.

Transformers have driven advancements in language modelling for OpenAI’s GPT and Meta’s Llama family of models, as well as protein structure and protein language modelling through Google DeepMind’s AlphaFold and Meta’s Evolutionary Scale Modelling (ESM) models. We've now bought this architectural strength to Oxford Nanopore basecalling.

Advancing SUP performance

Switching our SUP model from LSTM to the transformer architecture has improved a wide range of accuracy metrics, while maintaining the same inference-compute budget. Our internal research shows that as training and inference budgets increase, accuracy continues to climb, showing potential for further gains as we scale.

However, we aren’t doing away with LSTMs. They remain highly competitive at smaller compute budgets. For example, high accuracy (HAC) basecalling can keep pace with a fully loaded PromethION 24 across a sequencing run. Choosing the right architecture for the right context remains central to our approach.

Transformer architecture

The transformer basecaller has three layers (see Figure 2), each with a distinct function:

• Preprocessing: at any given moment, the raw signal reflects a short stretch of nucleotides. We apply a compact stack of 1D convolutional layers to reduce the signal into feature vectors that capture local patterns.

• Transformer: a stack of encoder blocks processes the features. Self-attention enables neighbouring positions to exchange information efficiently. The architecture incorporates rotary position embeddings (RoPE), post-RMSNorm, SwiGLU, and DeepNet residual branch scaling.

• Decoding: the transformer output at each position is projected to a vector containing a score for transitioning between states. The states here correspond to kmers — groups of k nucleotide bases — and the scores are part of a connectionist temporal classification conditional random field (CTC-CRF) output head. Beam search approximates the most probable kmer sequence, which is then converted to nucleotides.

The overall design balances expressive modelling with computational efficiency, tailored specifically to nanopore signal characteristics.

Performance and optimisation

The transformer architecture has dramatically increased the accuracy of our SUP model, but we can’t forget that transformers are computationally demanding, particularly in high-throughput sequencing environments. To meet that challenge, we developed custom, hand-tuned compute unified device architecture (CUDA) and Metal kernels to precisely control graphics processing unit (GPU) operations and maximise throughput.

Key innovations for performance and scalability:

• Sliding window multi-head attention restricts attention to a local window of neighbouring positions, ensuring computational complexity scales linearly, rather than quadratically, with sequencing length.

• Internal sequence compression reduces input length before the transformer stack and restores it during decoding, significantly improving throughput and reducing compute cost.

Our LSTM implementations remain highly optimised too. They leverage NVIDIA INT8 Tensor Cores, using quantisation to approximate floating-point operations with 8-bit integers while maintaining basecalling accuracy. Persistent kernel techniques allow recurrent weights to be cached efficiently within GPU registers to maximise memory bandwidth.

We will continue to optimise both transformer and LSTM models at the architectural and implementation levels.

What this means for you

Improved basecalling accuracy strengthens the foundation of all downstream analysis. Whether you are studying structural variation, isoforms, or epigenetics, accurate basecalls increase confidence in your results.

For researchers exploring areas such as cancer genomics or complex disease biology, more accurate sequencing data can help reveal subtle variation across large datasets. Our aim is simple: give you data you can trust, so you can focus on the biological questions that matter.

What’s next?

Our machine learning team is actively exploring further model compression techniques to reduce computational cost while preserving performance. We currently quantise our models, so operations execute using 8-bit integers, rather than 16- or 32-bit floats, making computation run faster.

We are also investigating knowledge distillation, sparsity methods, and alternative decoder heads such as autoregressive next-base prediction to better understand performance benefits and trade-offs.

The adaptability of transformers to different computational platforms promises explorations into alternative hardware acceleration. This has the potential to broaden the scope of genomic research and applications and deliver on our vision to enable the analysis of anything, by anyone, anywhere.

In our commitment to openness and collaboration, we are releasing the transformer model’s source code for training and inference, and our model weights. We invite you to engage, adapt, and build on it. Innovation moves faster when we work together.

A clearer signal is just the beginning. Explore our beginner’s guide to file formats to make the most of every run.

Oxford Nanopore Technologies products are not intended for use for health assessment or to diagnose, treat, mitigate, cure, or prevent any disease or condition.