Bommasani, R. et al. On the alternatives and dangers of basis fashions. Preprint at https://arxiv.org/abs/2108.07258 (2021).
Dalla-Torre, H. et al. Nucleotide Transformer: constructing and evaluating sturdy basis fashions for human genomics. Nat. Strategies 22, 287–297 (2025).
Nguyen, E. et al. HyenaDNA: long-range genomic sequence modeling at single nucleotide decision. In Adv. Neural Data Processing Programs Vol. 36 https://proceedings.neurips.cc/paper_files/paper/2023/hash/86ab6927ee4ae9bde4247793c46797c7-Abstract-Conference.html (2023).
Nguyen, E. et al. Sequence modeling and design from molecular to genome scale with Evo. Science 386, eado9336 (2024).
Fu, X. et al. A basis mannequin of transcription throughout human cell sorts. Nature 637, 965–973 (2025).
Chen, J. et al. Interpretable RNA basis mannequin from unannotated information for extremely correct RNA construction and performance predictions. Preprint at https://arxiv.org/abs/2204.00300 (2022).
Li, S. et al. CodonBERT massive language mannequin for mRNA vaccines. Genome Res. 34, 1027–1035 (2024).
Lin, Z. et al. Evolutionary-scale prediction of atomic-level protein construction with a language mannequin. Science 379, 1123–1130 (2023).
Nijkamp, E. et al. ProGen2: exploring the boundaries of protein language fashions. Cell Syst. 14, 968–978 (2023).
Chen, B. et al. xTrimoPGLM: unified 100-billion-parameter pretrained transformer for deciphering the language of proteins. Nat. Strategies 22, 1028–1039 (2025).
Jumper, J. et al. Extremely correct protein construction prediction with AlphaFold. Nature 596, 583–589 (2021).
Abramson, J. et al. Correct construction prediction of biomolecular interactions with AlphaFold 3. Nature 630, 493–500 (2024).
Zhang, Z. et al. Protein illustration studying by geometric construction pretraining. In Worldwide Convention on Studying Representations (ICLR) https://openreview.net/forum?id=to3qCB3tOh9 (2023).
Theodoris, C. V. et al. Switch studying permits predictions in community biology. Nature 618, 616–624 (2023).
Hao, M. et al. Massive-scale basis mannequin on single-cell transcriptomics. Nat. Strategies 21, 1481–1491 (2024).
Cui, H. et al. scGPT: towards constructing a basis mannequin for single-cell multi-omics utilizing generative AI. Nat. Strategies 21, 1470–1479 (2024).
Bai, D. et al. scLong: a billion-parameter basis mannequin for capturing long-range gene context in single-cell transcriptomics. Nat. Commun. 17, 2380 (2026).
Luo, R. et al. BioGPT: generative pre-trained transformer for biomedical textual content era and mining. Transient. Bioinform. 23, bbac409 (2022).
Ma, J. et al. Phase something in medical photographs. Nat. Commun. 15, 654 (2024).
Bunne, C. et al. Tips on how to construct the digital cell with synthetic intelligence: priorities and alternatives. Cell 187, 7045–7063 (2024).
Xing, E. P., Deng, M. & Hou, J. Critique of world mannequin. Preprint at https://arxiv.org/abs/2507.05169 (2025).
Xing, E. & Track. L. A world mannequin of the digital cell. GenBio https://genbio.ai/research/virtual-cell-may-3.pdf (3 Might 2026).
Devlin, J., Chang, M. -W., Lee, Okay. & Toutanova, Okay. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Convention of the North American Chapter of the Affiliation for Computational Linguistics (NAACL-HLT) 4171–4186 https://aclanthology.org/N19-1423/ (2019).
Radford, A., Narasimhan, Okay., Salimans, T. & Sutskever, I. Bettering language understanding by generative pre-training. OpenAI https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf (2018).
van den Oord, A., Vinyals, O. & Kavukcuoglu, Okay. Neural discrete illustration studying. In Adv. Neural Data Processing Programs Vol. 30, 6306–6315 (Curran Associates, 2017).
van den Oord, A., Li, Y. & Vinyals, O. Illustration studying with contrastive predictive coding. Preprint at https://arxiv.org/abs/1807.03748 (2018).
Radford, A. et al. Studying transferable visible fashions from pure language supervision. In Proc. thirty eighth Worldwide Convention on Machine Studying (ICML), PMLR Vol. 139, 8748–8763 (2021).
Chen, B. et al. MSAGPT: neural prompting protein construction prediction through MSA generative pre-training. Adv. Neural Inf. Course of. Syst. 37 https://proceedings.neurips.cc/paper_files/paper/2024/hash/41f3347f8f47c17bbadaed584e68d8bd-Abstract-Conference.html (2024).
Solar, N. et al. Combination of specialists allow environment friendly and efficient protein understanding and design. Preprint at bioRxiv https://doi.org/10.1101/2024.11.29.625425 (2024).
Li, P., Cheng, X., Track, L. & Xing, E. Retrieval augmented protein language fashions for protein construction prediction. Preprint at bioRxiv https://doi.org/10.1101/2024.12.02.626519 (2024).
Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. In fifth Intl. Convention on Studying Representations (ICLR, 2017).
Dai, H., Dai, B. & Track, L. Discriminative embeddings of latent variable fashions for structured information. In Proc. thirty third Worldwide Convention on Machine Studying (ICML), PMLR Vol. 48, 2702–2711 https://proceedings.mlr.press/v48/daib16.html (2016).
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In Proc. thirty fourth Worldwide Convention on Machine Studying (ICML), PMLR Vol. 70, 1263–1272 https://proceedings.mlr.press/v70/gilmer17a.html (2017).
Ellington, C. N. et al. Correct and common DNA representations emerge from genome basis fashions at scale. Preprint at bioRxiv https://doi.org/10.1101/2024.12.01.625444 (2024).
Zou, S. et al. A big-scale basis mannequin for RNA perform and construction prediction. Preprint at bioRxiv https://doi.org/10.1101/2024.11.28.625345 (2024).
Zhang, J. et al. Balancing locality and reconstruction in protein construction tokenizer. Preprint at bioRxiv https://doi.org/10.1101/2024.12.02.626366 (2024).
Ho, N. et al. Scaling dense representations for single cell with transcriptome-scale context. Preprint at bioRxiv https://doi.org/10.1101/2024.11.28.625303 (2024).
Lundberg, S. M. & Lee, S.-I. A unified strategy to deciphering mannequin predictions. In Adv. Neural Data Processing Programs Vol. 30, 4765–4774 (Curran Associates, 2017).
Lengerich, B. et al. Contextualized machine studying. Preprint at https://arxiv.org/abs/2310.11340 (2023).
