stDyer enables spatial domain clustering with dynamic graph embedding

Ke Xu, Yu Xu 许煜, Zirui Wang, Xin Maizie Zhou, Lu Zhang

February, 2025

stDyer framework

Abstract

Spatially resolved transcriptomics (SRT) data provide critical insights into gene expression patterns within tissue contexts, necessitating effective methods for identifying spatial domains. We introduce stDyer, an end-to-end deep learning framework for spatial domain clustering in SRT data. stDyer combines Gaussian Mixture Variational AutoEncoder with graph attention networks to learn embeddings and perform clustering. Its dynamic graphs adaptively link units based on Gaussian Mixture assignments, improving clustering and producing smoother domain boundaries. stDyer’s mini-batch strategy and multi-GPU support facilitate scalability to large datasets. Benchmarking against state-of-the-art tools, stDyer demonstrates superior performance in spatial domain clustering, multi-slice analysis, and large-scale dataset handling.

Type

Journal article

Publication

Genome Biology

Overview

stDyer is an innovative deep learning framework designed for spatial domain clustering in spatially resolved transcriptomics (SRT) data. The framework integrates:

Gaussian Mixture Variational AutoEncoder (GM-VAE) for learning meaningful embeddings
Graph Attention Networks (GAT) for capturing spatial relationships
Dynamic Graph Construction that adaptively links spatial units based on Gaussian Mixture assignments
Mini-batch Strategy and Multi-GPU Support for handling large-scale datasets

Key Features

1. Dynamic Graph Embedding

Adaptively constructs graphs based on clustering assignments
Produces smoother spatial domain boundaries
Captures both transcriptomic similarity and spatial proximity

2. Scalability

Mini-batch processing strategy
Multi-GPU support for large datasets
Efficient handling of high-resolution SRT data

3. Superior Performance

Outperforms state-of-the-art tools in spatial domain clustering
Excellent performance in multi-slice analysis
Robust handling of large-scale datasets

Citation

If you use stDyer in your research, please cite:

@article{xu2025stdyer,
  title={stDyer enables spatial domain clustering with dynamic graph embedding},
  author={Xu, Ke and Xu, Yu and Wang, Zirui and Zhou, Xin Maizie and Zhang, Lu},
  journal={Genome Biology},
  volume={26},
  number={1},
  pages={34},
  year={2025},
  publisher={BioMed Central},
  doi={10.1186/s13059-025-03503-y}
}