Abstract Automatic and high-precision video-based animal re-identification (Re-ID) has become an important tool for wildlife conservation and behavioral research. However, state-of-the-art image-based methods, while effective on static images, fail to make full use of the rich dynamic information in videos, such as gait and motion patterns. To bridge this gap, we propose HST-Former, a novel framework that extends data efficiency and species-agnostic principles into the temporal domain. The core innovation of this method is the Hierarchical Spatio-Temporal Transformer Aggregator (HSTTA)—a customized transformer architecture designed to process and integrate all local features from an entire animal trajectory. By modeling both spatial and temporal feature dependencies, HSTTA learns complex long-range relationships and produces a single, highly discriminative video-level descriptor. The hierarchical design first summarizes intra-frame features and then aggregates them across frames, effectively addressing the computational challenges of standard transformers on long sequences. In addition, we introduce a spatio-temporal consistency constraint to enhance the geometric verification step, improving re-ranking accuracy. Our model significantly outperforms current state-of-the-art baselines. To validate its effectiveness, we conduct comprehensive evaluations on three public datasets, where HST-Former achieves the best performance across all key metrics, including Top-1, Top-3, and Top-5. Similar content being viewed by others Data availability The datasets used during the current study are available from the corresponding author on reasonable request. References Araujo, A. et al. Recurrent neural networks for person re-identification revisited. IEEE Trans. Multimed. 21, 2813–2824 (2019). Whytock, R. C. et al. Optimizing the automated recognition of individual animals to support population monitoring. Ecol. Evol. 13, e10210 (2023). Crouse, D. et al. Identification of animal individuals using deep learning: A case study of the giant panda using infrared camera images. Biol. Conserv. 242, 108405 (2020). Francis, D. Review of the current research in animal individual recognition. Modern Sci. 1, 80–85 (2023). Clapham, M.,