Abstract In this paper, a hybrid convolutional neural network (CNN) and transformer architecture for image classification that explicitly exploits multiscale spatial representations while maintaining computational efficiency is proposed. A convolutional backbone is first used to extract hierarchical feature maps, which are subsequently tokenized and fused through a cross-scale token fusion (CSTF) mechanism. In addition, several token-level and CNN-level pruning strategies are evaluated to examine whether redundant spatial tokens or convolutional features can be removed without substantially degrading performance. Extensive experiments on Caltech-101 and Oxford-IIIT Pets under low-data training conditions show that the best proposed configurations are dataset-dependent: the single 14\(\:\times\:14\) scale achieves 70.28\(\:\pm\:1.91\)% accuracy on Caltech-101, while the single 7\(\:\times\:\)7 scale achieves 28.35\(\:\pm\:1.62\)% accuracy on Oxford-IIIT Pets. Among the multi-scale fusion models, 7\(\:\times\:\)7+14\(\:\times\:\)14 performs best on Caltech-101 69.68 \(\:\pm\:\)1.77%, whereas 7\(\:\times\:\)7+14\(\:\times\:\)14+28\(\:\times\:\)28 performs best on Oxford-IIIT Pets 27.57\(\:\pm\:\)1.05%. CNN kernel pruning achieves the strongest pruning performance on both datasets, reaching 68.37\(\:\pm\:\)1.06% on Caltech-101 and 28.15\(\:\pm\:\)1.15% on Oxford-IIIT Pets. These results indicate that multi-scale token fusion can provide competitive performance, but careful scale selection is more important than simply increasing the number of spatial token scales. Author information Authors and Affiliations Corresponding author Ethics declarations Competing interests The authors declare no competing interest. Additional information Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary Information Below is the link to the electronic supplementary material. Rights and permissions Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are