1. Introduction Over the past decade, deep learning as a field has grown quite significantly, whether it be the compute capacity of hardware or the ingenuity behind architectures that utilize that hardware. But if you think about it for more than a second, the underlying architecture has remained consistent in a few key areas. We’ve seen a massive shift from convolutional networks to the new Transformer architectures that power today’s large language models, but the way these networks route information from one layer to another hasn’t changed all that much. Recently, researchers at DeepSeek-AI released a paper titled “mHC: Manifold-Constrained Hyper-Connections,” (Xie et al., 2025b)1 which proposes an entirely new redesign of this routing system. To really appreciate the solution they came up with, let’s look at how signal propagation has evolved over the past few generations of models, and why the current methods are hitting a wall. 2. The Backbone: Standard Residual Connections Firstly, to understand the specific problem that the authors are trying to solve, we need to talk about where it all started–The standard Residual Connection (He et al., 2015)2. Introduced back in 2015 with ResNets, the residual connection is arguably one of the most important architectural design choices used in every AI model out there. Mathematically, it looks like this: It simply means that the final output of a layer is the sum of its output and the input it originally got. The key component here is that bare xl term in the residual stream, which we call the identity mapping. It’s important because it acts as an uninterrupted pathway for the gradient signal to flow through the entire network from start to finish. This property is exactly what prevents gradients from vanishing or exploding during training and allows us to successfully train models with hundreds