Dynamic Loads Demand a Dynamic Network: How Hyve Switches Optimize for AI on the Fly
AI workloads punish networks with traffic patterns that shift in milliseconds. To connect 100,000 GPUs, static configurations fail. Hyperscalers need intelligent networks that adapt on the fly, rerouting traffic at the silicon level to prevent bottlenecks before they form.
Hyve’s hyperscale infrastructure approach combines hardware intelligence with adaptive software to create networks that tune themselves. Our switches prevent bottlenecks before they form, rerouting traffic patterns at the silicon foundation. Concurrently, Hyve switches implement thermal solutions that maintain breakthrough performance even as raw connectivity speeds double.
The Tiered Network Challenge
Modern hyperscale deployments follow a relatively predictable ratio: for example, three compute racks to one networking rack. Scaling that ratio across hundreds of racks creates a tiered architecture wherein each layer adds latency and potential congestion points.
The math compounds quickly. Connect 512 GPUs through a single switch , and you minimize hops. Scale to 100,000 GPUs across multiple tiers (scale-out), and traffic must navigate aggregation layers and spine switches. Each hop adds microseconds that collectively multiply across millions of training iterations.
Conventional approaches to this congestion, such as static buffering, fixed routing tables, and manual threshold adjustments, can’t respond fast enough. Networks need to self-optimize.
Hardware Foundation: Tomahawk Ultra and Tomahawk 6
As we discussed in prior blogs, Hyve’s new switch line builds on Broadcom’s latest switch ASICs, each of which optimizes for different deployment patterns. The Tomahawk Ultra targets tightly coupled GPU clusters where latency matters most. The Tomahawk 6 doubles throughput for massive scale-out fabrics. The following table gives a quick comparison of these two options.
Key specifications:
| Feature | Tomahawk Ultra | Tomahawk 6 |
| Total Switching Capacity | 51.2 Tbps | 102.4 Tbps |
| Switch Latency | 250 nanoseconds | Optimized for throughput |
| Packet Performance | 77 billion packets/sec (line rate) | ~38 billion packets/sec (throughput-optimized) |
| SerDes Configuration | 512 lanes @ 100G | 1,024 @ 100G or 512 @ 200G |
| Port Density | 64 ports @ 800GbE | 128 @ 800GbE or 64 @ 1.6TbE |
| Process Technology | 5nm | 3nm |
| Form Factor | Standard 2RU | 2RU with DLC option |
| Lossless Features | LLR, CBFC | LLR, CBFC, Cognitive Routing 2.0 |
| Pin Compatibility | 100% with Tomahawk 5 | New architecture |
Understanding the SerDes (serializer/deserializer) difference is key to appreciating these performance gaps. In this context, SerDes pertains to the high-speed electrical circuits connecting the switch ASIC to physical ports. It allows high data-rate transfer over fewer physical channels, reducing the number of required I/O pins and signal traces. The faster the SerDes, the higher the throughput. However, faster SerDes also means more power consumption and signal integrity challenges. Moving from 100G to 200G SerDes doubles capacity but demands sophisticated thermal management and PCB design.
The apparent drop in packet performance is potentially misleading. The Tomahawk Ultra prioritizes raw packet processing speed for latency-sensitive tasks, while the Tomahawk 6 architecture optimizes for maximum data throughput in massive scale-out fabrics. It’s a classic engineering trade-off between latency and bandwidth, and capturing that in speed specs is a bit like quantifying a bulldozer in miles per hour.
This choice in hardware is the first layer of intelligence, allowing Hyve to engineer solutions that master the competing demands of AI workloads from the silicon up.
Intelligent Congestion Management
Hyve switches implement multiple layers of adaptive control:
Credit-Based Flow Control (CBFC) operates at the hardware level. When receiver buffers fill, the switch signals upstream devices to pause transmission. Unlike older Priority Flow Control mechanisms, CBFC tracks buffer credits per flow, enabling finer-grained control.
Dynamic buffer allocation allows ports experiencing traffic bursts to borrow headroom from quieter ports. Rather than statically partitioning buffer space, the switch reallocates resources based on instantaneous demand.
In-band telemetry monitors queue depths, packet drop rates, and flow completion times across the fabric. This telemetry feeds into adaptive routing algorithms that steer traffic away from congested paths.
Explicit Congestion Notification (ECN) marks packets when queue depths exceed thresholds, signaling endpoints to reduce transmission rates before packet loss occurs. The key innovation: thresholds adjust dynamically based on traffic patterns rather than relying on manual configuration.
Switch-to-NIC Communication
Network interface cards (NICs) in GPU servers need coordination with switches to prevent packet drops. The switches send pause frames when buffers approach capacity, giving NICs time to throttle transmission rates.
This bidirectional communication extends to Link Layer Retry (LLR), a sort of error-correction mechanism that detects transmission faults at the physical layer and requests retransmission without involving higher network layers. By handling retries at the link level, LLR reduces end-to-end latency and prevents application-level timeouts.
Thermal Engineering for 200G SerDes
The transition from 800 Gbps to 1.6 Tbps Ethernet represents a true doubling of lane speeds from 100G to 200G SerDes. This progression follows IEEE 802.3dj standards and brings significant thermal challenges.
The Tomahawk 6 ASIC consumes 20% more power than previous generations. Optical transceivers now approach 20 to 25 watts each. Traditional air cooling would require expanding switches from 2RU to 4RU form factors, but many hyperscalers cringe at sacrificing precious rack space.
Hyve’s direct liquid cooling (DLC) solution cools both the ASIC and optical transceivers. Cold plates mounted directly on components remove heat more efficiently than air fins, maintaining controlled operating temperatures while preserving rack density. Hyve’s DLC engineering helps keep the entire switch solution in a 2RU package.
Impact on Hyperscale AI
These optimizations target AI/ML hyperscalers operating clusters of 100,000+ GPUs. The benefits of this switch evolution cascade:
Reduced training time: Lower network latency means GPUs spend less time waiting for gradient updates during distributed training. A 50% latency reduction doesn’t translate to 50% faster training, but even 10 to 15% improvements save weeks on month-long training runs.
Higher GPU utilization: Congestion-free networking keeps GPUs fed with data. When networking bottlenecks, expensive compute sits idle.
Predictable performance: Lossless fabrics eliminate timeout-and-retry cycles that cause unpredictable job completion times. Consistent performance leads to better capacity planning.
Lower total cost: Fewer switch tiers (enabled by higher radix switches) mean fewer optics, less power consumption, and reduced physical footprint. A three-tier network consuming 67% more optics and roughly 2x the network power becomes a two-tier network.
The vertical integration matters here. Hyve controls both switch manufacturing and rack integration, enabling optimization across the entire stack. Switches arrive pre-configured for customer topologies, cabled and tested, ready for immediate deployment.
Hyve positions at the forefront of this transition with DLC-enabled Tomahawk 6 platforms, combining Broadcom’s silicon innovation with intelligent network orchestration. Adopting datacenters achieve networks that scale performance without scaling physical space, adapting dynamically to AI workload demands rather than requiring constant manual tuning.
Ultimately, the era of static network configuration is ending. As AI infrastructure scales monumentally, the network must evolve from a passive fabric to an intelligent, self-optimizing system. By integrating Broadcom’s leading-edge silicon with sophisticated thermal engineering and adaptive software, Hyve delivers networks that scale performance within the same footprint and ensure that customers’ most valuable compute resources are never left waiting.