Historical background

Many things have changed in the data center world over the last decade. In fact, so much has changed that the “old” three-layer model of access, aggregation/distribution, and core is no longer ideal.  This 3-Tier data center networks were the generally recommended data center network design in the past. They worked very well since the majority of traffic moved were North-South (from outside the data center in) or vice versa, as shown in the diagram. A packet flows to the core, is routed to the correct distribution switch, then forwarded on to the access switch where the endpoints were connected; moving through only 3 physical hops which limits the amount of latency added per-packet flow.  

The issue with 3-tier design

The issue with this 3-tier design for the modern data center is that

  • much more intra-Data Center traffic is the new normal and due to server to server traffic flow, three hops now quickly become four, five or more thereby adding significant latency per data flow as well as adding more possibility for bottlenecks, buffer overflow and dropped packets.
  • 3-Tier data center networks introduced loops, which requires correct spanning-tree protocol configuration. Spanning-tree issues are notorious for causing network outages as a spanning-tree failure causes continuous looping.

Thanks to virtualization technologies of data, there is much more of an “east to west” flow of data between virtualized workloads in an expanded layer 2 domain.

The Solution 

A 2-Tier, or spine-leaf architecture (also called Folded-CLOS), to meet the needs of modern applications: high-throughput and low-latency is recommended.

Two-Tier spine-leaf network design

Leaf-spine is a two-layer data center network topology that’s useful for data centers that experience more east-west network traffic than north-south traffic.

The topology is composed of leaf switches (to which servers and storage connect) and spine switches (to which leaf switches connect). Leaf switches mesh into the spine, forming the access layer that delivers network connection points for servers.

Every leaf switch in a leaf-spine architecture connects to every switch in the network fabric. No matter which leaf switch a server is connected to, it has to cross the same number of devices every time it connects to another server. (The only exception is when the other server is on the same leaf.) This minimizes latency and bottlenecks because each payload only has to travel to a spine switch and another leaf switch to reach its endpoint. Spine switches have high port density and form the core of the architecture.

  • A. A spine switch and a leaf switch can be added with redundant connections between them.
  • B. A spine switch can be added with at least 40 GB uplinks.
  • C. A leaf switch can be added with a single connection to a core spine switch.
  • D. A leaf switch can be added with connections to every spine switch.

Only leaf-to-spine. No leaf-to-leaf and spine-to-spine links exist.

Traffic between servers within the data center