Case of CLOS Architecture in China’s Data Center
The design of the business network of the computer room construction project of the data center of a Chinese central enterprise adopts 25G-100G networking, with 25G access, and 100G links for Leaf and core layers.
Compared with 40G-10G networking, it provides 2.5 times the network capacity, which can meet the network requirements of current mid-to-high-end servers and storage.
Monitoring room data center network server weak current cabinet 2 meters
CLOS architecture networking
CLOS architecture networking, high-performance load-sharing links for east-west traffic within POD, ensures the internal requirements of modern data centers. At the same time, CLOS architecture provides horizontal elastic scalability, on-demand deployment, and smooth expansion, which has reached the networking standards of large Internet data centers. The current network design, combined with the application of EVPN and RDMA technology, builds a high-performance data center that supports virtualization.
- High performance: high bandwidth and throughput performance in the east-west direction, line-speed forwarding of all packet lengths, ultra-low latency network
- Standardization: refer to the design scheme recommended by the IETF standard, scientific and open
- Economic and efficient: box-type equipment networking, reducing single investment; low power consumption, unified intelligent management, efficient operation and maintenance, low maintenance cost
- Advanced technology: Ethernet-based RDMA technology (RoCE) accelerates and improves storage and GPU business performance
- Flexibility: on-demand deployment, horizontal expansion, and providing differentiated connection methods according to business node requirements
- High reliability: through routing load sharing, all traffic provides high reliability across devices
- Virtualization: EVPN networking, support for virtualized business applications and deployment, and perfect support for VM/container migration
- Universality: universal network equipment requirements, more providers to choose from, not bound by frame devices
The module rooms under the data center computer room include: CPU cluster, GPU cluster, storage cluster, etc. The production network and office network equipment of the core business use Mellanox3700 as the core switch, 2700 as the Leaf switch, and 2410 as the TOR access device.
Module Description
In terms of network equipment selection, high bandwidth and low latency requirements for business-level transmission are given priority (25G transmission for single node downlink, 100G transmission between switches, and 40G bandwidth cross-link between security devices), which fully guarantees the transmission performance of the upper and lower layers of the network and the bandwidth requirements of east-west traffic.
The switch systems in this solution are all Mellanox’s open Cumulus system. The latest Unnumbered BGP protocol is used in the project configuration phase to simplify the configuration difficulty between devices. At the same time, the network system continues to use the LINUX control command line to facilitate the use and maintenance of network administrators in the later stage.
VXLAN technology
EVPN-VXLAN enables enterprises to use Layer 2 virtual bridging to connect geographically dispersed locations. EVPN-VXLAN provides the scale that cloud service providers need and is often the technology of choice for data center interconnects.
EVPN is a multi-tenant, highly scalable overlay that typically uses resources from different data centers to deliver a single service. It can provide Layer 2 connectivity or enable Layer 3 routing for devices in a virtual network over the physical infrastructure.
Because it acts as a MAC address learning control plane for overlay networks, EVPN can support different data plane encapsulation technologies. This flexibility is particularly attractive for network fabrics that are not strictly based on MPLS.
VXLAN encapsulates Layer 2 Ethernet frames in Layer 3 UDP packets, which means that virtual Layer 2 subnets can span the underlying Layer 3 network. Similar to traditional VLAN IDs, VXLAN network identifiers (VNIs) are used to segment each Layer 2 subnet.
A VXLAN tunnel endpoint (VTEP) is a VXLAN-enabled device that encapsulates and decapsulates packets. In a physical network, switches typically act as Layer 2 or Layer 3 VXLAN gateways and are considered hardware VTEPs. The virtual network equivalents are called software VTEPs, and they are hosted in hypervisors such as VMware ESXi or vSphere.
RoCE
RDMA over Converged Ethernet (RoCE) is a network protocol that leverages Remote Direct Memory Access (RDMA) capabilities to significantly accelerate communications between applications hosted on server clusters and storage arrays.
RoCE incorporates IBTA RDMA semantics, allowing devices to perform direct memory-to-memory transfers at the application level without involving the host CPU.
Both transfer processing and memory translation and placement are performed by hardware, resulting in significantly lower latency, higher throughput, and better performance compared to software-based protocols.
About IoT Cloud Platform
IOT Cloud Platform (blog.iotcloudplatform.com) focuses on WiFi modules, IoT programming, sensor programming, smart home sensors, CNC machine tools and IoT programming, smart machine tools, motion valves, mechanical valves, water pump control valves, smart gateways, smart homes, smart communities, smart nursing homes, smart cities, RFID, lora devices, IoT systems, IOT modules, IOT solutions, lithium batteries and other technology products and knowledge.
