Company

Equinix Steps Up to 10 Gigabit Ethernet Exchange Services with Force10 Networks

Today, the continued rise of broadband traffic, a competitive bandwidth pricing environment and high core router costs are squeezing Internet Service Provider (ISP) margins while forcing continued network expansion. For ISPs, one of the largest overhead costs is Internet peering, where they physically interconnect their network to all other ISPs, to ensure global Internet reach. It is the Internet Exchanges (IXs) that enable scalable, cost-effective Internet peering for hundreds of ISPs by providing a neutral meeting place and large-scale Ethernet switch fabrics to aggregate inter-network traffic. And as the Internet has increased in importance, IXs have had to meet increasingly stringent reliability requirements.

Equinix (NASDAQ: EQIX), the leading provider of IX services, has seen the traffic in its expansive Internet Business Exchange (IBX) centers rapidly increase to the point that customers are now demanding widespread 10 Gigabit Ethernet services. To continue scaling, increase reliability and access dependable line-rate features, the company deployed the Force10 TeraScale E-Series family of switch/routers as the foundation of its next generation Equinix Exchange™ solution.

Adding Capacity to the Internet

With an excess of 1.3 million square feet across 15 IBX centers in key cities throughout the U.S. and Asia Pacific region, Equinix offers direct access to more than 200 networks, including all major broadband providers, global Tier 1 backbones and large content portals. The company’s IBX centers leverage leading data center design principles to ensure high volumes of reliable power, cooling, physical security, and copper and fiber cross-connect wiring. In these high performance data centers, Equinix has built a unique ecosystem of customers, including Fortune 500 IT integrators, managed service providers, large enterprises, financial institutions, performance-sensitive online gaming companies and high-end content providers, that leverage each other’s services, enabling what once was a cost center for carriers to become a booming profit center.

While operating its IX service on legacy Gigabit Ethernet switches, Equinix saw solid trends as early as two years ago leading to the inevitable need for much higher port speeds and densities. To accommodate the anticipated growth, Equinix launched an extensive RFP for the best carrier-class 10 Gigabit Ethernet switch.

"Although our biggest need was for sheer Gigabit and 10 Gigabit Ethernet port densities, we also had important reliability and feature issues to address," said Lane Patterson, director of research and development for Equinix. "In a cost competitive Ethernet marketplace, we weren’t sure there was a vendor that could adequately address all three areas. Then we began testing Force10 and were really impressed with their uncompromising carrier-class architecture."

Delivering Long-Term Scalability

To build in the scalability it required, Equinix deployed the Force10 TeraScale E1200 and E600 switch/routers with a combination of line-rate 48-port copper and fiber Gigabit Ethernet cards and line-rate 4-port 10 Gigabit Ethernet cards.

"It was important to have a solution that couldn’t be oversubscribed, because our ISP customers already aggregate their traffic to us and fully utilize their ports," said Patterson. "In our largest centers, the 14-slot chassis [E1200] was critical for our projected uptake of 10 Gig ports. No one wants to waste a lot of ports trunking together a large cluster of switches, and the E1200 allowed us to meet our scalability goals with a single pair of redundant switches."

Supporting an industry-leading 672 line-rate Gigabit Ethernet and 56 line-rate 10 Gigabit Ethernet ports per chassis, the Force10 TeraScale E-Series provides Equinix with the high performance scalability it requires to seamlessly expand its exchanges as demand increases.

Additionally, the Force10 TeraScale E-Series provides a level of investment protection that extends the useful life of the product well beyond the typical five to seven years of competitive products. With its five Terabit per second backplane, the TeraScale E-Series supports industry-leading densities today and, more importantly, can support future density increases as well as the transition to 100 Gigabit Ethernet without the high cost of a backplane upgrade.

Patterson estimates the Force10 TeraScale E-Series has a useful equipment life fully 50 percent longer than other alternatives. "When we applied our demand forecasts to each vendor, we were impressed to see how much longer the Force10 equipment would last for us. Considering the costs, staff resources and customer impact of upgrading, it was a huge advantage on multiple fronts."

Delivering Reliability for Increased Performance

In addition to its scalability requirement, Equinix had strict reliability and uptime goals to meet as well. With service level agreements guaranteeing 99.999 percent uptime, Equinix can only have 315 seconds of downtime per year.

"The world’s largest networks rely on our exchanges," said Patterson. "If we have problems, the entire Internet takes notice. Reliability is our number one concern."

To ensure maximum uptime in its exchanges, Equinix leveraged the high resiliency of the E-Series, which combines a fully redundant architecture, including fault-tolerant switch fabrics with hitless failover technology that ensures the secondary components are activated with zero packet loss in the event of a failure.

"Reliability places high demands on every part of a switch," said Patterson. "Force10 was able to meet these demands with proper support for features in hardware, a robust multi-processor control plane, hitless failover on the processor cards and a modular FTOS software architecture with plenty of process prioritization, protection and debug support so that when bugs do arise, the impact is isolated and quickly diagnosed."

Key to the reliability of the Force10 TeraScale E-Series is its three CPU architecture, which distributes switching, routing and management functionalities between three distinct processors, protecting each control plane process and ensuring continued packet forwarding at line rate. The TeraScale E-Series also has redundant power supplies and switch fabric modules, further increasing reliability.

In addition to the reliability that is built into its hardware, Force10 has brought reliability into its software as well. Software development, quality assurance testing and support are an integral part of delivering reliability. By maintaining a rigorous discipline around software release planning as well as extensive real-world feature and load testing, Force10 ensures that new features never compromise its primary goal of providing carrier-class reliability.

Combining High Performance Features and Scalability

To ensure predictable network performance, Equinix is leveraging several key features of the Force10 TeraScale E-Series, including sFlow, line-rate ACL support, Rapid Spanning Tree and flexible link aggregation.

To provide comprehensive interconnection statistics to customers and billing support for the Equinix Direct product, Equinix is using Force10’s sFlow feature. As a traffic analyzing tool, sFlow provides powerful statistics to understand traffic patterns before they lead to congestion, identifies unauthorized network activity and DDoS attack patterns and understands Internet application mix.

Equinix also makes use of media access control (MAC) access control lists (ACLs) on every customer port to provide an increased layer of security and stability. The Force10 TeraScale E-Series supports one million ACLs at line rate, enabling Equinix to provide a secure peering environment for its customers.

In addition to providing scalable security as Equinix grows, the Force10 TeraScale E-Series increases reliability with its rapid failover. Using dark fiber and 10 Gigabit Ethernet dense wavelength division multiplexing (DWDM), Equinix has linked same-metro IBX centers to accommodate increasing traffic. To ensure resiliency on this metro ring architecture and provide rapid failover, Equinix relies on Force10’s support for IEEE 802.1s Multi-Instance Rapid Spanning Tree.

"Getting away from traditional Spanning Tree is really important to us," says Patterson. "Customers expect fully protected paths between our metro sites with rapid failover, and 802.1s provides a standards-based solution for that."

Additionally, the Force10 TeraScale E-Series provides flexible link aggregation support that allows Equinix to scale both internal and customer links in an incremental fashion.

One of the biggest advantages of the Force10 TeraScale E-Series for Equinix has been the flexibility to combine features and the scalability to apply these features on hundreds of ports carrying line-rate traffic.

"One of the factors that differentiated the Force10 architecture was the testing we conducted with feature combinations. In one test, we created a logical link aggregation interface, applied a MAC ACL to it, turned on sFlow, and mirrored it to another port. And our scalability tests showed we could expect to use these features on hundreds of ports on the same chassis without hitting limitations," explained Patterson. "That points to a combination of really good software and hardware architecture plus obsessive QA [quality assurance] testing."

Force10 Networks: Committed to Delivering Carrier-Class Infrastructure

Force10 Networks delivered the scalability, reliability and features required to meet Equinix’s rigorous carrier-class Ethernet RFP. Industry-leading density, a high level of resiliency and a robust feature set enable the TeraScale E-Series to deliver unmatched scalability and the flexibility that enables Equinix to reliably connect the world’s ISPs. And with a significantly longer useful lifetime than competing products, the TeraScale E-Series also reduced total cost of ownership as well as upgrade cycles.

Patterson continues to expect great things from Force10 in the future. "One of our reasons for choosing them was to stick with leaders. As we continue to expect advances in carrier-class features, a future including 100 Gig and other new developments, we know that Force10 is prepared for the challenge."