On May 7th, QTS hosted a NAP Summit at their Richmond data center and Ocean Specialists Inc (OSI) was there to understand how data center growth will interact with and drive subsea network development.
Located half-way between Virginia Beach and Ashburn, Richmond has recently become an area with significant data center development driven, at least in part, by its proximity to submarine cable connectivity. The NAP summit provided an opportunity for QTS, Facebook and others to interact with both their customers and the wider community. Organized as a one-day conference, the NAP summit provided a fascinating mix of regional politics, economic development, data center infrastructure, network architecture, and global strategy.
First a bit of background: the QTS Datacenter was originally built as a memory chip plant in 1994 in a joint venture between Motorola and Siemens. After changing hands several times, the one million-square-foot fab was shut down in 2009. QTS acquired the $1.2B facility for $12M – one cent on the dollar – and set about refurbishing it as a data center. The site is served by redundant 220KV lines and a 25,000-ton cooling plant – both able to support about 75 MVA of data center load. In 2017, Facebook selected an adjacent property as a location for one of its hyperscale data centers. Together, the two facilities form the nucleus of a cluster that hopes to rival Ashburn, Virginia, an area well known for its abundance of data centers.
The first sessions focused on economic developments and what it takes to support data center infrastructure. Among the key points:
- Data centers have big economic benefits; an 8:1 ratio of taxes paid to public services consumed was quoted.
- Power consumption of all data centers in Virginia totals about 1.2 GW of which 200 MW is outside of Richmond.
- No one wants to live near a power plant, so they are mostly in far off, rural areas. Security and diversity in the power transmission grid are essential for large-scale data centers.
- Henrico County has actively courted data center providers and has adopted tax policies that support the full range of data center businesses without favoring any single company.
The highlight of the summit was a panel on subsea cables. Najam Ahmad, VP Network Engineering for Facebook, Frank Rey, Director Global Network Strategy for Microsoft and Fafael Arranz of Telxius Cable were the panelists. Some of the takeaways from that session:
- The Virginia Beach landing was selected for a number of reasons: Superstorm Sandy made us aware of the vulnerability of NJ landings stations and the need for diversity. After eliminating NY, NJ, the Carolinas (due to environmental concerns) and Florida (due to hurricanes) Virginia was what remained.
- Adding compute capacity drives everything else. Connecting computers, users, data centers and peering is the big problem to be solved. There needs to be sufficient diversity so that no single site is too big to fail. At the same time, there needs to be a balance between subsea, terrestrial and data centers.
- Building out data center compute capacity takes 3-5 years; submarine cable builds easily fit into that timeframe.
- For subsea, the problem to be solved is to find the balance between fiber pair capacity, number of fiber pairs, number of cables and whether there’s enough traffic demand from that region to justify the investment
- Machine-to-machine traffic is over six times greater than machine-to-user traffic.
The panelists also offered a few bold predictions for the future:
- Best-effort internet traffic will gradually give way as five-nines push ever closer to the end-user.
- 5G will dramatically drive capacity demand.
- We will continue to struggle to find the right balance between data center space and power, submarine capacity, terrestrial capacity and interconnect.
The third panel featured a discussion of peering and interconnect; this topic may be less familiar, but quite a few interesting points were raised:
- Enterprises are now connecting directly to peering points to improve connectivity with cloud providers.
- Data centers have space and power but no eyeballs.
- Interconnect is needed to reach eyeballs.
- Subsea cables are for machine-machine traffic.
- We need to move the data center closer to the end-user.
- We don’t use submarine cables to serve end-users, except as a fallback.
- Video is the easiest to serve, very predictable and can be cached.
- Streaming is the most difficult; one can’t predict where it will come from or go to.
- If a million streams are dropped due to a network fault, would we even realize something happened?
- Streams travel an average of 700 miles from server to eyeball; there is a big need to put content closer to the end-user, lots of optimization still to be done.
- Direct connections to cloud providers help with network security.
- 5G will change things, where will radio access network (RAN) to IP handoff occur?
The afternoon concluded with an enterprise panel – the chief concern there is finding qualified personnel to support cloud services development – and a facility tour.
In conclusion, the QTS NAP Summit was a useful and engaging day. Many thanks to QTS for hosting and to all the panelists for their insight into both the subsea cable and data center businesses.
