By Phil Britt

AFCOM's Communique newsletter / December 2008

By taking a new approach to building data centers, users could save themselves money and deliver better efficiency to end users, according to computer scientists at UC San Diego's Jacobs School of Engineering.

While data center managers say these new design suggestions have some merit, they also point out that there may be some issues that would keep the ideas from working as effectively as researchers expect. Additionally, the ability to follow the suggested design methods depend largely on company executives, some of whom are entrenched in staying with know construction and design methods even if some newer methods can provide some benefits.

Researchers envision creating a data center that will have scalable interconnection bandwidth, making it possible for an arbitrary host in the data center to communicate with any other host in the network at the full bandwidth of its local network interface. Their approach requires no modifications to the end-host network interface, operating system or applications, and is fully backward compatible with Ethernet, IP and TCP. Ideally, the data center would also use inexpensive, off- the-shelf Ethernet switches as the basis for large-scale data center networks, thereby replacing high-end switches in much the way that commodity personal computers have displaced supercomputers for high-end computing environments.

By connecting in such a manner, the equipment has faster throughput, says Amin Vahdat, a professor of computer science and engineering in UC San Diego's Jacobs School of Engineering and director of UCSD's Center for Networked Systems. He says that such a design becomes more critical as companies develop larger and larger data centers with tens of thousands or more computers and as 10G switches start to become the norm. Vahdat explains that this design sharply reduces communications overhead because the number of messages would be significantly reduced. Using current architectures, the number of failure/non-failure messages grows by the square of the number of nodes.

Data center computers and traditionally connected via a network architecture including a tree of routing and switching elements regulated by specialized equipment with expensive, non-commodity switches at the top of the hierarchy. However, these networks can only support a small fraction of the combined bandwidth available to end hosts, limiting overall cluster size. Application design is further complicated by non-uniform bandwidth among data center nodes, which limits overall system performance.

However, Vahdat admits that much of the benefit of such a design depends on proper design. I f data centers use an excessive amount of point-to-point cabling, they could be left with a random spaghetti mess of wires making it difficult to get the most efficient connections.

"it's a really nice theory, and it is something we need to do, but I don't see us needing to do it with that type of horsepower at this point," says Richard Penn, global business head of transformation consulting for Wipro Technologies, an IT outsourcing firm based in. Bangalore. Ind. "The technology is very solid, but I don't know why you would do it to that kind of magnitude at this time."

Penn likens the idea to the movement from mainframes to client server environment for computing several years ago. While he does not see the need today or the size of application that Vahdat discusses, larger data center solutions could be needed in the future, Penn adds.

A critical component of enhancing data center efficiency is replacing copper connections with fiber optics, which require much less power. By using fiber optic connections rather than copper, a company can save itself as much as 10 watts per port, says Ken Sanders, general manager for DataSite, a 130,000-square-foot data facility in Orlando. Multiply that by hundreds or thousands of connections and the rising cost of energy, and the savings can be significant. A company with as few as 48 connections can achieve a positive return on investment in as little as 360 days, according to Sanders.

Lior Blik, acting CIO of Hoboken University Medical Center and president and CEO of Network Infrastructure Technologies, an IT solutions services provider, questions Vahdat's ideas. "My concern is the security you 're limiting your ability to segregate the network at a single point." By segregating the network at a single point, it's easier to isolate a security issue, Blik says.

Blik also wonders about the wisdom of using off-the-shelf Ethernet switches because they may not be compatible with one another; quality of service could drop. He counters that it is better to stay with the high-end switches from a single manufacturer. Vahdat says that using the design he proposes could save companies $6 out of every $7 spent on switching equipment. Despite these concerns, Bilk does say Vahdat's suggestions could work in some smaller data centers.

The next step in moving the idea forward, according to Vahdat, is the ongoing discussions with switch and router designers to incorporate these ideas into the next versions of their products. Vahdat suggests that the optimum time for managers of existing data centers to consider such a changes in when they replace connections equipment, which tends to have a three-to five-year life-cycle. There are a number of new data centers under construction that could adopt this design much more readily, Vahdat says.

Phil Britt is a freelancer writer based in South Holland, III.