With demand for network, storage and computing capacity on the rise, South African end-user organisations and service providers are investing heavily in revamping their existing datacentres, as well as building new facilities.
As they do so, they’re starting to put greener, more energy-efficient cooling and power technologies in place and replacing what is left of their legacy hardware and software with commoditised products such as Intel servers.
Even though falling hardware costs, automation solutions and technologies such as virtualisation are helping to ease datacentre management, operating managers and CIOs are feeling the squeeze from tight budgets and rapidly rising energy costs.
“We are caught between a rack and a hard place,” says Craig MacPherson, senior operations manager at Internet Solutions (IS). “We are expected to do more than before with less resources than ever.”
Datacentre technology is shrinking, becoming more efficient and more affordable as a result of commoditisation, says Eckart Zollner, business development manager at New Telco SA, a company that provides colocation services to multiple telecom operators and internet service providers so that they can all interconnect at a single facility.
But these trends are offset somewhat by the constant growth in the bandwidth, processing and storage requirements of the average company. What’s more, the shift towards online business means that most companies, especially service providers, must invest heavily in creating highly available infrastructure.
As they revamp their datacentres, many South African organisations are building them to The Uptime Institute’s Tier 3 or even Tier 4 standards for redundancy and resilience. To achieve Tier 3 certification, companies need an infrastructure that offers 99.982 percent availability.
The datacentre site needs to have full redundancy of its core capacity components such as processors, dual-powered equipment, redundant generators and uninterruptible power supplies (UPSes), and multiple telecoms uplinks.
This year, New Telco SA, a joint venture between JSE-listed Jasco and Germany’s New Telco, completed the construction of a Tier 3 datacentre with 250m² of floor space. This includes ensuring the site has autonomy from the Eskom grid in the event of a power failure and installing redundant telecoms connections from multiple operators.
Downtime experienced by South African cellular operators in recent months and the interruption of the international BlackBerry service have highlighted the importance of uptime to South African companies, says Zollner. Few companies can afford the loss of revenues and the corporate embarrassment that accompany a prolonged systems outage.
Where it all happensFor the JSE, the datacentre is such a critical element of its business that it has decided to retain in-house control over its infrastructure in a market where many organisations are outsourcing. “In our industry, the datacentre is critical to our daily operations,” says JSE CIO Riaan van Wamelen.
The JSE, like other stock exchanges around the world, is making heavy investments in its datacentre. It is investing significantly in the renewal of its applications as well as in upgrading its datacentre infrastructure to bring it in line with Tier 3 standards. During the course of 2012, the JSE aims to move its equity trading system from the London Stock Exchange to Johannesburg. In addition, it plans to replace back-office systems such as internal surveillance systems, clearing systems and accounting applications.
The relocation of its trading engine will allow the JSE to improve the stability of its trading environment by removing the risks of downtime caused by instability of international connectivity. In addition, the JSE is planning to offer colocation services in future.
The benefit for JSE members and their clients is that they will be better able to use algorithmic trading strategies if they locate their servers near the bourse’s matching engine. This is a high-pressure environment where every millisecond counts. The key word is latency. Like stock exchanges around the world, the JSE is looking to reduce latency.
Says Van Wamelen: “One of the key considerations in the day-to-day running of a datacentre lies not only in having the required infrastructure environment, but also in running and maintaining the datacentre operations in line with best practice. The skills required to manage the operations appropriately are scarce on the ground and aren’t cheap.”
On the infrastructure side, the major challenge lies in managing infrastructure and server sprawl, he adds. The JSE is using server virtualisation technologies to manage its server environment in an efficient manner. Sophisticated monitoring tools are being implemented as part of the datacentre upgrade.
Power struggleNew Telco SA, too, has invested in automation technologies to keep its overheads down. Environmental monitoring and remote management tools at the New Telco SA datacentre alert technicians when there is a power failure, a sudden rise in the temperature of the datacentre or other issues that need to be addressed. The network is monitored online from the operations centre in Frankfurt, Germany. “We automate as much as possible to keep staff levels down and pricing competitive,” says Zollner.
Electricity, which used to be cheap and reliable in South Africa, is a major problem for datacentre managers who are trying to keep costs down. Zollner says power costs in South Africa are still reasonably low compared to many other countries, but have crept to nearly 20 percent of the average datacentre’s running costs. New Telco SA is investigating prospects in other parts of Africa and has found that in some countries, power costs could exceed 35 percent of operating expenditure.
The fact that South Africa has ambient data temperatures slightly higher than most European countries and buildings that are not as well insulated contributes to the power bill because the cooling requirements are a little higher, Zollner adds. The cost of power is a major driver for green datacentres across the world.
New Telco in Germany is investing in green datacentre technologies and is looking at building datacentres based on renewable energy sources such as fuel-cell technology. The local joint venture may adopt these technologies once they have proven themselves.
South African companies and regulators are still dealing with the carbon footprint of power-hungry datacentres in a reactive manner, says Zollner. Organisations in Europe are being more proactive because subsidies and regulations give them more incentive to invest in green technologies.
IS is one South African company that is taking the greening of its datacentre seriously. Last year, the Dimension Data division began the construction of a new Tier 3 datacentre in Johannesburg to cater for the growth of its business and to improve efficiencies in its datacentre environment.
Completion of the first phase of the facility was scheduled for mid-November at the time of writing. The new facility will eventually accommodate more than 700 cabinets, with more than 200 to be provisioned in the first phase.
Green datacentresTechnology has advanced a great deal in the 18 years or so that have passed since IS was established, says MacPherson. IS realised that it might not be cost-effective to revamp its existing datacentres with the new infrastructure technologies that are available today.
Up until 15 years ago, servers were configured in towers rather than racks, blanking panels were not yet in wide use, and UPS technologies were not as efficient as the ones available today. With a clean slate, IS has been able to put the latest cooling and management technologies in place in the new datacentre.
IS has designed the new datacentre around cooling technology from KyotoCooling, which it expects to provide a 93 percent saving on cooling costs and a 40 percent saving on the power bill. The KyotoCooling technology should allow IS to achieve an average annual mechanical power usage effectiveness (PUE) of below 1.2. Older cooling systems generate a PUE of around 1.8 or higher.
The cost savings mostly come from the KyotoCooling concept, which delivers ‘free cooling’ for 75 percent of the year. The concept of free cooling is to use outside air to remove heat from the datacentre whenever possible rather than expensive, power-hungry refrigeration. The Kyoto heat wheel separates the outside air from the inside air to regulate temperature in the datacentre.
Building a new datacentre or expanding an existing one isn’t just about the technology, but also about its physical location and construction. MacPherson says one of the biggest factors in setting up a new datacentre is deciding where it should be located to strike a balance between cost and convenience.
A datacentre needs to be on an urban node with a reliable electricity supply and good telecoms links from multiple suppliers, and it should preferably have line of sight to high spots for wireless transmitters. In addition, it needs to be in a part of town that employees and customers will feel safe driving to in the dead of night.
There’s also a lot of municipal and administrative red tape, which can in some cases take as much time as building the infrastructure, says Zollner. Often, it can take months for a company to secure city council permission to trench the road to lay down some fibre. Whereas European companies will use machines to do the digging, South Africa still relies on manual labour to do the work, Zollner notes.
The upgrade treadmillZollner says the rapid pace of technology change is perhaps the biggest challenge companies face in their datacentres. Ideally, companies should be revamping their datacentres every three years to keep up with the advance of technology, but most datacentres are filled with technology that is seven or more years old.
Many end-user organisations that invested heavily in their own datacentre infrastructure a few years back are now looking to defray some of the costs by commercialising them and selling off some of their spare capacity, says Zollner. Others are moving towards outsourcing to get off the upgrade treadmill while still having access to the latest technology.
The JSE is one organisation that is frequently refreshing its technology. Historically, it has replaced its equities trading engine around every five years, for example. As a result, it has little legacy technology in its datacentre because it has introduced new technologies such as virtualisation, blade servers and storage area networks over the past few years.
Where possible, it has used industry-standard technologies to power its applications. But Van Wamelen admits the costs of staying up to date with datacentre technology are high. With the upgrade of its datacentre to cater for colocation services, the JSE is hoping to create a new revenue stream.
The commoditisation of computer hardware is rapidly changing the face of the datacentre, with the footprint of the technology shrinking all the time and vendors finding innovative ways to rapidly deploy datacentre infrastructure.
Datacentres are becoming industrialised to provide more flexibility to end-user organisations. The likes of IBM, for example, are offering modular datacentres that feature standardised and repeatable designs so they can be deployed quickly on customer sites.
Telecoms industry veteran Zollner says that when South Africa’s cellular operators were building their datacentres in the early 1990s, the trend was towards building large, centralised infrastructures. Now, the trend is towards building a network of smaller, distributed datacentres to provide a higher level of redundancy.
One concept that is beginning to gain currency in Europe is the ‘containerised’ datacentre, which crams datacentre gear such as server racks and storage into a shipping container for easy portability, says Zollner.
These solutions are seen as perfect for remote sites, for organisations that need mobile infrastructure, and for organisations that want to grow out infrastructure without tearing down and rebuilding their existing datacentres.