Credit: Dreamstime About the only thing harder than building a data centre is dismantling one, because the potential for disruption of business is much greater when shutting down a data centre than constructing one.
The recent decommissioning of the Titan supercomputer at the Oak Ridge National Laboratory (ORNL) in America reveals just how complicated the process can be.
More than 40 people were involved with the project, including staff from ORNL, supercomputer manufacturer Cray, and external subcontractors. Electricians were required to safely shut down the nine megawatt-capacity system, and Cray staff was on hand to disassemble and recycle Titan’s electronics and its metal components and cabinets.
A separate crew handled the cooling system. In the end, 350 tons of equipment and 10,800 pounds of refrigerant were removed from the site.
While most enterprise IT pros aren’t likely to face decommissioning a computer the size of Titan, it is likely they’ll be involved with dismantling smaller-scale data centres given the trend for companies to move away from on-premises data centres.
The pace of data centre closure is going to accelerate over next three or four years, according to Rick Villars, research vice president, data centre and cloud, at IDC.
“Every company we’ve spoken to is planning to close 10 per cent to 50 per cent of their data centres over the next four years, and in some cases even 100 per cent. No matter who you talk to, they absolutely have on the agenda they want to close data centres,” Villars says.
Successfully retiring a data centre requires navigating many steps. Here’s how to get started.
Inventory data-centre assets
The first step is a complete inventory. However, given the preponderance of zombie servers in IT environments, it’s clear that a good number of IT departments don’t have a handle on data centre asset management.
“They need to know what they have. That’s the most basic. What equipment do you have? What apps live on what device? And what data lives on each device?” says Ralph Schwarzbach, who worked as a security and decommissioning expert with Verisign and Symantec before retiring.
All that information should be in a configuration management database (CMDB), which serves as a repository for configuration data pertaining to physical and virtual IT assets.
A CMDB “is a popular tool, but having the tool and processes in place to maintain data accuracy are two distinct things,” Schwarzbach says.
A CMDB is a necessity for asset inventory, but “any good CMDB is only as good as the data you put in it,” says Al DeRose, a senior IT director responsible for infrastructure design, implementation and management at a large media firm.
“If your asset management department is very good at entering data, your CMDB is great. [In] my experience, smaller companies will do a better job of assets. Larger companies, because of the breadth of their space, aren’t so good at knowing what their assets are, but they are getting better.”
Map dependences among data centre resources
Preparation also includes mapping out dependencies in the data centre. The older a data centre is, the more dependencies you are likely to find.
It’s important to segment what’s in the data centre so that you can move things in orderly phases and limit the risk of something going wrong, says Andrew Wertkin, chief strategy officer with BlueCat Networks, a networking connectivity provider that helps companies migrate to the cloud.
“Ask how can I break this into phases that are independent – meaning ‘I can’t move that app front-end because it depends on this database,’” Wertkin says.
The WAN is a good example. Connection points are often optimised, so when you start to disassemble it, you need to know who is getting what in terms of connections and optimised services so you don’t create SLA issues when you break the connection.
Changing the IP addresses of well-known servers, even temporarily, also creates connection problems. The solution is to do it in steps, not all at once.
Questions to ask decomissioning providers
Given the complexities and manpower needs of decommissioning a data center, it’s important to hire a professional who specialises in it.
Experience and track record are everything when it comes to selecting a vendor, says Mike Satter, vice president at OceanOceanTechech, which provides data centre decommissioning and IT asset disposition services.
There are a lot of small companies that say they can decommission a data centre and fail because they lack experience and credentials, he says. “I can’t tell you how many times we’ve come into a mess where we had to clean up what someone else did. There were servers all over the floor, hardware everywhere,” Satter says.
His advice? Ask a lot of questions.
“I love having a client who asks a lot of questions,” Satter says. “Don’t be shy to ask for references,” he adds. “If you are going to have someone do work on your house, you look up their references. You better know who the contractor will be. Maybe 10 per cent of the time have I had people actually look into their contractor.”
Among the processes you should ask about and conditions you should expect are:
- Have the vendor provide you with a detailed statement of work laying out how they will handle every aspect of the data centre decommissioning project
- Ask the vendor to do a walkthrough with you, prior to the project, showing how they will execute each step
- Find out if the vendor outsources any aspect of data centre decommissioning, including labor or data destruction
- Inquire about responsible recycling (see more below)
- Ask for references for the last three data centre decommissioning clients the vendor serviced
- Ask if the vendor will be able to recover value from your retired IT hardware. If so, find out how much and when you could expect to receive the compensation
- Ask how data destruction will be handled. If the solution is software-based, find out the name of the software
- Learn about the vendor’s security protocols around data destruction
- Find out where the truck goes when it leaves with the gear
- Ask how hazardous materials will be disposed
- Ask how metals and other components will be disposed
Recycle electronics responsibly
As gear is cleared out of the data centre, it’s important to make sure it’s disposed of safely, from both a security and environmental standpoint.
When it comes to electronics recycling, the key certification to look for is the R2 Standard, Satter says. R2 – sometimes referred to as the responsible recycling certification – is a standard for electronics recyclers that requires certified companies to have a policy on managing used and end-of-life electronics equipment, components and materials for reuse, recovery and/or recycling.
But R2 does more than that; it offers a traceable chain of custody for all equipment, tracking who touched every piece and its ultimate fate. R2 certified providers “aren’t outsourced Craigslist tech people. These are people who do it every day,” Satter says.
“There are techniques to remove that gear. They have a group to do data security on site, and a compliance project manager to make sure compliance is met and the chain of custody is met.”
And don’t be cheap, DeRose adds. “When I decommission a data centre, I use a well-known company that does asset removal, asset destruction, chain of custody, provides certifications of destruction for hard drives, and proper disposal of toxic materials,” DeRose says.
“All that needs to be very well documented not [only] for the environment’s protection but [also] for the company’s protection. You can’t wake up one morning and find your equipment was found dumped in a landfill or in a rainforest.”
Documentation is critical when disposing of electronic waste, echoes Schwarzbach. “The process must capture and store info related to devices being decommissioned: What is the intent for the device, recycling or new service life? What data resides on it? Who owns the data? And [what is] the category of data?”
In the end, it isn’t the liability of the disposal company if servers containing customer or medical information turn up at a used computer fair, it’s the fault of the owners. “The creator of e-waste is ultimately liable for the e-waste,” Schwarzbach says.
Control who’s coming into the data centre
Shutting down a data centre means one inevitability: You will have to bring in outside consultants to do the bulk of the work, as the ORNL example shows.
Chances are, your typical data centre doesn’t let anywhere near 40 people inside during normal operations. But during decommissioning, you will have a lot of people going in and out, and this is not a step to be taken lightly.
“In a normal scenario, the number of people allowed in the data centre is selected,” Wertkin says. “Now, all of a sudden, you got a bunch of contractors coming in to pack and ship, and maybe there’s another 50 people with access to your data centre.
“It’s a process and security nightmare if all these people have access to your boxes and requires a whole other level of vetting.” His solution: Log people in and out and use video cameras.
Any company hired to do a decommissioning project needs to clearly identify the people involved, DeRose says. “You need to know who your company is sending, and they need to show ID.” People are to be escorted in and out and never given a keycard.
In addition, contractors should not to be left to decommission any room on their own. There should always be someone on staff overseeing the process, DeRose says.
In short, the decommissioning process means lots of outside, non-staff being given access to your most sensitive systems, so vigilance is mandatory.
None of the steps involved in a data centre decommissioning should be hands-off, even when it requires outside experts. For the security and integrity of your data, the IT staff must be fully involved at all times, even if it is just to watch others do their work.
When millions of dollars (even depreciated) of server gear goes out the door in the hands of non-employees, your involvement is paramount.
Tags Data Centre
More about BlueCat NetworksCrayOak Ridge National LaboratorySymantec
read more at https://www.cio.com.au/tax/news/ by Andy Patrizio
Cio
