Sunday 22 August 2010
- Site location
- Natural disaster risks
- The site location should be where the risk of natural disasters are acceptable. Natural disasters include but are not limited to forest fires, lightning storms, tornadoes, hurricanes, earthquakes and floods.
- Man-made disaster risks
- The site location should be located in an area where the possibility of man-made disaster is low. Man-made disasters include but are not limited to plane crashes, riots, explosions, and fires. The site should not be adjacent to airports, prisons, freeways, stadiums, banks, refineries, pipelines, tank farms, and strike routes.
- The electrical utility powering the site should have a 99.9% or better reliability of service. Electricity must be received from two separate substations (or more) preferably attached to two separate power plants. Water should be available from more than one source. Using well water as a contingency should be an option. There must be connectivity to more than one access provider at the site.
- Sole purpose
- A data center should not share the same building with other offices, especially offices not owned by the organization. If space must be shared due to cost then the data center should not have walls adjacent to other offices
- Site Perimeter
There should be a fence around the facility at least 6 metres from the building on all sides. There should be a guard kiosk at each perimeter access point. There should be an automatic authentication method for data center employees (such as a badge reader reachable from a car). The area surrounding the facility must be well lit and should be free of obstructions that would block surveillance via video cameras and security patrols. Where possible, parking spaces should be a minimum of 8 metres from the building to minimize damage from car bombs. There should not be a sign advertising that the building is in fact a data center or what company owns it.
There should be surveillance cameras outside the building monitoring parking lots and neighboring property. There should be security guards patrolling the perimeter of the property. Vehicles belonging to data center employees, contractors, guards, and cleaning crew should have parking permits. Service engineers and visitor vehicles should be parked in visitor parking areas. Vehicles not fitting either of these classifications should be towed.
- Outside Windows and Computer Room Placement
The site location must not have windows to the outside placed in computer rooms. Such windows could provide access to confidential information and insight into the locations layout and processes. The windows also cast sunlight on servers unnecessarily introducing heat to the computer rooms. Computer rooms should be within the interior of the data center. If a computer room must have a wall along an outside edge of a data centre there should be a physical barrier preventing close access to that wall.
- Access Points
Loading docks and all doors on the outside of the building should have some automatic authentication method (such as a badge reader). Each entrance should have a mantrap (except for the loading dock), a security kiosk, physical barriers (concrete barricades), and surveillance cameras to ensure each person entering the facility is identified. Engineers and cleaning staff requiring badges to enter the building must be required to produce picture ID in exchange for the badge allowing access. A log of equipment being placed in and removed from the facility must be kept at each guard desk listing what equipment was removed, when and by whom. Security Kiosks should have access to read the badge database. The badge database should have pictures of each user and their corresponding badge. Badges must be picture IDs.
- Computer Rooms
There should be signs at the door(s) marking the room as restricted access and prohibiting food, drink, and smoking in the computer room. There should be an automatic authentication method at the entrance to the room (such as a badge reader). Doors should be fireproof. There should only be two doors to each computer room (one door without windows is probably a violation of fire code). Access should be restricted to those who need to maintain the servers or infrastructure of the room. Access should be restricted to emergency access only during moratoriums for holidays. Service Engineers must further go to the Network Operations Centre to obtain access to computer room badges.
Computer Rooms should be monitored by surveillance cameras. Each computer room should have redundant access to power, cooling, and networks. There should be at least an 45cm access floor to provide for air flow and cable management. Computer rooms should have air filtration. Computer rooms should have high ceilings to allow for heat dispersal.
Each computer room should have temperature between 13 and 24 degrees Celsius and a humidity of between 20 and 80 percent. Environmental sensors should log the temperature and humidity of the room and report it to the Network Operations Centre for monitoring and trend analysis.
- Fire Prevention
There should be Inergen or Eckoshielda fire suppressant solutions in place in each computer room. There must be fire extinguishers located in each computer room. There must be emergency power off switches inside each computer room. There may be respirators in computer rooms. There must not be wet pipe sprinkler systems installed.
- Shared Space
If the space is being leased then the computer room will probably be shared space. A clause should be entered into the lease stating that competitors of the business may not have equipment located in the same computer room. Lists of clients utilizing the same room should be monitored to ensure compliance. Computer equipment in shared spaces must at a minimum be in a locked cabinet.
- Cooling Towers
There must be redundant cooling towers. Cooling towers must be isolated from the Data Centre parking lot.
There must at least be battery backup power on site with sufficient duration to switch over to diesel power generation. If there is no diesel backup then there should be 24 hours of battery power. There should be diesel generators on site with 24 hours of fuel also on site. A contract should be in place to get up to a week of fuel to the facility.
All papers containing sensitive information should be shredded on site or sent to a document destruction company before being discarded. Dumpsters and bins should be monitored by surveillance cameras.
- Network Operations Centre
The Network Operations Centre must have fire, power, weather, temperature, and humidity monitoring systems in place. The Network Operations Centre must have redundant methods of communication with the outside. The Network Operations Centre must be manned 24 hours a day. The Network Operations Centre may monitor news television stations and websites for events which effect the health of the data center.
- Cooling Towers
- Disaster Recover
- Disaster Recovery Plan
The data centre must have a disaster recovery plan. Ensure that the plan addresses the following questions: What constitutes a disaster? Who gets notified regarding a disaster and how? Who conducts damage assessment and decides what back-up resources are utilized? Where are backup sites located and what is done to maintain them on what schedule? How often and under what conditions is the plan updated? If the organization does not own the data center what downtime does the service level agreement with the center allow? A list of people within the organization to notify must be maintained by the Network Operations Centre of the data center including work, home, and mobile numbers. (Including Skype and other Instant Message Names if available) How often are those people updated?
- Off site backup
There must be regular off site backups of essential information. There must be a backup policy in place listing the procedure for restoring from backup and allowing for the scheduling of practice runs to test that the backups work.
- Redundant site
Redundant servers may be set up in another data center. If these are set up then they must be tested during a "dry run" to ensure that they will switch over properly during a disaster.
- Security officers and guards
Security guards should submit to criminal background checks. Security guards should be trained to follow and enforce physical security policy strictly (for example ensuring that everyone in the facility is wearing a badge).
- Cleaning staff
Cleaning crews should work in groups of at least two. Cleaning crew should be restricted to offices and the Network Operations Centre. If cleaning staff must access a computer room for any reason they must be escorted by Network Operations Centre personnel.
- Service engineers
Service engineers must log their entering and leaving the building at the entrance to the building. The Network Operations Centre should log their badge exchange to access a computer room.
Visitors must be escorted by the person whom they are visiting at all times. Visitors must not be allowed access to a computer room without written approval from data center management. All visitors who enter computer rooms must sign a register or log.
Users must be educated to watch out for potential intruders who may shoulder surf or directly attempt social engineering. Users should be educated on securing workstations and laptops within the facility and laptops outside the facility, awareness of surroundings, and emergency procedures.
All users at the facility must sign Non Disclosure Agreements. A Physical Security Policy should be signed by each user and enforced by security guards.
- Disaster Recovery
- Organizational chart
An organizational chart should be maintained detailing job function and responsibility. Ideally the org chart would also have information on which functions the worker has been cross trained to perform.
- Job function documentation
It is important to document what employees currently know about existing systems and hardware. Also all new work requests and changes, must be documented.
- Cross training
Data Centre employees should be cross trained in a number of other job functions. This allows for a higher chance of critical functions being performed in a crisis.
- Contact information
A contact database must be maintained with contact information for all data centre employees.
Data centre employees should regularly practice telecommuting. If the data centre is damaged or the ability to reach the data centre is diminished then work can still be performed remotely.
- Disparate locations
If the organization has multiple data centres then personnel performing duplicate functions should be placed in disparate centres. This allows for job consciousness to remain if personnel at one center are incapacitated.
- Airflow management (mechanical)
The efficiency and effectiveness of a datacenter conditioning system is heavily influenced by the path, temperature and quantity of cooling air delivered to the IT equipment and waste hot air removed from the equipment.
- Eliminate mixing and recirculation of hot equipment exhaust air
- Hot aisle/Cold aisle
- Rigid nnclosures
- Flexible strip curtains
- Blank unused rack positions
- Design for IT airflow configuration
- Select racks with good internal airflow
- Maximize Return Air Temperature by Supplying Air Directly to the Loads
- Use Appropriate Diffusers
- Position Supply and Returns to Minimize Mixing and Short Circuiting
- Minimize Air Leaks in Raised Floor System
- Optimize Location of Computer Room Air Conditioners
- Provide Adequately Sized Return Plenum or Ceiling Height
- Provide Adequately Sized Supply
- Use an Appropriate Pressure in Underfloor Supply Plenums
- Air Handler Systems (mechanical)
The air handler fan is typically the second largest energy use in the mechanical system, and can even exceed the energy use of the cooling plant in some cases. Optimizing the air handler system for datacenter use, as opposed to relying on traditional air handler design rules developed over years of office system design, is essential to achieve an efficient and cost effective system.
- Minimize Fan Power Requirements
- Low pressure drop system design
- Use Redundant Air Handler Capacity in Normal Operation
- Use an Optimized Airside Economizer
- Implement an Airside Economizer
- Design for Medium Temperature Air
- Control to Avoid Unnecessary Humidity Loads
- Use Large Centralized Air Handle
- Use Load Diversity to Minimize Fan Power Use
- Optimize Air Handler for Fan Efficiency and Low Pressure Drop
- Configure Redundancy to Reduce Fan Power Use in Normal Operation
- Use Premium Efficiency Motors and Fans
- Control Volume by Variable Speed Drive on Fans Based on Space Temperature
- Humidification (mechanical)
Humidification specifications and systems have often been found to be excessive and/or wasteful in datacenter facilities. A careful, site specific design approach to these energy-intensive systems is usually needed to avoid energy waste.
- Design System to Actual Equipment Requirements
- Use Widest Suitable Humidity Control Band
- Specify Humidity Sensor Calibration Schedule
- Provide Appropriate Sensor Redundancy
- Control Humidity with Dedicated Outdoor Air Unit
- Eliminate Over Humidification and/or Dehumidification
- Ensure Proper Economizer Lockout
- Maintain coil temperature above 13 degrees Celsius
- Centralize Humidity Control
- Use Efficient Humidification Technology
- Use Waste Return Air Heat to Humidify
- Use Adiabatic Humidifiers for Humidity and Evaporative Cooling
- Use Lower Power Humidification Technology
- Plant Optimization (mechanical)
When a chilled water plant is used, all the standard design best practices apply, with a few additions. The unusual nature of a datacenter load, which is mostly independent of outside air temperature and solar loads, makes free cooling very attractive and increases the importance of efficiency over first cost. Also, the typical level of redundancy and reliability can influence the value of various design options.
- Maximize the chiller system efficiency
- Select Chiller for High Efficiency
- Implement an Aggressive Condenser Water Reset
- Minimize Tower Fan Power and Size Towers for Close Approach
- Use Free Cooling / Waterside Economization
- Use a Medium Temperature Chilled Water Loop
- Use Primary Only Variable Flow Chilled Water Pumping
- Consider Thermal Storage
- Monitor System Efficiency
- Rightsize the Cooling Plant
- IT Equipment: Selection
The IT equipment is the reason for the facility. Increasingly, there are reasonable opportunities to increase the efficiency of IT equipment, reducing the need for mechanical infrastructure and ongoing energy use directly at the load level through the selection of IT equipment.
- Specify Efficient Server Equipment
- Specify High Efficiency Power Supplies
- Consider Equipment Power Consumption in Specifications
- Use Cooled Equipment Racks
- Use Equipment Racks with Integral Coil
- Consider Direct Liquid Cooling
- Specify Efficient Server Equipment
- Electrical Infrastructure
Protection from power loss is a common characteristic of datacenter facilities. Such protection comes at a significant first cost price, and also carries a continuous power usage cost that can be reduced through careful design and selection.
- Design UPS System for Efficiency
- Maximize Unit Loading
- Select Most Efficient UPS Possible
- Specify Minimum Unit Efficiency at Expected Load Points
- Evaluate UPS Technologies for Most Efficient
- Do Not Overspecify Power Conditioning Requirements
- Use Self-Generation for Large Installation
- Eliminate Standby Generator
- Recover Waste Heat for Local Heating Uses
- Recover Waste Heat for Datacenter Cooling Use
- Eliminate UPS Systems
data centres are typically lightly occupied. While lighting is a small portion of the total power usage of a datacenter, it can be often be safely reduced through mature, inexpensive technologies and designs.
- Use Active Sensors to Shutoff Lights When Datacenter is Unoccupied
- Occupancy sensors
- Design Light Circuiting and Switching to Allow for Greater Manual Control
- Bi-Level Lighting
- Task Lighting
- Commissioning and retrocommissioning
An efficient datacenter not only requires a reliable and efficient design, it also requires proper construction and operation of the space. Commissioning is a methodical and thorough process to ensure the systems are installed and operating correctly in all aspects, including efficiency.
- Engage Additional Design Expertise for Review and Guidance.
- Perform a Peer Review
- Engage a Commissioning Agent
- Perform System Commissioning
- Document Testing of All Equipment and Control Sequences
- Measure Equipment Energy Efficiency Onsite
- Provide Appropriate Budget and Scheduling for Commissioning
- Perform Full Operational Testing of All Equipment
- Perform Retrocommissioning
- Perform a Full Retrocommissioning
- Recalibrate All Control Sensors
- Where Appropriate, Install Efficiency Monitoring Equipment