Data Center from Conception to Operation In 30 minutes or less

What is a Data Center? A data center is a facility used for housing electronic equipment, typically computers and communications equipment. Data centers can be private, serving a single company or, public serving several companies. Practically every company that is mid-sized or larger has some kind of data center.
Joe Joplin Data Center from Conception to Operation

What type of Data Center are you designing? The Uptime Institute created a 4 Tier rating system of data centers, Tier-1 to Tier-4. Tier-1 data centers are the most basic while a Tier-4 is fundamentally immune to planned and unplanned downtime. Avoid terms like near Tier-3 data center and Tier-3 plus. These classifications do not exist. The Institute has revised its standards to better explain their ratings. The rating system is an absolute. You are only as good as your weakest link.
Joe Joplin Data Center from Conception to Operation

SOURCE: Uptime Institute warns against tier standard misuse, Matt Stansberry, 27 Apr 2006, SearchDataCenter.com

Tier I - Basic Site

Non-redundant capacity components Single non-redundant path distribution paths
Cabling Power Cooling

Performance Test
Any capacity component failure will impact the computer systems. Any distribution path failure will impact the computer systems.

Planned work will require most or all of the systems to be shut down 99.671% availability
equivalent of 28.8 hours of annual downtime or 4.74 min/day
Joe Joplin Data Center from Conception to Operation

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Tier II - Redundant Capacity Components

Redundant capacity components Single non-redundant distribution paths Performance Test
Any capacity component failure may impact the computer systems. Any distribution path failure will impact the computer systems.

Redundant UPS modules and engine generators are required

Redundant Cooling units, chillers, and pumps.

Planned work will require most or all of the systems to be shut down 99.741% availability
equivalent of 22.7 hours of annual downtime or 3.73 min/day

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Joe Joplin Data Center from Conception to Operation

Tier III - Concurrently Maintainable

Redundant capacity components Multiple distributions paths One active path Performance Test
A capacity component can be removed without impacting the system A distribution path can be removed without impacting the system

Dual power inputs required for all computer hardware Planned work will not require the systems to be shut down
But may elevate the risk of disruption

99.982% availability
equivalent of 1.6 hours of annual downtime or .26 min/day (15.6 sec/day)

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Joe Joplin Data Center from Conception to Operation

Tier IV Fault Tolerant

Redundant capacity components Multiple distributions paths Multiple active paths Performance Test
A single worst-case failure of any capacity system or distribution element will not impact the computer system Any capacity component or distribution path can be removed without impacting the computer system

Dual power inputs required for all computer hardware Planned work will not require the systems to be shut down 99.995% availability
equivalent of .4 hours of annual downtime or .07 min/day (4.3 sec/day)

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Joe Joplin Data Center from Conception to Operation

Performance Standard by Tier Level

Tier Requirement Source System Component Redundancy Distribution Paths Compartmentalization Concurrently Maintainable Fault Tolerance Tier 1 Tier II Tier III System N+1 1 normal and 1 alternate Yes Yes No Tier IV System + System Minimum of N+1 2 simultaneously active Yes Yes Yes
Joe Joplin Data Center from Conception to Operation

System System N 1 No No No N+1 1 No No No

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Tier Level Design

You are only as good as your weakest link To meet a tier level you must pass the performance test Each tier represents an infrastructure topology Tier 1 and Tier II solutions are to support current IT Technology.
First Dedicated spaces for computer equipment Typical components for an improvement environment
UPSs, dedicated cooling systems, engine generators

Tier III and Tier IV solutions are long term solutions to outlast current IT Technology
Tier III introduces concurrent maintenance to every component Tier IV introduces fault tolerance to every component

Joe Joplin Data Center from Conception to Operation

Two Primary Design Concepts Power Cooling

Joe Joplin Data Center from Conception to Operation

Power Calculations
Current Load for Critical Equipment
Servers, routers, computers, storage devices, telecommunications equipment, etc.

Future Load for Critical Equipment Load for UPS & Batteries Load for non-critical Equipment
Lighting, Security, Fire, Monitoring, etc.

Load for Cooling Demands

Joe Joplin Data Center from Conception to Operation

Current Load for Critical Equipment

TYPICAL RACK Component Switch Router Blade Server 7U 4836 W TOTAL 200 W .67 x 120V x 2A Nameplate Rating 67% (Volts x Amps) Load (kW) 0.20 0.16 4.84 5.20

Current Load = 6 Racks x 5.2 kW = 31.2 kW

Joe Joplin Data Center from Conception to Operation

Future Load for Critical Equipment Estimated that equipment will be increased by 33% in the next 3 years Future Load = Current Load x 33% Future Load = 31.2 kW x 33% = 10.3 kW

Joe Joplin Data Center from Conception to Operation

Load for UPS & Batteries

UPS
Support Critical Loads from power outages & anomalies Effectively pass through 85% of the input power UPS Load = 15 % x (Current Load + Future Load) UPS Load = .15 x (31.2 kW + 10.3 kW) = 6.23 kW

Batteries
Charging load to be 20% after a full or partial discharge Battery load = 20% x (Current Load + Future Load) Battery load = .20 x (31.2 kW + 10.3 kW) = 8.30 kW

Joe Joplin Data Center from Conception to Operation

Loads for non-critical Equipment Lighting based on average consumption

2 watts per square foot 100 ft2 x 2 W/ft2 / 1000 = .20 kW

Security, Fire, and Monitoring Systems

Total of the nameplate ratings (600 W + 250 W + 250 W) / 1000 = 1.1 kW

Total non-critical Load = .20 kW + 1.1 kW = 1.3 kW

Joe Joplin Data Center from Conception to Operation

Peak Power Draw Peak power draw

(Current Load + Future Load) x Power Factor (31.2 kW + 10.3 kW) x 1.05 = 43.58 kW

Additional power needed

Peak power draw Steady State Load 43.58 (31.2 kW + 10.3kW) = 2.08 kW

Joe Joplin Data Center from Conception to Operation

Total Load Calculations

Current Load = 31.20 kW Future Load = 10.50 kW UPS Load = 6.23 kW Batteries Load = 8.30 kW Non-critical load = 1.30 kW Peak Power Adj. = 2.08 kW TOTAL POWER = 59.40 kW
Joe Joplin Data Center from Conception to Operation

Cooling Power Load

Cooling systems vary in their efficiencies
Direct Expansion systems (DX) are less efficient
operating at a 1:1 ratio to peak load supported

Chilled Water systems are more efficient

operating at a 7:10 ratio to peak load supported

Assuming the worst case scenario of 1:1 Cooling requirement will be 100% of the Total Load
1 x 59 kW = 59 kW

Joe Joplin Data Center from Conception to Operation

Power Usage by Category

Critical Future UPS Batteries Lighting Security Peak Adj Cooling

Joe Joplin Data Center from Conception to Operation

Total Power Requirements

Total Power = Total Load + Cooling Load
Total Power = 59.4 kW + 59.4 kW 120 kW

Total Power Required

NEC Code requires a 125% of the Total Power 1.25 x 120 kW = 150 kW

Electric Service needed

480V Three-Phase Service Current = (kW x 1000) / (Volts x 1.73) Current = (150 kW x 1000) / (480 V x 1.73) 180 A

Joe Joplin Data Center from Conception to Operation

Calculating Generator Size

Category Critical Load Future Load UPS Batteries Lighting Peak Adjustment Cooling Load 31.20 10.30 6.23 8.30 0.20 2.08 59.40 Factor 1.3 1.3 1.3 1.3 1.3 1.3 1.5 Generator Load 40.57 13.39 8.09 10.79 0.26 2.70 89.10 164.89
Joe Joplin Data Center from Conception to Operation

TOTAL Generator Load

Calculating Generator Size Calculated Size 165 kW Generators are typically classified by kVAs
The type power factor for these ratings is .8
Determine the power factor for your generator

Generator Size = 165kW / .8 200 kVA

Joe Joplin Data Center from Conception to Operation

Electrical System Tier 1

AC 208V/480V Gen N

Utility Switch Gear

Mech Switch Gear UPS N Mech System

UPS Output Switch Gear

PDU

Equipment

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Joe Joplin Data Center from Conception to Operation

Electrical System Tier 2

AC 208V/480V Gen N Gen +1

Utility Switch Gear

Gen Switch Gear

Mech Switch Gear UPS N UPS +1 Mech System

UPS Output Switch Gear

PDU

Equipment

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Joe Joplin Data Center from Conception to Operation

Electrical System Tier 3

AC 208V/480V Gen N Gen +1 AC 208V/480V

Utility Switch Gear

Gen Switch Gear

Utility Switch Gear

Mech Switch Gear UPS N UPS +1 Mech System

Mech Switch Gear

UPS Output Switch Gear

Critical MCC Critical Fan or Pump

Critical MCC

ALT Output Switch Gear

PDU

Equipment

PDU

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Joe Joplin Data Center from Conception to Operation

Electrical System Tier 4

AC 208V/480V Gen N Gen +1 Gen N Gen +1 AC 208V/480V

Utility Switch Gear

Gen Switch Gear

Gen Switch Gear

Utility Switch Gear

Mech Switch Gear UPS N UPS +1 Mech System

Mech Switch Gear UPS N UPS +1

UPS Output Switch Gear

Critical MCC Critical Fan or Pump

Critical MCC

ALT Output Switch Gear

PDU

Equipment

PDU

SOURCE: Industry Standard Tier Classification, Uptime Institute, www.upsite.com/TUIpages/tuiwhite.html

Joe Joplin Data Center from Conception to Operation

Cooling Key Components

Physical Layout
Hot Aisle Cold Aisle

Air flow control

Substantial air flow

Consistent Maintenance
Cooling systems rarely provide diagnostics for proactive warnings

Joe Joplin Data Center from Conception to Operation

Physical Layout
Hot Aisle Cold Aisle
Industry Standard for all equipment
Draw in cool air from the front and expel hot from the rear

Align your cabinets with the floor tiles

Keep at least 4 open (2 tiles) on your Cold Aisle Keep at least 3 open (1 tile) on your Hot Aisle

Create a Floor Master plan Put heavy/hotter equipment on the bottom of your racks Distribute power consumption (heat) evenly among your racks Cabling should be routed between racks or in the hot aisles
Avoid air damns on the front of your equipment

Place Cooling Units at the end of the Hot Aisles

Joe Joplin Data Center from Conception to Operation

Air flow control

FRONT VIEW

MIXING AIR

REAR

SIDE VIEW

FRONT

Joe Joplin Data Center from Conception to Operation

Air flow control

FRONT VIEW
FRONT
REAR REAR REAR

SIDE VIEW

FRONT

FRONT

Joe Joplin Data Center from Conception to Operation

Air Flow Control

Raised Floor
4'-0"

RACK

RACK

12'-0"
PERFORATED TILES DATACOM CABLES POWER CABLES DATACOM CABLES

Joe Joplin Data Center from Conception to Operation

Air Flow Control

Overhead Cabling
POWER CABLES

DATACOM CABLES

DATACOM CABLES

RACK

RACK

4'-0"

12'-0"

Joe Joplin Data Center from Conception to Operation

Air Flow Control

Overhead Cabling

SOURCE: Liebert, XDO Overhead Cooling Module, http://www.liebert.com/assets/products/english/products/env/xtreme/60Hz/bro_2pg/acrobat/sl_16660.pdf

Joe Joplin Data Center from Conception to Operation

Air Flow Control

Overhead Cabling

SOURCE: PANDUIT FIBERRUNNERTM and OPTICOMTM HD Cable Management Rack System

Joe Joplin Data Center from Conception to Operation

Air Flow Control

The Uptime institute found that 78% of the total bypass airflow is caused by unsealed cabling holes in the raise floor
Which Allows cold air to circulate directly into the hot aisle

Maximum perforated tiles

Equaled to the total cooling unit airflow divided by 750

Total CFM must equal or exceed your equipment total

Joe Joplin Data Center from Conception to Operation

Consistent Maintenance
In a recent study by the Uptime Institute of 19 Data Centers
10% of the cooling units had failed, be were not providing an alarm

Monitor power usage compared to cooling capicity

Your total kilowatts per hour should not exceed 50% of your total cool capacity

Regular scheduled checks and maintenance

Weekly system checks Bi-Annual on-site maintenance

Joe Joplin Data Center from Conception to Operation

Questions Thank you for your attention

Joe Joplin Data Center from Conception to Operation

Sources
Uptime Institute Whitepapers
http://www.upsite.com/TUIpages/tuiwhite.html

APC UPC Selector Sizing Applications

http://www.apcc.com/template/size/apc/index.cfm

Dell configuration Spread sheet

http://www.dell.com/downloads/global/products/pedge/en/DellConfigCalculat or.xls

Power Calculation
http://lms.globalknowledge.com/ilearn/en/learner/jsp/clients/APC/customer/re gister.jsp?tsk=

Joe Joplin Data Center from Conception to Operation