c

Data Center
Facilities
Design for IT
E-book

CHAPTER 3

BACKUP POWER,
DISTRIBUTION
AND CONVERSIONS
Targeted steps to lower data center power
consumption and strengthen the bottom line
BY MARK FONTECCHIO AND STEPHEN J. BIGELOW

+
+
+
+
+

Conservation starts with utility providers

Saving power at the server
The debate of AC vs. DC power
Replacing batteries with flywheels
Measuring efficiency progress

CHAPTER 3

Backup power,
distribution and
conversions
Feeding the data centers energy needs
without breaking the bank is possible
if you know a few tricks.

Power consumption has become the reference of

choice when describing the size of a data center. Before
the rising costs of power caught the industrys eye, data
center managers described their facilities in terms of
square footage. Now they talk about how many megawatts
the data center consumes. In fact, some data centers still
have plenty of physical room, but cant expand. The power
capacity is tapped to the point that data center managers

and the government are

eager to reward good energy behavior now, it wont be
long before power and carbon regulations are in place
to punish bad behavior.

CONSERVATION STARTS
WITH UTILITY PROVIDERS

cant even plug in more IT equipment.

But why should data center managers care if they arent currently
hurting for power capacity? A shortterm benefit of controlling power consumption is a lower bottom line on
your companys electric bill. In the
long term, it can forestall new data
center construction projects, which
could cost tens or hundreds of millions of dollars. Those are financial
benefits that company executives
favorand could prompt salary
bumps or promotions.
Finally, while utility companies

The first step in conserving power

in a new data center begins with the
utility providers. Because power
usage has become a dominant factor
in data center operations, begin discussions with local utility providers
early when youre planning to build
a new facility.
If a companys data center is large
enough, it may be able to negotiate
with the utility provider to help pay
for a new electrical substation. The
company might also agree to limit
electricity use when the power grid
is straining.

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

CHAPTER 3

Some utility providers have incentive programs that refund companies

for making their data centers more
energy efficient. A leader in this field
is San Francisco-based Pacific Gas

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

SAVING POWER
AT THE SERVER

IT equipment is
the largest energy
consumer within
the data center
accounting for
about half of the
energy used.

THE DEBATE OF
AC VS. DC POWER

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

and Electric (PG&E). The utility company has 25 different energy-efficiency incentive programs, including one
for deploying server virtualization and
another that offers discounts to companies that install air-side or waterside economizers.
The online marketplace eBay has
taken advantage of utility provider
rebates. Olivier Sanche, former senior
director of eBays data center services, said the company did a simple
tech refresh, installed solar panels on
the roof of its San Jose, Calif., data
center and purchased economizers to
help reduce energy consumption and
save money.
The incentives help us do the right
thing, said Sanche. Would we be
doing some of these things without
the incentives? Probably, yes. But this
[incentives] helps the projects hap-

pen quicker.
Not all utility providers have these
incentive programs in place, so its
important to check with each
provider individually. Rebate money
comes from the rate base, meaning
that rate payers fund the program,
said Chris Johnston, national critical
facilities chief engineer for New Yorkbased data center design firm Syska
Hennessy. In essence, smart data
center users benefit financially from
other not-so-smart users.

SAVING POWER AT THE SERVER

IT equipment is the largest energy

consumer within the data center
accounting for about half of the energy used, which makes it the best
place to start when trying to save
energy. A simple tech refresh can go
a long way toward saving energy in
the data center.
Equipment manufacturers are creating new generations of processors,
memory and power supplies that are
more energy efficient than previous
generations. This is mainly because
of pressure from within the industry
as power consumption has grown
into a major data center issue.
The Environmental Protection
Agency (EPA) now has an Energy Star
rating for data center servers, including 1U, 2U and blade servers. Servers
that meet Energy Star requirements
are recognized for their energy efficiency and power-saving features.
In June 2010, the EPA began issuing
Energy Star ratings for large data cen-

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

CHAPTER 3

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

ters. The Standard Performance Evaluation Corp., an industry benchmarking organization based in Warrenton,
Va., also publishes several powerrelated metrics. The first, called
SPECpower ssj2008, compares a
servers Java performance to its
power consumption. The SPECvirt_
sc2010 benchmark evaluates server
performance in a virtual data center.
Taking advantage of these free tools
can help determine which servers are
best from a power perspective.

Server virtualization is one of

the most common ways to reduce
power costs in a data center. Once a
server is virtualized, its possible to
consolidate multiple virtual workloads onto one physical server, allowing the same number of workloads to
operate on fewer physical machines.
This results in vastly improved server
use and considerable power and cooling savings, and is a key element to
power conservation strategies. By
comparison, non-virtualized servers

SAVING POWER
AT THE SERVER

THE DEBATE OF
AC VS. DC POWER

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

FIVE WAYS TO SAVE POWER

IN THE DATA CENTER
Buy energy-efficient servers. Newer servers tend to be more energy
efficient; a simple tech refresh can actually save money. The Environmental
Protection Agency (EPA) lists servers that qualify under its Energy Star
energy-efficient server standard.
Optimize server efficiency. Deploying virtualization increases server use,
which can help reduce overall power consumption. Turning on power-saving
server features, such as a sleep mode, can also save money.
Improve data center cooling infrastructures. Data center cooling
accounts for about 37% of a center's power consumption. Making smart cooling choices and planning efficient airflow designs will help cap energy use.
Use flywheel power supplies. Flywheel uninterrupted power supply (UPS)
units rely on kinetic energy instead of huge strings of batteries. Because of
this, flywheel UPSes can provide temporary backup power to IT equipment
and save money.
Measure, measure, measure. Although you wont see direct savings simply
from measuring your energy use, knowing where you stand in regard to
power consumption gives you a starting point for any efficiency evaluation,
and provides a basis for making informed decisions.

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

CHAPTER 3

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

SAVING POWER
AT THE SERVER

THE DEBATE OF
AC VS. DC POWER

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

often run at low usage rates, sometimes in the single digits.

The mantra with data center servers
used to be one app, one server. Virtualization software, however, has
challenged this, allowing IT managers
to stack several virtual machines
(VMs) on top of a single physical box.
This means that IT managers can get
rid of older servers or delay the purchase of additional servers.
Server power supplies have also
received a lot of attention as a way to
save energy. The 80 Plus initiative, an
electric utility-funded program based
in Portland, Ore., certifies server power
supplies that are at least 80% efficient. This means that 20% or less of
the energy coming from the servers
supply turns into wasted heat. Older
servers could have power supplies
with efficiency levels as low as 60%.
Finally, more data center administrators are implementing the powerdown features available in modern
servers, which often are already
embedded into operating systems
and just need to be activated. When
IT loads are low, power-down features put unused servers into sleep
mode, much like desktop computers
go into standby mode. When IT
needs spike, servers turn on again.
If you can adjust all of these server
components to be more energy efficient, the benefits can add up. In
2007, the EPA reported to Congress
that a data center server incorporating energy efficiencies throughout all
of its components could consume
25% less energy in the data center.

Power supplies on
older servers could have
efficiency levels
as poor as 60%.

Today, were seeing more energyefficient server components and

power-saving features appear in
the latest server models.
Cooling equipment is the second
largest energy consumer in the data
center. Cooling consumes up 37% of
a data centers power envelope, noted
an Emerson Network Power report.
Carefully raising the temperature in
the data center, avoiding bypass and
re-circulated air and using variablespeed cooling devices are just a few
ways to save money when controlling
your data centers climate.
.
THE DEBATE OF AC VS. DC POWER

According to the SearchDataCenter.com 2011 Data Center Decisions

survey, more than 27% of respondents have implemented direct current (DC) power or plan to do so in
the next year. Still, there are questions
about whether DC power is more efficient than alternating current (AC).
The AC versus DC dispute dates
back to the 19th century, when DC
power advocate Thomas Edison

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

Because right-sized data center

infrastructure is good business strategy
Optimize your cash flow and efficiency with our
scalable, flexible, and adaptable InfraStruxure
Introducing Next Generation InfraStruxure

Whether you just acquired a new company or must increase its ever-expanding customer
or inventory database capacity, youre most likely facing pressing demands on your
companys IT infrastructure. Your existing data center infrastructure may not be able to
handle these up-to-the-minute changes. Thats where APC by Schneider Electric steps in
with its proven high-performance, scalable, data center infrastructure. As the industrys oneof-a-kind, truly modular, adaptable, and on-demand data center system, only InfraStruxure
ensures that your data center can adapt effectively, efficiently, and, perhaps most
important, quickly, to business changes.

InfraStruxure data centers mean business

We say that InfraStruxure data centers mean business. But what does that mean to you?
The answer is simple. A data center means business when it is always available, 24/7/365,
and performs at the highest level at all times, is able to grow at the breakneck speed of
business, lets you add capacity without waiting on logistical delays (e.g., work orders),
enables IT and facilities to keep pace with the business in a synchronized way, continues to
achieve greater and greater energy efficiencyfrom planning through operations, is able
to grow with the business itself, and supportsinstead of hindersbusiness.

The triple promise of InfraStruxure deployment

InfraStruxure fulfills our triple promise of superior quality, which ensures highest
availability; speed, which ensures easy and quick alignment of IT to business needs; and
cost savings based on energy efficiency. What better way to mean business than to
enable quality, speed, and cost savingssimultaneously?

2
3

The only right-sized data center

architecture engineered as a system
1

Cooling. Rack-, row-, and room-based cooling options, including new overhead InRow
cooling units, for greater efficiency

2 Management. End-to-end monitoring and

management software for greater efficiency
and availability
3

Physical Security. A single-seat view for

monitoring and surveillance across the facility.

4 Power. Modular power distribution and

parallelling capabilities on UPS for loads from
10 kW to 2 MW
5 Rack Systems. Any-IT vendor-compatible
rack enclosures and accessories for
high densities

Classification of Data
Center Operations
Technology (OT)
Management Tools
> Executive summary

Contents

7
9
10

Discover which physical infrastructure management

tools you need to operate your data center
download White Paper #104, Classification of Data Center
Operations Technology (OT) Management Tools, today!
Visit www.apc.com/promo Key Code b542v Call 888-289-APCC x9809 Fax 401-788-2797

2011 Schneider Electric. All Rights Reserved. Schneider Electric, APC, InRow, and InfraStruxure are trademarks owned by Schneider Electric Industries SAS or its affiliated companies.
e-mail: esupport@apc.com 132 Fairgrounds Road, West Kingston, RI 02892 USA 998-5038_US

CHAPTER 3

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

squared off against AC power backers

George Westinghouse and Nikola
Tesla. AC power won largely because
it could be transferred more easily over
long distances. The battle has since
been dubbed the war of currents.
In 2006, the Lawrence Berkeley
National Laboratory performed a
demonstration project in California to
compare the use of AC power versus
DC power in the data center. The laboratory claimed data centers could
see up to 20% savings using DC

power. How is this possible? The

difference between the two currents
is in the conversions.
In a typical AC-powered data center, electricity comes from the utility
as AC power at either 480 V or 600
V. By the time it reaches the servers
components, its usually running at 12
V of DC power. In between the moderate-voltage AC and low-voltage DC
levels, there are multiple points where
the power is converted back and forth
from AC to DC power.

SAVING POWER
AT THE SERVER

THE DEBATE OF
AC VS. DC POWER

THE DATA CENTER POWER TRAIN

the utility company to IT equipment involves several
steps all data center staff should know. From the utility grid, the power goes
through switchgear to the automatic transfer switch (ATS). In a power emergency, the ATS changes loads from the utility to fuel-based generators. These
generators keep the data center running until the main power returns
or the data center backs up to a disaster recovery site.
The wait for obtaining a 2 MW diesel generatorthe most popular for data
centerscan often be longer than a year. Uninterruptible power supply (UPS)
units act as immediate backup power sources for the IT load. If the utility
power feed is on when a current goes through the UPS input, it powers IT
equipment. If the power goes out, the load is transferred to the UPS until the
generators can get up and runningusually a few seconds.
Traditionally, UPS units have large cabinets of batteries. The amount of time
the UPS unit can handle the load depends on the size of the load and how many
batteries are available for backup. Some data centers now also use flywheels in
place of or in conjunction with batteries to run UPS units.
The next stop in the data center power train for the IT load is the power distribution units. These units are typically installed in server cabinets and act as
power strips for servers, storage and networking gear. While cheaper models
do indeed resemble power strips, more costly ones can provide power consumption data for either the entire unit or individual receptacles back to data
center staff via a Web browser.
TRANSFERRING POWER FROM

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

CHAPTER 3

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

SAVING POWER
AT THE SERVER

THE DEBATE OF
AC VS. DC POWER

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

Proponents of DC power claim that

each of those conversions introduce
losses that affect efficiency. In a DCpowered data center, there is just one
conversion at the beginningAC to
DC. But this has its drawbacks.
DC-powered IT equipment can be
more difficult to find on the market.
Transferring AC power over long distances is easier; doing so with DC
power requires much thicker wires.
This is why AC power eventually won
the war of currents.
There is an ongoing study that
questions whether DC power presents any savings at all. The Green
Grid, a Beaverton, Ore.-based nonprofit focused on data center efficiency, released a white paper on different
data center power distribution configurations. In part, the study concluded
that there were no major differences
between AC- and DC-powered data
centers.
When considering efficiency, there
is no single AC or DC configuration
that provides superior efficiency at all
loads or in all situations, the report
stated. The efficiency differences
among the contemporary implementations are relatively minor. All components and configurations have
undergone great efficiency improvements in the last decade and will likely
continue to improve.

REPLACING BATTERIES
WITH FLYWHEELS

Some data centers have looked at

using flywheels in place of or in con-

junction with batteries to run their

uninterruptible power supply (UPS)
units. Generating electricity using a
flywheel is not new; the technology
has been around for decades.
The main power source for the data
center jumpstarts the flywheel and
starts it spinning. This builds kinetic
energy based on the mass of the flywheel and the speed at which it
rotates, which can be as fast as
54,000 rotations per minute. When
the power goes outeven if its just
for a second or twothe flywheel
releases its built-up kinetic energy
into the data center until primary
power returns or a backup generator
starts.
The chief criticism of using flywheels is that they dont provide
power to the data center for as long
as batteries can. While flywheels can
provide ride-through backup power
for about 15 seconds, batteries can
provide power for at least 15 minutes,
depending on the length of the battery string.

MEASURING PROGRESS

While there are several established

methods to improve data center energy efficiency, none of them matter
unless you can properly measure
improvements. Being able to prove
that a data center is more efficient
and how that efficiency equates to
lower energy bills will help justify
future efficiency projects.
The Green Grid created the basic
data center energy efficiency metric

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

CHAPTER 3

called power usage effectiveness

(PUE). The metric compares the
amount of power that enters a facility
to the power that goes to IT equipment. Dividing the former by the latter yields a ratio that improves as it
gets closer to 1.
According to The Uptime Institute,
a data center consulting group headquartered in Santa Fe, N.M., the aver-

age PUE as of May of 2011 was 1.8.

Therefore, for every 1.8 W entering a
data center, only 1 W is used to power
IT equipment. The remaining 0.8 W
are used to cool the facility and are
lost in wasted heat through transfers
between equipment.
Data centers can also use the PUEs
reciprocaldata center infrastructure
efficiency (DCIE)which is expressed

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

SAVING POWER
AT THE SERVER

THE DEBATE OF
AC VS. DC POWER

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

EPA DRAFTS DATA CENTER

FACILITY SPECS
is taking a closer look at data centers and their computing equipment. Interest began in 2007 when Lawrence Berkeley National Laboratory scientist Jonathan Koomey released a report on global data center energy
consumption. Koomey found that data centers consume about 1.2% of all electricity in the U.S. The Environmental Protection Agency (EPA) became interested in finding ways to make data centers more efficient. After developing an
Energy Star specification for servers, the EPA has developed a loose specification for measuring the energy efficiency of the actual data center facility.
Generally, the Energy Star spec leans heavily on power usage effectiveness
(PUE), a metric that compares the power entering a facility to the power that
servers consume. The closer these numbers are to one another, the more energy efficient the data center is. But some experts question the validity of devising an Energy Star specification around PUE alone since the metric doesnt
account for server functions.
If IT equipment sits idle and remains unused, the data center wastes energyno matter how efficient the PUE says it is. Its likely that the EPA will
eventually incorporate some measure of productivity as well as a way to measure energy efficiency into this spec.
Experts note the potential for different data center categories, but would
prefer that those categories are defined by the type of work the data center
performsnot necessarily what industry they are in. But its going to take some
time to hash it out; further development and refinement of the data center
Energy Star spec is ongoing.
THE FEDERAL GOVERNMENT

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

CHAPTER 3

as a percentage that shows improvement as it nears 100%. Measurements for this number should be
taken in two locations. Energy use
should be measured at or near the
facilitys utility meter.

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

SAVING POWER
AT THE SERVER

THE DEBATE OF
AC VS. DC POWER

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

It's important to
keep the PUE metric
in perspective. The
Green Grid and other
experts recommend
keeping PUE
numbers internal.
If the data center is in a mixeduse facility or office building, take a
measurement only at the meter thats
powering the data center. If it is not
on a separate utility meter, estimate
the amount of power that the nondata center portion of the building
consumes and remove it from the
equation.
IT equipment load should be measured after power conversion, switching and conditioning is completed.
According to The Green Grid, the
most useful measurement point is at
the output of the power distribution
units (PDUs), but it can also be taken
at the output of the UPS. If you take
the measurement at the output of the
UPS, your PUE should be lower, and
therefore less accurate, than if you
take it at the output of the PDUs,

because there are fewer power conversions. This measurement should

represent the total power delivered to
the server racks in the data center.
Its important to keep the PUE metric in perspective. Some data center
colocation companies release projected PUE numbers; however, The Green
Grid and other data center experts
recommend keeping PUE numbers internal since theyre not all-encompassing. PUE doesnt take into account
how productive IT equipment is.

STANDARDS AND
LEED CERTIFICATION

After years of research, the EPA has

developed an Energy Star specification for data center facilities, which
leans heavily on PUE. Data centers
can also obtain a Leadership in Energy and Environmental Design (LEED)
certification for energy efficiency.
The U.S. Green Building Council
designed the LEED certification for
any building, but data centers can
also participate. Some data centers
seek LEED certification for both the
energy efficiency aspects as well as
its possible public relations benefits.
The certification, however, does have
its drawbacks since it doesnt focus
specifically on data centers, which are
more power hungry than traditional
office environments.
Despite that, companies such as
Digital Realty Trust, a large data center realty company, have obtained
LEED certification for their data
centers.

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

10

CHAPTER 3

CONSERVATION
STARTS WITH
UTILITY PROVIDERS

SAVING POWER
AT THE SERVER

THE DEBATE OF
AC VS. DC POWER

USING FLYWHEELS
INSTEAD OF
BATTERIES

MEASURING
PROGRESS

LEED is a very holistic view of

what happens to a building, said Jim
Smith, Digitals VP of engineering.
Is LEED 100% appropriate for data
centers? Yes. Is it the best metric to
decide whether a data center is good
or bad? Probably not.
So what is the future of data center
energy efficiency? The industry is
continuing to define and refine energy
efficiency in data center design. And,
over time, data center managers will
adopt the goals of energy efficiency
as best practiceswhether they want
to or not.
Many large corporations now have
company-wide policies for controlling
energy consumption, and data centers are a large slice of that pie. The
federal government is moving toward
an energy policy that would reward
companies and people who save
energy and limit carbon dioxide emissions as well as punish those who
dont. Getting ahead of that curve will
bring job security for data center
managers.

SENIOR TECHNOLOGY EDITOR

Stephen J. Bigelow

SITE EDITOR

Nicole Harding

SENIOR MANAGING EDITORS

Michelle Boisvert
Lauren Horwitz

MANAGING EDITOR

Christine Casatelli

ASSOCIATE MANAGING EDITORS

Jeannette Beltran
Eugene Demaitre
Martha Moore

DIRECTOR OF ONLINE DESIGN

Linda Koury
ABOUT THE AUTHORS

Mark Fontecchio is currently site editor for

SearchOracle.com. Before heading SearchOracle.com, Mark spent four years reporting and writing about data centers and servers at SearchDataCenter.com.
Stephen J. Bigelow, senior technology writer, has
more than 20 years of technical writing experience
in the PC and technology industry. He has written
hundreds of articles and more than 15 feature
books on computer troubleshooting. Contact him
at sbigelow@techtarget.com.

EDITORIAL DIRECTOR

Cathleen Gagne

PUBLISHER

Marc Laplante

TechTarget
275 Grove Street
Newton, MA 02466
www.techtarget.com

REVISED: Data Center Facilities IT Handbook, 2009.

DATA CENTER FACILITIES DESIGN FOR IT SEARCHDATACENTER.COM

11

RESOURCES FROM OUR SPONSOR

See ad page 6

APC Trade Off Tools

Interactive InfraStruXure Tour
APC.com

About APC by Schneider Electric:

APC by Schneider Electric, a global leader in critical power and cooling services, provides industry
leading product, software and systems for home, office, data center and factory floor applications.
APC delivers pioneering, energy efficient solutions for critical technology and industrial
applications.