White Paper
Demand Response Opportunities for
Data Center Embedded Generation
and Energy Storage Systems
White Paper 4
October 2021
By: Ed Ansett CEng, FIET, FBCS & Kerr Johnstone CEng, MIET
EYP Mission Critical Facilities, Part of Ramboll (EYP MCF, Part of Ramboll)
Executive Summary
Historically the data center industry has not participated in demand response. In part this is due to a lack of awareness, risk of downtime concerns and complexities within the utility and regulatory domains.
Many data center owners and operators have publicly pledged to address climate change recognizing that data centers make a significant contribution to the national and global carbon footprint.
Data centers are uniquely positioned to participate in demand response and consequently benefit from additional revenue streams and reduce average and marginal carbon emissions.
Depending on location and regulatory factors data center demand services are likely to succeed in assisting with GHG abatement even with the installed base of emergency diesel generators, provided the type of fuel used has significantly reduced carbon emissions. However, the increasing market penetration of natural gas/ hydrogen generators and fuels cells will enable increased participation in demand response and greater contributions to greenhouse gas reduction.
Combining data center on site generation and battery storage should enable a symbiotic relationship between data centers and utilities where both groups benefit by reducing their carbon footprint and from the operational and financial rewards.
Content
-
Overview
-
Grid Decarbonization
-
Utility Supply and Demand
-
Variable Renewable Energy
-
Demand Response
-
GHG Abatement
-
Data Center Power Utilization
-
Data Center Assets
-
Data Center Energy Storage
-
Data Center Generation
-
Demand Response Adoption Barriers
-
Regulatory and Utility Issues
-
Aggregators
-
Conclusion
1. Overview
The data center industry is uniquely positioned to participate in demand response (DR), and by doing so, will reduce its carbon emissions and generate additional income for data center owners. Unlike conventional DR service providers, data centers have an inherent ability to provide multiple stacked DR services from the same location.
This paper aims to provide insights into DR opportunities in the context of the data center power system infrastructure and considers the most appropriate types of utility support services suited to data centers.
2. Decentralized Consensus and Byzantine Fault Tolerance
Governments are reducing their dependence on high-inertia fossil fuel generation and deploying increasing amounts of renewable energy sources.
For many countries, the process of grid decarbonization will continue for decades as they strive to attain their commitment to GHG abatement.
Figure 1 - IEA Renewables reach new heights while coal-fired generation steadily declines3
Paradoxically, a major consideration in terms of grid decarbonization is the rate at which the global population is increasing, and with it the amount of additional energy that will need to be generated.
It is predicted that world energy consumption will grow by nearly 50% between 2018 and 2050. Most of this growth comes from countries outside of the OECD, focused in regions where strong economic growth is driving demand, particularly Asia.³
3. Utility Supply and Demand
The electricity grid must maintain a nominal voltage and frequency within specific limits. The challenge for utilities is that supply and demand are always balanced irrespective of variations in load conditions, generation status and distribution system faults.
The issue of balancing is exacerbated in countries with a high penetration of wind and solar power since both these energy sources are intermittent, and for the most part do not follow the load in the same way that conventional synchronous generation does.
Consequently, utilities are dependent on third party embedded generation and energy storage companies to provide balancing services as consumer demand and grid capacity fluctuates. An equilibrium is essential to ensure consumer voltage and frequency stays within mandatory operating parameters.
4. Variable Renewable Energy
Whilst wind and solar power generation benefit the grid in terms of carbon abatement, both power sources are inherently variable and intermittent in terms of timing and quantity of power generated; unlike hydro-electric and geothermal generation which are continuous. Wind and solar are therefore referred to as variable renewable energy sources (VRE).⁴
To reduce greenhouse gas emissions and improve the security of energy supply, many countries are replacing fossil fuels with renewable energy generation. Technology advances and economies of scale have reduced the cost of solar PV and wind generation. Consequently, their share in the global energy mix is increasing substantially. However, integrating high levels of VRE into power system operations is challenging.
Figure 2 – Example of the Intermittent and Uncertain Characteristics of VRE - Source: Ela and others (2013)
The issue with VRE sources is how to match supply and demand when the wind stops blowing, or the sun isn’t shining. The answer is the utility must hold generation in reserve to balance the grid.
Aside from fast-start peaker power plants, often the reserve takes the form of carbon intensive fossil fuel plants which are typically expensive to start and required to run for extensive periods. 6
VRE does not provide the rotational inertia associated with traditional synchronous generators. Consequently, as the ratio of non-synchronous generation on the grid increases, the frequency stability of the grid decreases.
Island countries with limited interconnects and a high penetration of VRE, are more susceptible to frequency instability compared to countries with strong interconnections with neighboring grids. Notwithstanding, even countries with strong interconnected grids require third party frequency support, particularly real time Short Term Operating Reserve (STOR) and frequency response to counter generation and distribution faults.
Consequently, utilities have responded by seeking demand response services from third parties that can inject energy into the grid to pre-empt or counteract in real time unacceptable frequency excursions.
Ireland for example, has a high proportion of wind VRE, so the utility has imposed a limit of non- synchronous penetration, thereby forcing large consumers, most notably data centers to run in island mode using low carbon embedded generation, i.e., natural gas reciprocating engines or turbines.
5. Demand Response
General
DR is the adjustment in demand relative to grid generating capacity, designed to address supply and demand imbalance, high wholesale electricity prices and assist with grid reliability.
The underlying objective of DR is to actively engage customers in modifying their generation or consumption in response to pricing signals. The goal is to reflect supply expectations through consumer price signals or controls and enable dynamic changes in consumption relative to price.
Generally, DR is achieved by consumers reducing energy consumed or by injecting energy into the grid based upon either the predicted day ahead market or the instantaneous real time market.
In the day ahead market, DR participants commit to buy or sell energy one day prior to the operating day, to help avoid price volatility, balance forecast generation and demand conditions. Whereas the real time market DR participants buy and sell electricity during the day in response to grid event driven utility signals.
The fundamental principles underpinning DR are illustrated in Figure 3.
Figure 3 - Balance between Electricity Generation and Consumption - Source: Next Kraftwerke GmbH
Companies participating in DR are paid according to the type of DR service they provide. Typically, DR services that are provided at the shortest notice, in real time, are more lucrative than day ahead services.
Types of DR Services
DR programs can generally be categorized as follows:
1. Load Curtailment
From a data center perspective, load shedding is the simplest and most obvious type of DR participation, where the data center disconnects from the grid and runs on its generators in island mode.
2. Load Shifting
Consumer power consumption is reduced during a peak demand period by shifting energy use to another time.
IT workload power use is typically not something that can be controlled by wholesale and colocation data center operators. However, it may be viable to reduce load to allow the on-site battery storage to partially discharge and support the IT load in parallel with, or in isolation from the grid.
End-user owned data centers can achieve load shifting by controlling compute and storage resources to reduce energy consumption or by shifting IT workloads to different geographical locations.⁵
3. Short Term Operating Reserve (STOR) and Load Reduction
The utility requests additional power when the actual demand is higher than forecast, or in the event of unforeseen generation unavailability.
DR providers help to meet the reserve requirement either by providing additional generation or demand reduction.
Typical requirements are:
-
Provide several megawatts of generation
-
Respond to a utility signal within 20 minutes
-
Sustain the response for a minimum of two hours
-
Respond again with a recovery period of not more than 20 hours
Data center participation in STOR involves supplying the grid and the data center from its generators, which will require the available data center generating capacity to be significantly higher than its load.
4. Frequency Response
Frequency response is often the most lucrative DR program. It's used by utilities to counter unplanned power generation and load imbalance that would otherwise cause a frequency stability problem. Typically, there are three categories of DR frequency support as shown in Figure 4.
Figure 4 - Frequency Regulation Processes and Activation Times⁷
-
Fast frequency response (FFR) is provided within a few seconds of a utility frequency event. The purpose is to reduce the initial extent of the disturbance.
-
Frequency containment reserve (FCR) is typically provided within 30 seconds to bring the frequency to a new steady state.
-
Frequency restoration reserve (FRR) is typically provided 30 seconds after the disturbance and is used to restore frequency to its nominal value.
Real time near instantaneous frequency response will normally be amongst the most lucrative DR services. It is envisaged this is probably most suited to bi-directional UPS Battery Energy Storage Systems (BESS).
In the United States, the PJM (a regional transmission organization - RTO) provides market-based compensation to resources that have the ability to adjust output or consumption in response to an automated signal. PJM generates two different types of automated signals that Regulation Market resources can follow. Regulation D signal is a fast, dynamic signal that requires resources to respond almost instantaneously, Whereas Regulation A is a slower signal that is meant to recover larger, longer fluctuations in system conditions. The two signals are managed so that they work together to match the system need for regulation.
Load curtailment, Load Shifting, Load Reduction and STOR can participate in either the day ahead market or real time incentive-based programs. Whereas frequency response services are predisposed to the real time market.
The types of programs available, integration complexity and remuneration values vary widely across utility geographies. The user should also be aware that utilities may penalize DR service providers that do not provide the agreed DR service when it is requested. Also, the terminology used to describe a particular DR service often varies between different territories.
The following types of DR service may be applicable to a minority of data centers:
5. Energy Arbitrage
Involves purchasing additional electricity from the utility during off-peak periods to charge the BESS, then discharging it back into the grid during peak periods.
6. Time Variant Pricing
Time variant pricing can be categorized as follows:
-
Real-time pricing
-
Time-of-use pricing
-
Critical peak pricing
-
Critical peak rebate
Figure 5 refers.
Figure 5 - Time-variant electricity pricing⁸
7. Distribution Deferral
Delaying, reducing the size of, or entirely avoiding utility investments in distribution system upgrades necessary to meet projected load growth on specific regions of the grid.
8. Transmission Congestion Deferral
ISOs charge utilities to use congested transmission corridors during certain times of the day. Assets including energy storage can be deployed downstream of congested transmission
corridors to discharge during congested periods and minimize congestion in the transmission system. ⁹
9. Transmission Deferral
Delaying, reducing the size of, or entirely avoiding utility investments in transmission system upgrades necessary to meet projected load growth on specific regions of the grid.⁹
10. Black Start
In the event of a grid outage, black start generation assets are needed to restore operation to larger power stations to bring the grid back online. In some cases, large power stations are themselves black start capable. ⁹
6. GHG Abatement
DR services contribute to GHG abatement by reducing the average carbon footprint and marginal emissions of the utility. The reduction in both types of utility emissions will also be reflected in a reduction of the data center carbon footprint. The extent of GHG reduction will be the utility fuel mix and the type of data center standby generation.
CO2e Reduction
Consider the situation when there is insufficient grid capacity, in response the utility signals DR service providers to disconnect from the grid and switch to island mode operation.
The impact on GHG depends on the prevailing grid emission factor (GEF) at the time. If for example, we assume a 50MW facility in China has natural gas generators as its emergency power source, the gas generator emission factor is approximately 486 g CO2e/kWh, compared to the national combined margin grid emission factor of 852 g CO2e kWh. For the purposes of this example, we will assume 852 g CO2e /kWh is the prevailing GEF when the generators are operating. Then if the standby generators are running in island mode for 10 hours, this results in 183,000kg CO2e saving, calculated as follows.
[852g CO2e /kWh x 50,000kWh] – [486g CO2e /kWh x 50,000kWh] = 18,300kg CO2
There is also a case for running diesel generators in island mode using alternative fuels such as Ultra-Low-Sulphur Diesel (ULSD), Gas-To-Liquid (GTL) or Hydrotreated Vegetable Oil (HVO). Each application should be assessed taking into consideration the difference in CO2e between the grid emission factor and the emission factor of the diesel engine running on the alternative fuel type.
Marginal Emissions Reduction
Marginal emissions occur when the utility brings a different type of generating plant on the grid. This is relevant when demand reaches a point such that the utility needs to start up non-CCS coal or oil- fired generators.
Consider the example where the load on a grid is 20GW, comprising 7.5GW of wind and 7.5GW of PV renewable energy and 5GW of CCGT all running to match the load. Then the demand changes to 20.1GW. At this point the utility must add generation by bringing on line one of its oil-fired plants, albeit just to cover off the 0.1GW necessary to meet demand. Assuming the oil-fired generation has 777 g CO2e /kWh emission factor, then an additional 777g of CO2e is required per kWh associated with the additional load.
Figure 6 – The Impact of Marginal Emissions
Consider a situation where a 100MW data center was able to react to a signal from the utility and disconnect from the grid and run in island mode on its lower emission generators, this would avoid the requirement to start up the utility oil-fired generators, thereby reducing marginal emissions.
The wider implications of operating data centers using low carbon energy storage and generation systems will be discussed in a separate paper by the EYP MCF, Part of Ramboll i3 GHG Abatement Group, titled Low GHG energy Trading Opportunities for Large Scale Data Centers.
7. Data Center Power Utilization
One of the most pervasive issues affecting data centers is low utilization values. Utilization is defined as the ratio of maximum actual load to the amount of load contracted by the end user. The general perspective is presented in Figure 7, which states the typical utilization across a large portfolio of data centers is just 40%.
Figure 7 – Power Over-provisioning in Data Centers – Source Equinix
A more granular perspective in terms of load growth is shown in Figures 8 and 9 for a hyperscale end-user and a connectivity provider respectively over a 24-month period with maximum utilization levels of 43% and 45% respectively.
Figure 8 – Hyperscale Load Utilisation Example – Source: ServerFarm
Figure 9 - Connectivity Provider Load Utilisation Example – Source: ServerFarm
The over-provisioned power and cooling infrastructure is wasteful, both in terms of investment and carbon footprint. Evidently there is an opportunity for data center owners and end-users to exploit what are otherwise underutilized power assets by using the excess capacity to provide DR services, and in doing so provide additional revenue and reduce carbon footprint.
Data Center Assets
Data centers are protected from grid disturbances and outages by UPS. The purpose of the UPS is to condition the power supply to the critical IT load and provide continuous power that bridges the gap between the start of a utility outage and the time taken for emergency generators to accept the load.
Most data centers still use diesel generators to provide emergency power, and either lead acid (VRLA) or more recently, lithium-ion batteries to perform energy ride-through.
Data centers are not designed to be bi-directional microgrids. However, they are designed to operate as unidirectional microgrids occasionally, i.e., during a utility failure when generators are used to supply the data center load.
Data centers have inherent energy storage in their UPS batteries and flywheels as well as generating capacity in their emergency generators that can be used to provide DR services in the appropriate circumstances.
As Wierman et al explain, when grid operators look at data centers, they should view them as large- scale energy storage installations that are sitting unused due to a lack of appropriate programs… each data center represents millions of dollars of unused fast-response storage-equivalent capacity.¹¹
Data Center Storage
Most utility events are short duration transient disturbances lasting less than a few seconds11 where the UPS energy store momentarily provides power during the utility event.
The UPS energy storage is also used to provide power during longer utility events, to provide continuous power to the IT load, and sometimes to maintain cooling circulation from the moment the outage commences until the data center generators accept load.
UPS energy storage is usually provided by batteries or flywheels. The energy storage capacity of the UPS is calculated at full load and is typically 5 to 10 minutes for batteries, and 10 to 20 seconds for flywheels.
Historically most UPS have used lead acid batteries. However, new data centers are increasingly using lithium-ion batteries.
Different types of batteries and their application to data centers are discussed in a separate paper by the EYP MCF, Part of Ramboll & i3 GHG Abatement Group titled the Assessment and Application of BESS to data centers.
In developed countries where the utility grid is stable, UPS batteries and flywheels remain idle most of the time. Until recently, this was considered the inherent price to pay for the occasional time when they were needed to support the critical load. However, with the introduction of DR service incentives by utilities, there is an opportunity to utilize UPS energy storage to provide DR support services.
This has been recognized by UPS suppliers such as ABB, Eaton, Schneider12, and Vertiv13 who have configured their UPS batteries to provide grid support services. In principle, flywheels could also provide frequency support albeit only short duration FFR services.
The DR applications associated with UPS energy storage are frequency response services due to the speed in which UPS can respond, but limited by the energy storage capacity of the UPS.
The specific types of DR frequency response applicable in general are shown in Table 10.
Table 10 – UPS Energy Storage Potential DR Services
10. Data Center Generation
Like UPS, data center generation equipment is idle almost all the time except for the occasional utility outage and periodic testing. Generators are even less utilized than UPS energy storage since they cannot respond quickly enough to start and accept load during a short duration utility event. Therefore, during these extensive periods of inactivity, these assets are non-productive.
Most data center generators use conventional diesel fuel. However, there is increasing pressure from governments to look at alternatives to diesel engines. Consequently, whilst the installed base almost entirely comprises diesel reciprocating engines, the use of lower carbon NG engines, turbines and PEM is increasing.
From a GHG abatement perspective, both diesel and NG gas engines can be used to provide DR services under the appropriate circumstances. In other words, when their use in providing DR services has a net positive contribution to the reduction of GHG emissions. This is determined by the GEF associated with each application and the marginal emissions that could be avoided.
With diesel generators in particular, the predisposition to providing DR services is determined by the type of DR service under consideration, the type of engine and the type of fuel used i.e., GTL, ULSD or Dipetane and biofuels such as HVO in lieu of conventional diesel.
The type of engine, i.e., standby, prime or continuous will determine the duration and frequency of runtime limits, and the fuel type will determine the emission factor of the engine relative to local GEF. Given, the typical minimal actual runtime of diesel generators, especially in developed countries, it is likely that even standby rated machines can contribute to DR, albeit less than prime and continuously rated machines.
Fuel cells can be used to provide DR services. Solid Oxide Fuel Cells (SOFCs) are unsuitable to provide real time frequency support services due to their relatively slow load following capability. Whereas hydrogen Proton-Exchange Membrane (PEM) fuels cells are quicker in terms of load following ability and may be suitable for FFR, FCR and FRR frequency services. However, in principle SOFCs and PEM fuel cells are both able to provide load curtailment, load shifting and STOR services.
The various types of DR services that could be provided by generators and fuel cells are indicated in Table 11.
Table 11 – Generator and Fuel Cell Potential DR Services
11. Demand Response Barriers
The majority of the data center industry has not yet participated in demand response services for several reasons.
-
Lack of awareness DR programs and policies
-
Unsuitable DR programs for data centers, e.g., the ability to stack services
-
Perceived additional risk to the critical load
-
DR is not a core business
-
Complexities of implementation and operation
-
Utility interface issues
-
Regulatory issues i.e., limited hours of operation for Diesel generators
Each of the above considerations can have merit, however wholesale and colocation data center owners are confronted with two major commercial issues.
-
Data center owners have made public commitments to sustainability. It is evident that major reductions in carbon footprint will not be achieved by incremental efficiency improvements. And the purchase of RECs and carbon offsetting are losing credibility. So, there must be a new factor that will make a significant difference to a company’s carbon footprint.
-
Secondly, with new data center companies constantly entering the market, competition has eroded margins. Therefore, the opportunity to generate revenue by accessing what are essentially stranded assets through DR is an obvious consideration. Each company will base their decision to participate on their own circumstances and the relative financial and carbon merits in the territories they operate.
The interoperability of DR services is complex and still evolving. The extent of DR services available and the complexity of participation varies considerably across countries.
12. Regulatory and Utility Issues
Some key regulatory issues that need to be addressed include the removal of barriers that preclude the use of multiple stacked services. Also, the restructuring of business rates to reflect the value that energy storage can provide to the grid via temporal, locational, and attribute-based functionality, making utilities indifferent to the distinction between distributed and centralized resources.⁹
13. Aggregators
To participate in DR, data center owners have two choices: either go it alone or engage with an aggregator.
The benefit of using an aggregator is that the complexities of DR participation are simplified. Decisions about what services to offer, grid integration, control signaling, and practical implementation are all handled by the aggregator. The only downside is that the aggregator takes a share of the revenue, that would not otherwise happen if the data center operator were to work directly with the utility.
Where the data center owner decides to go it alone, they are responsible for all implementation, operational and commercial issues.
For large data centers organizations with a large portfolio, supported by in-house technical and commercial expertise, it may be appropriate to provide DR services independently without an aggregator. On the other hand, smaller data center companies will probably benefit most from using an aggregator.
14. Conclusion
Data centers represent a significant and increasing load on the grid. Given the sustainability imperatives and the obvious desire to improve margins it seems logical that the data center industry will increasingly participate in DR programs.
This is likely to initially involve load curtailment, i.e., operating in island mode. As the industry gains confidence and the regulatory authority recognizes the immense potential of the data center sector, this will be followed by frequency response services.
Undoubtedly, combining on-site generation with UPS BESS with import and export connections to the grid will unlock stranded investment in UPS batteries and generators.
For contemporary data centers, on-site power will probably see the increased use of gas generators or biofuel/ low carbon diesel engines (also hydrogen) to meet environmental legislation. Both are likely to provide significant GHG reductions.
The highest income from participation in DR will probably be derived from premium prices paid under real time load shedding, STOR and fast frequency response.
Utilities will need to adapt industry requirements and permit data centers to provide multiple stacked DR services to fully realize the untapped potential of data centers in terms of available capacity, and to ensure equitable DR programs that benefit the utility and data center owners.
Abbreviations and Acronyms
BES Battery Energy Storage
CCGT Combined cycle gas turbine
CCS Carbon capture and storage
CO2e Equivalent carbon dioxide
DR Demand Response
EIA US Energy Information Administration
FCR Frequency containment reserve
FERC Federal Energy Regulatory Commission
FFR Fast frequency response
FRR Frequency restoration reserve
GEF Grid emission factor
GHG Greenhouse gas
GTL Gas to liquid
GW Gigawatts
HVO Hydrotreated vegetable oil
ISO Independent system operator
kW Kilowatts
kWh Kilowatt hours
MW Megawatts
MWh Megawatt hours
NG Natural gas
OECD Organization for Economic Cooperation
PEM Proton exchange membrane
PV Photovoltaic
RE Renewable Energy
REC Renewable energy credit
RTO Regional transmission organization
SNSP System non-synchronous penetration
SOFC Solid oxide fuel cell
STOR Short term operating reserve
ULSD Ultra-low sulfur diesel
UPS Uninterruptible power supply
VRE Variable Renewable Energy
References
-
International Energy Agency - Emissions Factors 2021 https://www.iea.org/data-and- statistics/data-product/emissions-factors-2021
-
Institute for Global Environmental Strategies (2021). List of Grid Emission Factors, version 10.10. https://pub.iges.or.jp/pub/iges-list-grid-emission-factors
-
US Energy Information Administration https://www.eia.gov/todayinenergy/detail.php?id=41433#
-
World Bank Group, Livewire – Integrating Variable Renewable Energy into Power System Operations
-
Ghatikar - LBNL Demand Response Opportunities and Enabling Technologies for Data Centers: Findings from Field Studies (2012)
-
Johnson et al - Understanding the impact of non-synchronous wind and solar generation on grid stability and identifying mitigation pathways (2020)
-
Sandelic et al - Battery Storage-Based Frequency Containment Reserves in Large Wind Penetrated Scenarios: A Practical Approach to Sizing (2018)
-
All Electricity is Not Priced Equally: Time-Variant Pricing 101, EDF http://blogs.edf.org/energyexchange/2015/01/27/all-electricity-is-not-priced-equally-time- variant-pricing-101/
-
Fitzgerald et al, Rocky Mountain Institute – The Economics of Battery Energy Storage
-
Wierman et al, Opportunities and Challenges for Data Center Demand Response
-
Grebe et al, EPRI - An assessment of distribution system power quality
-
Thompson and Avelar, Schneider – Monetizing Energy Storage in the Data Center
-
Di Filippi and Valentini, Vertiv – How to Maximize Revenues from Your Data Center Energy Storage System with Grid-Interactive UPS
Contact Us
For further information about the i3 EYP MCF, Part of Ramboll GHG Abatement Group, please email David Eisenband deisenband@eypmcfinc.com or Kerry Neville kerry.neville@i3.solutions