Better data centers through machine learning

It’s no secret that we’re obsessed with saving energy. For over a decade we’ve been designing and building data centers that use half the energy of a typical data center, and we’re always looking for ways to reduce our energy use even further. In our pursuit of extreme efficiency, we’ve hit upon a new tool: machine learning. Today we’re releasing a white paper (PDF) on how we’re using neural networks to optimize data center operations and drive our energy use to new lows.

It all started as a 20 percent project, a Google tradition of carving out time for work that falls outside of one’s official job description. Jim Gao, an engineer on our data center team, is well-acquainted with the operational data we gather daily in the course of running our data centers. We calculate PUE, a measure of energy efficiency, every 30 seconds, and we’re constantly tracking things like total IT load (the amount of energy our servers and networking equipment are using at any time), outside air temperature (which affects how our cooling towers work) and the levels at which we set our mechanical and cooling equipment. Being a smart guy—our affectionate nickname for him is “Boy Genius”—Jim realized that we could be doing more with this data. He studied up on machine learning and started building models to predict—and improve—data center performance.
The mechanical plant at our facility in The Dalles, Ore. The data center team is constantly tracking the performance of the heat exchangers and other mechanical equipment pictured here.

What Jim designed works a lot like other examples of machine learning, like speech recognition: a computer analyzes large amounts of data to recognize patterns and “learn” from them. In a dynamic environment like a data center, it can be difficult for humans to see how all of the variables—IT load, outside air temperature, etc.—interact with each other. One thing computers are good at is seeing the underlying story in the data, so Jim took the information we gather in the course of our daily operations and ran it through a model to help make sense of complex interactions that his team—being mere mortals—may not otherwise have noticed.
A simplified version of what the models do: take a bunch of data, find the hidden interactions, then provide recommendations that optimize for energy efficiency.

After some trial and error, Jim’s models are now 99.6 percent accurate in predicting PUE. This means he can use the models to come up with new ways to squeeze more efficiency out of our operations. For example, a couple months ago we had to take some servers offline for a few days—which would normally make that data center less energy efficient. But we were able to use Jim’s models to change our cooling setup temporarily—reducing the impact of the change on our PUE for that time period. Small tweaks like this, on an ongoing basis, add up to significant savings in both energy and money.

The models can predict PUE with 99.6 percent accuracy.

By pushing the boundaries of data center operations, Jim and his team have opened up a new world of opportunities to improve data center performance and reduce energy consumption. He lays out his approach in the white paper, so other data center operators that dabble in machine learning (or who have a resident genius around who wants to figure it out) can give it a try as well.

How green is the Internet?

More than ever, people are using the Internet to shop, read, listen to music and learn. And businesses rely on Internet-based tools to operate and deliver their services efficiently. The Internet has created all kinds of new opportunities for society and the economy—but what does it mean for the environment?

We’ve been working to answer that question and enlisted the help of Lawrence Berkeley National Laboratory (Berkeley Lab) to gather more data. Their study (PDF), released today, shows that migrating all U.S. office workers to the cloud could save up to 87 percent of IT energy use—about 23 billion kilowatt-hours of electricity annually, or enough to power the city of Los Angeles for a year. The savings are associated with shifting people in the workforce to Internet-based applications like email, word processing and customer relationship software.


These results indicate that the Internet offers huge potential for energy savings. We’re especially excited that Berkeley Lab has made its model publicly available so other researchers and experts can plug in their own assumptions and help refine and improve the results.

Of course, understanding the impact of shifting office applications to the cloud is only part of the story, which is why last week we hosted a summit called “How Green is the Internet?” to explore these questions in greater detail. At the summit, experts presented data on how the growth of Internet infrastructure, including devices like phones and tablets, can impact the environment. We also saw great excitement about the potential for entirely new Internet-enabled tools in areas like transportation, e-commerce and digital content to deliver huge energy and carbon savings. We’ve posted the videos from those sessions and invite you to take a look.



One of our goals in hosting the summit and supporting the Berkeley Lab study was to identify and encourage new research on this topic. We’ll continue to work to answer some of these questions, and we hope others will too.

Investing in a South African solar project

As we search for investments that can help speed up the adoption of renewable energy, we’ve been looking beyond the U.S. and Europe to parts of the world where our investments can have an even greater impact. We’ve just closed our first investment in Africa: $12 million USD (103 million Rand) investment in the Jasper Power Project, a 96 megawatt solar photovoltaic plant in the Northern Cape province of South Africa. Upon completion, Jasper will be one of the largest solar installations on the continent, capable of generating enough electricity to power 30,000 South African homes. The project, developed and funded by SolarReserve, Intikon Energy and the Kensani Group, is also backed by Rand Merchant Bank, the Public Investment Corporation, Development Bank of South Africa and the PEACE Humansrus Trust.


View Jasper Power Project in a larger map
The Jasper Power Project is located in the Northern Cape Province of South Africa, near Postmasburg


When we consider investing in a renewable energy project, we focus on two key factors. First, we only pursue investments that we believe make financial sense. South Africa’s strong resources and supportive policies for renewable energy make it an attractive place to invest—which is why it had the highest growth in clean energy investment in the world last year. Second, we look for projects that have transformative potential—that is, projects that will bolster the growth of the renewable energy industry and move the world closer to a clean energy future. The Jasper Power Project is one of those transformative opportunities. To explain why, perhaps some background would be helpful.

Back in 2008, South Africa experienced a severe energy shortage, which resulted in blackouts throughout the country and slowed down economic growth. Since then the South African government has been actively supporting the growth of new sources of electricity to power the nation. While today South Africa is primarily dependent on fossil fuels, there’s lots of potential for renewable energy—it’s a country blessed with abundant wind and solar resources—and the government has set an ambitious goal of generating 18 gigawatts (GW) of renewable energy by 2030 (as a comparison, the entire South African grid is currently 44 GW).

To meet this goal, the South African government has established the Renewable Energy Independent Power Producer Procurement Program (REIPPPP). Through the program, renewable energy projects compete on the basis of cost and contribution to the local economy to be awarded a contract with Eskom, South Africa’s state-owned energy utility. Jasper and the other projects being developed through the REIPPPP have the potential to transform the South African energy grid. And given South Africa’s position as an economic powerhouse in Africa, a greener grid in South Africa can set an example for the whole continent.

Once constructed, the project will use solar panels like these.

Just as compelling are the economic and social benefits that the project will bring to the local community. Jasper will create approximately 300 construction and 50 permanent jobs in a region experiencing high rates of unemployment, as well as providing rural development and education programs and setting aside a portion of total project revenues—amounting to approximately $26 million over the life of the project—for enterprise and socio-economic development. We appreciate how forward-thinking the South African government has been in designing the REIPPPP to encourage these kinds of local economic benefits.

Google has committed more than $1 billion to renewable energy investments and we continue to search for new opportunities. Our search has brought us from the U.S. to Europe and now to Africa. We’re excited to see where else it might lead.

Expanding options for companies to buy renewable energy

We’re always looking for ways to expand the use of renewable energy. To date we’ve committed more than $1 billion to renewable energy project investments, signed agreements to procure wind power near our data centers, and installed solar panels at our corporate headquarters.

It’s also important to work directly with our utility partners to find solutions that will make more renewable energy available for us and for others. The most straightforward way to do this is for utilities to offer a renewable power option for companies that request it—something that’s not currently offered by most utilities. We’ve just published a white paper (PDF) laying out our thoughts on how and why such programs might work.

We’re also announcing our first effort to put this idea into practice. We’re expanding our Lenoir, N.C. data center, and our local electricity provider, Duke Energy, has pledged to develop a new program for large companies like Google who want to buy renewable power for their operations. Duke will file the plan with their state commission within 90 days.

Our Lenoir, N.C. data center

Offering companies like Google a renewable energy option has many advantages. Because the service is made available to a wide range of customers, companies that don’t have the ability or resources to pursue alternative approaches can participate. And by tapping utilities’ strengths in power generation and delivery, it makes it easier for companies to buy renewable energy on a larger scale. Of course, the approach is not without its challenges: utilities will need to work out the mechanics of the service within their local regulatory structure, and in many cases state utility commissions will need to approve the programs. There’s also the challenge of finding cost-effective renewable projects.

We'll continue to find creative ways to supply our facilities with renewable energy, but we think this solution can provide an important new way to increase the use of renewable energy nationwide. We look forward to working with utilities, state utility commissions, companies and other stakeholders to make it a reality.

Energy efficiency in the cloud

We’re obsessed with building energy efficient data centers that enable cloud computing. Besides helping you be more productive, cloud-based services like Google Apps can reduce energy use, lower carbon emissions and save you money in the process. Last year, we crunched the numbers and found that Gmail is up to 80 times more energy-efficient (PDF) than running traditional in-house email. We’ve sharpened our pencils again to see how Google Apps as a whole—documents, spreadsheets, email and other applications—stacks up against the standard model of locally hosted services. Our results show (PDF) that a typical organization can achieve energy savings of about 65-85 percent by migrating to Google Apps.

Lower energy use results in less carbon pollution and more energy saved for organizations. That’s what happened at the U.S. General Services Administration (GSA), which recently switched its approximately 17,000 users to Google Apps for Government. We found that the GSA was able to reduce server energy consumption by nearly 90 percent and carbon emissions by 85 percent. That means the GSA will save an estimated $285,000 annually on energy costs alone, a 93 percent cost reduction.

How is the cloud so energy efficient? It’s all about reducing energy use for servers and server cooling. Here’s how it works:


A typical organization has a lot more servers than it needs—for backup, failures and spikes in demand for computing. Cloud-based service providers like Google aggregate demand across thousands of people, substantially increasing how much servers are utilized. And our data centers use equipment and software specially designed to minimize energy use. The cloud can do the same work much more efficiently than locally hosted servers.

In fact, according to a study by the Carbon Disclosure Project, by migrating to the cloud, companies with over $1 billion in revenues in the U.S. and Europe could achieve substantial reductions in energy costs and carbon emissions by 2020:

  • U.S. companies could save $12.3 billion and up to 85.7 million metric tonnes of CO2
  • U.K. companies (PDF) would save £1.2 billion and more than 9.2 million metric tonnes of CO2
  • French companies (PDF) could save nearly €700 million and 1.2 million metric tonnes of CO2

We’ve built efficient data centers all over the world, even designing them in ways that make the best use of the natural environment, and we continue working to improve their performance. We think using the super-efficient cloud to deliver services like Google Apps can be part of the solution towards a more energy efficient future.

Posted by Urs Hoelzle, Senior Vice President for Technical Infrastructure

(Cross-posted on the Google Green Blog)