Reducing AI carbon emissions: 17 concrete things AI developers can use

A summary of an article by Kasper Groes Albin Ludvigsen.

🌴 Plan

(1) Start estimating the carbon emissions - As the saying goes, measure what you treasure. The article explains how.

(2) Set an energy consumption limit.

🌴Start simple

(3) Start by developing simpler codes (which require less energy). Only write complex code if you truly need it.

(4) If you use neural networks, start by using few layers (less energy). Sometimes it works just as well and it will consume less energy.

(5) Stop training early if you notice it's underperforming.

(6) Debug on small subsets of your data first.

(7) Use pre-trained language models whenever possible.

🌴Carefully choose when and where you'll run your code

(8) If you're using a cloud service such as Google Cloud or AWS and you can choose where your computation will run, choose energy efficient regions (the article contains a list). This reduce emissions by up to 30x.

(9) Choose to run your code during "cleaner energy" times. Carbon intensity may vary throughout the day and some times of day are cleaner than others (the article contains data for Denmark)

(10) Use efficient data centers to run your code. For example, Cloud data centers can be 1.4-2x more energy efficient than typical centers, and ML-specific hardware can be 2-5x more efficient than non-specialized hardware.

(11) Consider turning your service off during selected times. Research shows that the energy consumption of a language model may be high even when it's not actively processing requests.

🌴Adjust your code

(12) Set up your hyper-parameter optimization function to optimize both accuracy and energy consumption (the article refers to a paper that does this).

(13) Design your code to run faster by choosing your activation function carefully. (The article refers to a paper with details)

(14) Distill large model into smaller models. For example, you can train smaller models to imitate larger ones, then use the smaller ones instead.

(15) Avoid trying to solve problems by throwing more compute at them. While it will probably make the code run more quickly, research has shown that it doesn't reduce energy consumption.

(16) When tuning your parameters, don't use the grid search method. It is inefficient. Instead you can use approaches such as random search and Bayesian optimization. They perform better in addition to consuming less energy.

(17) Use sparsely activated neural networks. They can yield the same performance with lower energy consumption.