NETL and West Virginia University researchers have successfully used reinforcement learning — which allows a computer program to learn without user input — to develop adaptive control strategies that could reduce environmental emission and treatment costs during flexible operation of the nation’s power plants.

Reinforcement learning is a type of machine learning technique that involves an intelligent agent, such as a computer algorithm, taking action in an environment and receiving rewards or penalties based on its actions.

NETL researcher Steve Zitney explained, using gaming as a metaphor: “People get better at games, whether its video games, card games or even board games, because they earn points, or equivalent rewards, for successful actions and penalties for unsuccessful actions. Over time, a player learns how to maximize rewards and avoid penalties to win the game.”

In this reinforcement learning research, “winning the game” means finding the optimal strategy for controlling nitrogen oxide (NOx) emissions in power plants. This is achieved by tuning the optimal injection rate of ammonia into a power plant’s flue gas so that it can be reacted in a selective catalytic reduction (SCR) unit, which converts NOx into nitrogen and water vapor.

Plant operators want to inject the right amount of ammonia and avoid ammonia slip, which is when the costly chemical passes through the SCR unit unreacted due to an excess injection or too low of temperatures. These problems are especially complex because the SCR process is characterized by significant nonlinearities, time delays, and temperature sensitivities.

“Using reinforcement learning, we created algorithms that demonstrated success in each of these control aspects under flexible plant operations,” Zitney said. “Our first algorithm showed significant improvement in tracking the NOx setpoint when compared to standard controls. And the second algorithm showed good performance with respect to ammonia slip.”

Zitney also noted that reinforcement learning has shown advantages over other machine learning approaches because there is no need to feed the algorithm vast data sets to capture the knowledge. In addition, the study demonstrated the promise of reinforcement learning for systems where it is difficult to obtain optimal controller tuning parameters in general without significant trial-and-error that may not be acceptable to plant personnel.

This collaborative work, which was funded and conducted as part of the Transformative Power Generation Program within the U.S. Department of Energy’s Office of Fossil Energy and Carbon Management through NETL, was recently highlighted in the article, “Reinforcement learning for online adaptation of model predictive controllers: Application to a selective catalytic reduction unit,” published in Computers and Chemical Engineering.

This work also aligns with the goals of NETL’s Science-based Artificial Intelligence and Machine Learning Institute (SAMI), which combines the strengths of the Lab’s energy computational scientists, data scientists, and subject matter experts with strategic academic and industry partners to drive solutions to today’s energy challenges. For more information on SAMI, visit the home page or contact the Institute directly. Subscribe to SAMI’s bi-weekly Ai4AE (Artificial Intelligence for Applied Energy) Update and stay apprised of AI/ML news and advancements across NETL and DOE FECM, government, academia and industry.

NETL is a U.S. Department of Energy national laboratory that drives innovation and delivers technological solutions for an environmentally sustainable and prosperous energy future. By leveraging its world-class talent and research facilities, NETL is ensuring affordable, abundant and reliable energy that drives a robust economy and national security, while developing technologies to manage carbon across the full life cycle, enabling environmental sustainability for all Americans.