A novel optimization algorithm for large-scale stochastic systems with deterministic constraints
Wind farm layout optimization requires estimating the Annual Energy Production (AEP) by propagating probability distributions of atmospheric conditions—wind speed and direction—through engineering wake models. The standard approach uses rectangular quadrature, dividing speeds and directions into bins (typically 8,000+ combinations) and evaluating the power at each one. This becomes computationally prohibitive as wind farms grow beyond a few hundred turbines, and the cost explodes when considering additional uncertain parameters like turbulence intensity, air density, or wind shear.
Instead of evaluating all atmospheric conditions at every iteration, our SGD approach randomly samples from the defined distributions and estimates the AEP gradient using Monte Carlo simulation. This allows for continuous resolution of uncertain variables—eliminating the need for discretization entirely—and scales efficiently regardless of how many uncertain parameters are considered.
The wind farm optimization problem has a stochastic objective (AEP depends on uncertain wind conditions) but deterministic constraints (turbine spacing and site boundaries are fixed). This is unusual in machine learning but common in mechanical engineering. We extended the Adam optimizer with a penalty-based constraint handling method that:
We benchmarked SGD against the industry-standard SLSQP algorithm across wind farms with 100 to 324 turbines. The results were striking: as farm size increased, SLSQP computation time grew dramatically (due to the quadratic scaling of spacing constraints), while SGD time remained nearly constant. For 324-turbine farms, SGD completed in 12 hours compared to 19 hours for SLSQP—and found layouts with 0.3–0.5% higher AEP.
The animation below demonstrates SGD optimizing a massive wind farm cluster—a scale that would be computationally infeasible with traditional deterministic methods. Each colored dot represents a wind turbine, and the background gradient shows the average annual wind speed across the site. The different colors indicate turbines belonging to different sub-farms within the cluster.
Watch as the turbines reorganize over 2,000 iterations, migrating toward higher-wind regions while respecting minimum spacing constraints and site boundaries. The stochastic nature of the algorithm allows turbines to escape local minima that would trap deterministic optimizers, ultimately finding layouts that better exploit the spatial variation in the wind resource.
Turbine positions evolving over SGD iterations. By sampling time as an uncertain variable rather than evaluating every wind resource entry, the algorithm achieves massive computational savings while converging to high-quality layouts.
The SGD optimizer is available in the open-source TOPFARM package, which interfaces with the PyWake wake modeling tool. Power gradients are computed via automatic differentiation, and the algorithm includes sensible defaults for learning rate scheduling, constraint multipliers, and early stopping criteria.
While demonstrated here for wind farm layout optimization, this algorithm addresses a general class of problems: stochastic objectives with deterministic constraints. This formulation appears throughout mechanical engineering—from structural optimization under uncertain loads to control system design under sensor noise. The constraint-handling approach developed in this work has potential applications far beyond the wind energy domain.