FunWake: a benchmark for LLM-discovered optimization algorithms

Wakes turn layout into a high-stakes optimization problem

Downstream turbines sit in the slowed, turbulent air of their neighbors. Layouts are optimized to squeeze wake losses out of a fixed patch of sea, and small modeled gains move real money.

Shid-Moosavi et al., “Modeling and experimentally-driven sensitivity analysis of wake-induced power loss in offshore wind farms,” Renewable Energy (2025).

Train, validate, and test on different farms

• Objective: maximize energy subject to feasible spacing and boundary constraints.
• The agent iterates on the training farm: it submits a script and gets back power and feasibility.
• After the loop, each script's AEP is computed on a held-out validation farm; the best one is deployed.
• The deployed script is scored on test farms it has never seen.

Three farms, three turbine platforms, four wind roses, deliberately mismatched so schedules must generalize.

The agent doesn't write the optimizer; it writes the schedules

Layouts are optimized by stochastic gradient descent on the sample-mean gradient of power P and constraint penalties γ. The agent's only levers are the hyperparameter schedules.

Four schedules to design: the learning rate lr(t), the constraint-penalty weight α(t), and the Adam momentum terms β₁(t) and β₂(t). Everything else is fixed.

Quick et al., “Stochastic gradient descent for wind farm optimization,” Wind Energy Science (2023), the baseline this harness wraps. Background on the SGD algorithm →

Each iteration, the agent reads, reviews, writes

1. Reads the task specification.
2. Reviews the current top-N scripts and their scores.
3. Writes a new candidate script against the API standard.
4. Gets it evaluated on the training farm.

Claude Code and Gemini CLI both climb past the 500-multistart baseline (dashed) on train and held-out validation AEP. Stars mark the deployed scripts.

The agents found cool new schedules

Claude Code · iter 192

A U-shaped penalty: relax constraints mid-run, tighten at the end

Gemini CLI · iter 118

Cyclic warm restarts across every hyperparameter at once

Neither schedule family exists in the TopFarm baseline (dashed black).

Here they are, optimizing

Winning-seed trajectories over the rose-averaged flow field on the DEI training farm (N=80). Darker = wake; brighter = faster average inflow. Final AEP: baseline 8714 GWh · Gemini 8805 GWh · Claude 8821 GWh.

Claude's schedule generalizes across farms and roses

500-multistart comparison across 3 farms × 4 wind roses × turbine counts. The Claude schedule is generally better than or on par with Gemini and the baseline, including on farms and turbines it never trained on.

Narrow the task, and the evaluation stays clean