Research
Learning and computation for adaptive decision systems
Many decisions are made repeatedly from slightly different information, as when a robot replans its motion dozens of times a second or a power grid rebalances its generation every few minutes. Such decisions can be recomputed from scratch each time, which is reliable but expensive. They can also be predicted by a learned model, which is fast but may generalize poorly and violate the limits of the system. My research studies how learned representations and explicit computation can be combined so that decisions exploit what is shared across instances while retaining the ability to adapt to each one.
Repeated computation
Learning solution maps
A decision procedure that is run again and again, with data that change only partially each time, repeats much of its earlier computation. Because the procedure is fixed, each decision is a function of the inputs \(p\) it receives. A learned map can capture that function once instead of recomputing the decision on every call. The best action in each instance is the one of least cost among those the system allows, so the map to be learned is
Here \(f\) is the cost to be minimized and \(\mathbf y^\star(p)\) is the solution returned by a fixed procedure. The equalities \(c\) encode relations that must hold exactly, such as the dynamics of a physical system, and the inequalities \(h\) express limits, for example safety margins or bounds on actions. Learning an approximation of \(p\mapsto\mathbf y^\star(p)\) amortizes the cost of repeated solves, but it transfers the burden of generalization from a procedure that is correct by construction to a representation fitted on samples. Because constrained solution maps can be smooth within locally stable active-set regimes but change behavior across their boundaries, the parameter space may split into combinatorially many critical regions, making finite-sample generalization and reliable constraint behavior central challenges. My research explores how structure from the underlying problem can be brought into the learning of these maps to meet both.
Learning Solution Maps Learned Model Correction LearningToOptimize Differentiating Optimization Models (with DiffOpt.jl)
Sequential context
Policies for multistage decisions
Decisions are often coupled over time. Energy used now is unavailable later, and a foothold chosen now constrains the steps that follow. When each decision changes the state from which the next one is made, a policy generates the data on which it is judged. A decision that is optimal in isolation can then be expensive downstream. A policy is therefore a solution map evaluated along the trajectory it creates. At each stage it weighs the cost of acting now against the value of the state it leaves for later:
Here \(t\in\{1,\ldots,T\}\) indexes the stages, \(V_t\) is the optimal cost from stage \(t\) onward, \(\mathbf x_t\) is the state passed to the next stage, such as stored energy or the configuration of a robot, \(\mathbf y_t\) is the recourse, the detailed actions that realize the transition, and \(w_{[t]}\) is the observed history through the current realization \(w_t\). The transition relation \(\mathcal T=\mathbf 0\) and operating constraints \(h\ge\mathbf 0\) define the admissible recourse; \(q_t\) is its least cost, with \(q_t=+\infty\) for an infeasible transition, and \(\mathcal V_{t+1}\) is the continuation value under the uncertainty law \(\mathbb P\), the counterpart of a value function in reinforcement learning.
The representation of a policy determines what is prescribed in advance, what is computed after new information arrives, and how future consequences enter the current decision. Because each decision is judged along a trajectory, its errors compound over time, and a cost incurred late must be credited to the choices made early. Training such a policy becomes expensive and unstable as the horizon grows. My research explores how reformulating the problem can expose separability across time, preserve the structure of the system in every decision, and make reliable policy learning compatible with parallel computation.
Policies for Multistage Decisions Continuation-Value Representations Planning and Realization Mismatch DecisionRules.jl HydroPowerModels.jl
Coupled response
Multi-level decision problems
The outcome of a decision often depends on how another process responds to it, be it an algorithm or an agent with goals of its own. A bidding strategy, for example, submits an offer, and a market, weighing it against the offers of competitors, determines how much of it is accepted and at what price. The market solves a separate problem, and the strategy is judged only by the result. Training such a strategy is therefore a bilevel or sometimes a multi-level problem. In its simplest, bilevel form, the strategy’s decision \(p\), here the offer, is chosen to minimize an outer criterion \(L\), evaluated at the solution that the inner problem returns for it:
Here \(\mathcal Y(p)\) is the set of decisions the inner problem allows, and \(L\) is the loss by which \(p\) is ultimately judged. The outer decision \(p\), set by a learned component, may be an offer to a market, the targets of a controller inside a policy, or the coefficients of a simplified model whose solution is implemented. When the outer criterion is the inner objective itself, the gradient of \(L\) with respect to \(p\) follows from the inner solution and its Lagrange multipliers without differentiating either. When they differ, the gradient also requires the derivative of the inner solution with respect to \(p\). The relevant learning signal is therefore determined by the response of the composite decision process, not by any learned component in isolation.
Learning the inner solution’s response to \(p\) from historical data as a generic function ignores the objective and constraints that determine it, which can make generalization to new values of \(p\) very hard. Moreover, that response is smooth only within a fixed set of active constraints, and the inner solution can jump when that set changes. My research explores how to compute the inner solution’s derivatives from models as they are written, nonconvex ones included, and when they inform the choice of \(p\). It also asks how learning can proceed where they are uninformative, for example through a prior over the outer criterion or by averaging that criterion over contexts.
Endogenous Response Differentiating Optimization Models (with DiffOpt.jl) Learned Model Correction OptimalBids.jl
Estimated conditional laws
Uncertainty and ambiguity
Distribution models estimated from data, such as scenario weights over past stock returns, or policies learned from visited trajectories, such as a controller tuned on visited paths, are judged on conditions that their training record can only partially identify. Selection against an estimate of the governing probability law tends to pick decisions whose apparent performance is inflated by estimation error. This is the selection bias by which hyperparameter search overfits its validation set or financial strategies can easily underperform out-of-sample. It grows with the capacity of the class searched, which makes it most severe for flexible learned policies. Instead of making decisions against an estimated law, one can limit the selection bias by hedging against possible outcomes without necessarily assigning a probability to each of them. This is the premise of robust optimization:
Stochastic programming (SO)
Robust optimization (RO)
Distributionally robust optimization (DRO)
Here \(u\) is the decision, \(\xi\) the uncertain outcome, \(\ell\) the loss, \(\widehat{\mathbb P}\) the estimated law, \(\mathcal U\) an uncertainty set of possible outcomes, and \(\mathcal A\) an ambiguity set of plausible laws. The estimated law and both sets may depend on the information available when \(u\) is chosen. The first problem is empirical risk minimization when \(\widehat{\mathbb P}\) is the empirical law, the second is robust optimization, and the third is distributionally robust optimization. The set \(\mathcal U\) bounds each realization of \(\xi\), while \(\mathcal A\) bounds how far the law itself can depart from the estimate. Ambiguity is therefore a property of the conditional law used to construct a policy, not merely a perturbation of its eventual outcome.
While robust approaches implicitly define the risk aversion of the decision maker besides the inherent ambiguity of finite samples, RO and DRO sets are often tuned on data, inheriting at least part of the selection bias. Moreover, the conditional law can shift over time, so a set chosen in one period may become too conservative, or not conservative enough, later on. My research explores how to size ambiguity sets by the quality of the decisions they induce and how to condition them on the information available at the moment of choice.