Bibliography
Work on choosing what to compute and learn next, and on making the result of that choice believable.
arXiv identifiers were resolved against the arXiv API and journal DOIs against Crossref in September and October 2026. Venues are given where the arXiv record states them.
The core
- Russell, S., and Wefald, E. (1991). Principles of Metareasoning. Artificial Intelligence 49(1–3), 361–395. Computations are actions valued by their effect on later actions. The rational agent performs the computation of highest net value and acts when none remains positive.
- Hay, N., Russell, S., Tolpin, D., and Shimony, S. E. (2012). Selecting Computations: Theory and Applications. arXiv:1207.5879; UAI 2012. Metalevel control as a Bayesian selection problem rather than a bandit problem, since only the final choice collects reward. Finite sampling bounds in some cases, and a counterexample to the conjecture that an optimal policy always reaches a decision.
- Callaway, F., Gul, S., Krueger, P. M., Griffiths, T. L., and Lieder, F. (2017). Learning to select computations. arXiv:1711.06892. Bayesian metalevel policy search. The value of information lies between the myopic value and the value of perfect information, and that bracket makes a learned metalevel policy tractable.
- Turtel, B., Wilczewski, P., Franklin, D., and Skothiem, K. (2026). Future-as-Label: Scalable Supervision from Real-World Outcomes. arXiv:2601.06336. Resolved outcomes as reward. The predictor sees causally masked information, a resolver establishes the outcome, and a proper scoring rule scores the forecast. A 27% Brier improvement for Qwen3-32B. Offline, binary outcomes only. Deployment-time loops are not evaluated.
- Li, Y., Yang, X., Yang, X., Xu, M., Wang, X., et al. (2025). R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization. arXiv:2505.15155; NeurIPS 2025. A full research loop over factors and models: research, development and feedback stages, with a multi-armed bandit choosing the next direction. Up to twice the annualised return of classical factor libraries with 70% fewer factors. Its limitations section names alternative data, domain knowledge and online adaptation as open.
- Gençay, E. (2026). What survives honest evaluation? Leakage-safe, search-aware assessment of LLM-driven trading strategy discovery. arXiv:2608.27734. Leakage-safe tools by construction and deflation by the agent’s own trial count. Rejects every LLM-discovered strategy tested, and shows a leaky oracle with Sharpe 35 passing both the deflated Sharpe ratio and the backtest-overfitting test.
- Russell, S. J. and Subramanian, D. (1995). Provably Bounded-Optimal Agents. arXiv:cs/9505103; Journal of Artificial Intelligence Research 2, 575–609. Replaces perfect rationality with bounded optimality: an agent’s program should be the best one its architecture allows for its task environment. Constructs bounded-optimal agents for a class of real-time environments, illustrated with a mail-sorting model, and gives universal programs that are asymptotically bounded-optimal under any real-time constraints.
Budgeted races
- Cotton, P. (2026). Wavefront Pruning in Budgeted Brownian Races. Working draft; software at microprediction/brownianbandit. A controller pays per unit time for every Brownian path kept alive and prunes to maximise the expected terminal maximum. A path is kept while its future pivotal value exceeds its carrying cost. The value of computation for parallel rollouts and agent fleets.
- Cotton, P. (2026). Posterior-Predictive Pass-at-k. Note. Pass@k is the expected-maximum payoff of a best-of-k batch. A posterior expectation replaces the downward-biased plug-in and cuts held-out log loss from 3.37 to 0.48.
- Cotton, P. (2026). When the Grass Is Greener: Three-Shot Search on Exponentiated Gaussian Landscapes. microprediction/browniansearch. The search-side sibling: where to spend a few evaluations of one path, paid at the final point.
- Cotton, P. (2026). Clay Shooting: Equilibrium Effort Solving Outstanding Problems. Working draft. The strategic sibling. A Brownian attempt buys success probability at a convex price, and the record of solved problems implies what an attempt is worth to whoever makes it.
Pricing information and computation
- Howard, R. A. (1966). Information Value Theory. IEEE Transactions on Systems Science and Cybernetics 2(1), 22–26. The value of information as the gain in expected utility from observing before deciding.
- Matheson, J. E. (1968). The Economic Value of Analysis and Computation. IEEE Transactions on Systems Science and Cybernetics 4(3), 325–332. Extends the value of information from observations to analysis. The earliest statement of the value of computation.
- Gittins, J. C. (1979). Bandit Processes and Dynamic Allocation Indices. Journal of the Royal Statistical Society, Series B 41(2), 148–164. Index policies for allocating effort among alternatives, optimal under the bandit-process assumptions.
- Horvitz, E. J. (2013). Reasoning About Beliefs and Actions Under Computational Resource Constraints. arXiv:1304.2759; UAI 1987. Decision-theoretic control of inference under resource limits: weigh the costs and benefits of approximation procedures, and of acting on a partial result.
- Hansen, E. A., and Zilberstein, S. (1996). Monitoring Anytime Algorithms. ACM SIGART Bulletin 7(2), 28–33. When to stop an algorithm that improves with time and can be interrupted.
- Tolpin, D. and Shimony, S. E. (2012). MCTS Based on Simple Regret. arXiv:1207.5536; AAAI 2012. UCT minimises cumulative regret, but search needs simple regret. Optimising the sampling is itself a metareasoning problem.
- Sezener, E. and Dayan, P. (2020). Static and Dynamic Values of Computation in MCTS. arXiv:2002.04335; UAI 2020. Values Monte Carlo simulations by their expected effect on the action eventually chosen, beyond the myopic horizon.
- Weitzman, M. L. (1979). Optimal Search for the Best Alternative. Econometrica 47(3). The Pandora’s box problem: boxes are opened at a cost, each reveals a random reward, and the searcher keeps the best reward found. Weitzman’s rule is the optimal strategy when the reward distributions are known.
- Luby, M., Sinclair, A., and Zuckerman, D. (1993). Optimal Speedup of Las Vegas Algorithms. Information Processing Letters 47(4), 173–180. A Las Vegas algorithm is always correct when it stops but has random running time, and is rerun with fresh randomness under a sequence of cutoffs. With the runtime distribution known, the best strategy repeats one fixed cutoff. Without it, a simple universal sequence has expected time O(ℓ. Log ℓ), where ℓ. Is the optimal expected time, and no universal strategy does better by more than a constant factor.
- Zilberstein, S. (1995). Optimizing Decision Quality with Contract Algorithms. IJCAI-95, Vol. 2, pp. 1576–1581. Meta-level control of a composition of contract algorithms, whose run time must be fixed before they start. Each component is described by a probabilistic conditional performance profile that maps input quality and time to a distribution of output quality, and time is allocated by an offline compilation that simplifies run-time monitoring.
- Poh, K.-L. and Horvitz, E. J. (1996). A Graph-Theoretic Analysis of Information Value. arXiv:1302.3596; UAI 1996. Dominance relations for the expected value of information on chance variables, read from their position in an influence diagram. These give non-numerical procedures for ordering uncertain variables by informational relevance.
- Chen, C.-H., Lin, J., Yücesan, E., and Chick, S. E. (2000). Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization. Discrete Event Dynamic Systems 10(3), 251–270. Allocates simulation replications across designs to select the best at lower total cost. More efficient than the compared procedures, including a two-stage procedure, with a speedup factor above 20 beyond that of ordinal optimisation itself on a 210-design example.
- Hansen, E. A., and Zilberstein, S. (2001). Monitoring and Control of Anytime Algorithms: A Dynamic Programming Approach. Artificial Intelligence 126(1–2), 139–157. When to stop an anytime algorithm and act, posed as a dynamic program. Handles uncertainty in the performance profile, uncertainty in the environment and the cost of monitoring progress, giving a non-myopic stopping policy.
- Horvitz, E. (2001). Principles and Applications of Continual Computation. Artificial Intelligence 126(1–2), 159–196. Continual computation: spending idle time on problems that may arrive later, not only on those already posed. Policies for allocating idle time in several settings, with applications.
- Kautz, H., Horvitz, E., Ruan, Y., Gomes, C., and Selman, B. (2002). Dynamic Restart Policies. AAAI 2002. Restart policies for randomized search that update beliefs about the runtime distribution from real-time observations of the instance and the solver, using learned Bayesian predictive models. On fresh test data, dynamic policies cut solution times by about 40% to 65% relative to the fixed optimal restart policy of Luby et al.
- Finkelstein, L., Markovitch, S., and Rivlin, E. (2003). Optimal Schedules for Parallelizing Anytime Algorithms: The Case of Shared Resources. arXiv:1106.5269; Journal of Artificial Intelligence Research 19, 73–138. Several anytime algorithm–problem pairs run on one shared processor until the first succeeds. An algorithm for the optimal schedule given the runtime distributions, analysed theoretically and empirically for several distribution types and tested on the Latin Square problem.
- Glynn, P. and Juneja, S. (2004). A Large Deviations Perspective on Ordinal Optimization. Proceedings of the 2004 Winter Simulation Conference. Asymptotically optimal allocation of a sampling budget for selecting the population with the best mean, for general rather than Gaussian distributions. The large deviations rate function turns the allocation into a concave program, and at the optimum the probabilities of falsely selecting each competitor are equal on a logarithmic scale.
- Giles, M. B. (2008). Multilevel Monte Carlo Path Simulation. Operations Research 56(3), 607–617. Multigrid ideas applied to Monte Carlo estimation of an expectation from a stochastic differential equation: cheap coarse paths and costly fine paths are combined in a telescoping sum. For a Lipschitz payoff with Euler discretisation, the cost to reach accuracy &epsilon. Falls from O(ε−3) to O(ε−2(log ε)2).
- Xu, L., Hutter, F., Hoos, H. H., and Leyton-Brown, K. (2008). SATzilla: Portfolio-based Algorithm Selection for SAT. Journal of Artificial Intelligence Research 32, 565–606. Chooses a SAT solver per instance: empirical hardness models predict each solver’s performance and the portfolio runs the predicted best. SATzilla07 won three gold, one silver and one bronze medal in the 2007 SAT Competition.
- Krause, A. and Guestrin, C. (2009). Optimal Value of Information in Graphical Models. arXiv:1401.3474; Journal of Artificial Intelligence Research 35, 557–591. Efficient optimal algorithms for selecting observations, and conditional observation plans, in chain graphical models such as hidden Markov models. On the other side, optimising value of information is NPPP-hard even for polytrees, and computing decision-theoretic value of information is #P-complete even for naive Bayes models.
- Shahaf, D., and Horvitz, E. (2009). Investigations of Continual Computation. IJCAI 2009. Extends continual computation to shared subtasks and multiple tasks, with complexity results and approximations for arbitrary models of computational performance. Also treats uncertainty about which algorithm to apply, learning about performance, and the cost of delayed use of results.
- Burns, E., Ruml, W., and Do, M. B. (2013). Heuristic Search When Time Matters. Journal of Artificial Intelligence Research 47, 697–740. Heuristic search under a utility that trades search time linearly against solution cost. Bugsy puts the utility function into the search itself instead of relying on a separately trained termination policy for an anytime algorithm. Offline parameter tuning performed best when representative training instances were available, and Bugsy did well without any training.
- Tolpin, D., Beja, T., Shimony, S. E., Felner, A., and Karpas, E. (2013). Towards Rational Deployment of Multiple Heuristics in A*. arXiv:1305.5030; IJCAI 2013. Lazy A* computes an expensive admissible heuristic only when it is needed for a decision, and rational lazy A* decides whether to compute it at all from a myopic value-of-information estimate. Both are analysed and evaluated on several search domains.
- López-Ortiz, A., Angelopoulos, S., and Hamel, A. M. (2014). Optimal Scheduling of Contract Algorithms for Anytime Problem-Solving. Journal of Artificial Intelligence Research 51, 533–554. Running a contract algorithm repeatedly with growing budgets makes it interruptible, at a cost measured by the acceleration ratio. Matching upper and lower bounds on that ratio for several problem instances on several parallel processors, resolving an open conjecture.
- Giles, M. B. (2015). Multilevel Monte Carlo Methods. Acta Numerica 24, 259–328; arXiv:1304.5472. A review of multilevel Monte Carlo since the 2008 path-simulation paper, stressing the simplicity, flexibility and generality of the approach, with some original ideas and open directions.
- Lin, C. H., Kolobov, A., Kamar, E., and Horvitz, E. (2015). Metareasoning for Planning Under Uncertainty. arXiv:1505.00399; IJCAI 2015, pp. 1601–1609. Online planning in an MDP when the state keeps changing while the agent thinks. General metareasoning is shown to be at most polynomially harder than solving the MDP, but optimal metareasoning is impractical, so the paper gives approximate procedures built on properties of the BRTDP planner.
- Agarwal, A., Luo, H., Neyshabur, B., and Schapire, R. E. (2017). Corralling a Band of Bandit Algorithms. arXiv:1612.06246; COLT 2017. A master algorithm over several bandit algorithms that aims to do almost as well as the best of them would do if run alone. Base algorithms inside a master receive less feedback, so the master must not starve one that starts poorly but would improve. A version of online mirror descent with a special mirror map and learning-rate schedule gives better regret bounds than earlier combiners.
- Karpas, E., Betzalel, O., Shimony, S. E., Tolpin, D., and Felner, A. (2018). Rational Deployment of Multiple Heuristics in Optimal State-Space Search. Artificial Intelligence 256, 181–210. The journal extension, adding lazy and rational lazy IDA* and methods for estimating the parameters the decision rule needs. The rational variants beat their non-rational lazy counterparts in several domains.
- Singla, S. (2018). The Price of Information in Combinatorial Optimization. arXiv:1711.00405; SODA 2018, 2523–2532. Extends Weitzman’s Pandora’s box, where values are revealed only by paying a price, to combinatorial problems such as spanning tree, matching, set cover, facility location and prize-collecting Steiner tree. A reduction to the versions of these problems without prices gives exact or approximation algorithms for expected utility or disutility.
- Binois, M., Huang, J., Gramacy, R. B., and Ludkovski, M. (2019). Replication or Exploration? Sequential Design for Stochastic Simulation Experiments. arXiv:1710.03206; Technometrics 61(1), 7–23. Shows that replication helps Gaussian process emulation of noisy simulators, both statistically and computationally. A lookahead sequential design decides whether the next run repeats an existing input or explores a new one and, paired with a heteroskedastic Gaussian process, learns how signal and noise vary across the input space. Demonstrated on inventory management and epidemiology simulations.
- Gupta, A., Jiang, H., Scully, Z., and Singla, S. (2019). The Markovian Price of Information. arXiv:1902.07856; IPCO 2019, LNCS 11480, 233–246. Each input of a combinatorial problem is the destination of a Markov chain that costs a price per step to advance, and rewards can be collected only for a subset meeting a constraint. Gives optimal and approximation algorithms that trade the value of the combinatorial solution against the total price paid, joining Markovian multi-armed bandits with discrete optimization, and treats robustness to the distribution parameters and commitment constraints.
- Jackson, C., Presanis, A., Conti, S., and De Angelis, D. (2019). Value of Information: Sensitivity Analysis and Research Design in Bayesian Evidence Synthesis. arXiv:1703.08994; Journal of the American Statistical Association 114(528), 1436–1449. Value-of-information analysis for a Bayesian model combining several evidence sources: which parameter uncertainties drive decision uncertainty, and the expected reduction in loss from collecting data of a given design. Applied to estimating HIV prevalence from surveys, registers and expert beliefs.
- Kleinberg, R., Leyton-Brown, K., Lucier, B., and Graham, D. (2019). Procrastinating with Confidence: Near-Optimal, Anytime, Adaptive Algorithm Configuration. arXiv:1902.05454; NeurIPS 2019. Algorithm configuration with runtime caps. Structured Procrastination with Confidence keeps the near-optimality and anytime guarantees of Structured Procrastination while adapting to the problem, so it runs much faster when many configurations are poor. Experiments show such settings are common and that the anytime property helps find good configurations quickly.
- Shperberg, S. S., Coles, A., Karpas, E., Shimony, S. E., and Ruml, W. (2020). Trading Plan Cost for Timeliness in Situated Temporal Planning. IJCAI 2020, 4176–4182. Extends deliberation scheduling for situated temporal planning, where a plan can become infeasible while the agent is still planning, to plans whose costs differ and are uncertain until found. Minimising expected cost is PSPACE-hard and, even with known costs and deadlines, the optimal schedule is contingent rather than sequential. A greedy scheme built on special cases performs well, including on problems generated from planner search trees.
- Heath, A., Strong, M., Glynn, D., Kunst, N., Welton, N. J., and Goldhaber-Fiebert, J. D. (2022). Simulating Study Data to Support Expected Value of Sample Information Calculations: A Tutorial. Medical Decision Making 42(2), 143–155. A step-by-step guide to simulating data from proposed studies for expected value of sample information calculations, including correlated outcomes, missingness, censoring and reuse of individual patient data, with R code.
- Angelopoulos, S., and Kamali, S. (2023). Contract Scheduling with Predictions. arXiv:2011.12439; Journal of Artificial Intelligence Research 77, 395–426. Contract scheduling with a possibly erroneous prediction of the interruption time, given either directly or as answers to binary queries. Tradeoffs between robustness to adversarial predictions and consistency when the prediction is correct.
- Elboher, A., Bensoussan, A., Karpas, E., Ruml, W., Shperberg, S. S., and Shimony, S. E. (2023). A Formal Metareasoning Model of Concurrent Planning and Execution. arXiv:2303.02664; AAAI 2023. Formalises concurrent planning and execution as an abstract metareasoning problem: acting before search concludes buys search time but risks wrong choices, especially when actions are irreversible. The general problem is intractable, some special cases are solvable in polynomial time, and greedy algorithms perform well on instances derived from search problems.
- Angelopoulos, S., Bienkowski, M., Dürr, C., and Simon, B. (2024). Contract Scheduling with Distributional and Multiple Advice. arXiv:2404.12485; IJCAI 2024, 3652–3660. Contract scheduling when the prediction of the interruption time is a probability distribution or a set of possible times. Schedules that are optimal when the prediction is accurate while keeping the best worst-case performance when it is adversarial.
- Coles, A., Karpas, E., Lavrinenko, A., Ruml, W., Shimony, S. E., and Shperberg, S. S. (2024). Planning and Acting While the Clock Ticks. arXiv:2403.14796; ICAPS 2024. Introduces concurrent planning and execution, where actions may be dispatched before planning terminates and wall-clock deadlines affect action applicability and goal achievement. Extends metareasoning for situated temporal planning to this setting, and under strong time pressure the approach outperforms situated temporal planning.
- Jaimungal, S. and Shi, X. (2024). The Price of Information. arXiv:2402.11864; SIAM Journal on Financial Mathematics 15(3), SC54–SC67. The indifference price an expected-utility investor would pay for partial information about an asset’s trajectory, comparing the partial-information control problem without the purchase to the problem after it. When the information can be bought at any stopping time before the horizon, the investor buys at a deterministic time.
- Bansal, N., Huang, Z., and Zhu, Z. (2025). Optimal 4-Approximation for the Correlated Pandora’s Problem. arXiv:2509.17029; FOCS 2025. Pandora’s box with correlated box values, a problem that also generalizes min-sum set cover. Gives a 4-approximation, matching the lower bound of 4 inherited from min-sum set cover.
- Cao, Z., Wang, H., Chew, E. P., Li, H., and Tan, K. C. (2025). A Budget-Adaptive Allocation Rule for Optimal Computing Budget Allocation. arXiv:2304.02377; European Journal of Operational Research 325(2), 247–260. The asymptotic OCBA rule follows from assuming a very large budget and so ignores the budget’s size. A budget-adaptive rule in which the share of budget given to each design depends on the total budget, with two heuristic algorithms built on it.
- Coles, A., Karpas, E., Shimony, S. E., Shperberg, S. S., and Ruml, W. (2025). Concurrent Planning and Execution Using Dispatch-Dependent Values. IJCAI 2025, 8483–8490. Earlier methods for concurrent planning and execution used a value function that assumes search finishes before any action is executed. A new value function accounts for dispatching actions incrementally, yields a simpler algorithm, and under time pressure significantly outperforms the previous state of the art.
- Li, Z., Wan, Y., and Hong, L. J. (2025). Sequential Additivity in Distributionally Robust Ranking and Selection. arXiv:2509.06147. Ranking and selection under input uncertainty picks the alternative with the best worst-case mean over several input distributions, giving a multiplicative number of scenarios. Any consistent procedure must sample an additive number of scenarios infinitely often, and a simple additive allocation attains this bound with exponentially decaying probability of incorrect selection. The scenarios sampled infinitely often need not be the true worst cases.
- Margraf, V., Koerner, T., Tornede, A., and Wever, M. (2025). RunAndSchedule2Survive: Algorithm Scheduling Based on Run2Survive. ACM Transactions on Evolutionary Learning and Optimization 5(3). Algorithm scheduling from runtime distributions modelled as survival functions, with an evolutionary algorithm searching over schedules. Best performance in 20 of 25 benchmark scenarios against earlier approaches.
- Nadimpalli, S., Qiao, M., and Rubinfeld, R. (2025). No Price Tags? No Problem: Query Strategies for Unpriced Information. arXiv:2511.06170. In the priced query model a known function is computed on an unknown input whose variables are revealed only at a cost, and earlier work assumes the costs are known in advance. Lets the costs be unknown, proves that this imposes an unavoidable overhead on every query strategy, and gives strategies that essentially match the lower bound and are competitive with the best cost-aware strategies for arbitrary Boolean functions.
- Wang, Y. and Zhou, E. (2025). Data-Driven Ranking and Selection With Simultaneous Input Data Collection and Simulation. arXiv:2503.11773; Naval Research Logistics 72(8), 1170–1183. Ranking and selection when input data arrive in streams that can be collected while simulations run. Simulation outputs under changing input distributions are pooled into one performance estimator, and a multi-stage procedure divides the budget between collecting input data and simulating, with consistency and asymptotic normality guarantees and numerical studies.
- Labiosa, A., and Hanna, J. P. (2026). When to Plan: Learning to Select Between Reactive Control and Deliberative Planning. arXiv:2607.16421; RLC 2026. Trains with reinforcement learning a meta-reasoning policy that chooses between a reactive policy and slower decision-time planning, conditioning on an uncertainty score for the reactive policy. In motion-planning and navigation environments it learns when the reactive action is good enough, and shifts toward fully reactive control as the reactive policy improves.
- Labiosa, A., and Hanna, J. P. (2026). Learning to Act While Thinking. RL in Big Worlds Workshop, RLC 2026. Formulates the Real-Time Meta-Reasoning MDP, in which an agent may request a plan at any time but the plan is costly and arrives after a delay while the environment keeps changing. A curriculum-based reinforcement learning method learns when to invoke the planner, how to act while waiting and how to follow the plan, and in two partially observable dynamic tasks it reaches a higher success rate with fewer plans than fixed meta-reasoning and heuristic baselines.
- Lyu, Z., Nejma, C., Wegel, T., Yang, F., and Dorner, F. E. (2026). Cutting LLM Evaluation Costs with SySRs: A Bandit Algorithm That Provably Exploits Model Similarity. arXiv:2606.07726; ICML 2026, PMLR 306. Successive Rejects run on paired comparisons: every surviving model is scored on the same new queries and one is eliminated per phase. Hyperparameter-free, with error guarantees that improve as the evaluated models become more similar. Lowest average error rate among the compared methods across 15 standard LLM benchmarks.
- Tuisov, A. (2026). Search as Computation Allocation. arXiv:2607.27871. Treats search as allocating costly computations that matter only through terminal decision loss, with Bellman equations for fixed budgets, priced computation and certification. Mutual information equals myopic value of computation under log loss, but under simple regret information gain can rank computations arbitrarily poorly, and weighted A* is recovered under an explicit heuristic-error model. Theory with illustrative instantiations, and no experiments.
- Xue, B., Cheng, J., Lyu, S.-H., Wan, Y., and Qiu, S. (2026). Test-Time Scaling via Budgeted Multi-Attribute Verification. arXiv:2609.34322. Verification of LLM answers under a shared budget, posed as multi-attribute good-arm identification: each candidate is scored on several costly attributes and is certified only if every attribute mean clears its threshold. BMA-GAI chooses which candidate and which attribute to sample next, certifies with anytime-valid bounds, and has a matching information-theoretic converse, so it is first-order optimal away from critical budget levels.
Resource-rational metareasoning
- Lieder, F., and Griffiths, T. L. (2017). Strategy Selection as Rational Metareasoning. Psychological Review 124(6), 762–794. Human strategy choice modelled as learned metareasoning.
- Lieder, F., Shenhav, A., Musslick, S., and Griffiths, T. L. (2018). Rational Metareasoning and the Plasticity of Cognitive Control. PLOS Computational Biology 14(4), e1006043. Cognitive control adapted by learning which computations are worth their cost.
- Lieder, F., and Griffiths, T. L. (2020). Resource-Rational Analysis: Understanding Human Cognition as the Optimal Use of Limited Computational Resources. Behavioral and Brain Sciences 43. The research programme, with open peer commentary.
- Milli, S., Lieder, F., and Griffiths, T. L. (2017). When Does Bounded-Optimal Metareasoning Favor Few Cognitive Systems?. Proceedings of the AAAI Conference on Artificial Intelligence 31(1). Restricts metareasoning to a choice among a few cognitive systems that do different amounts of computation, and asks how many a bounded-optimal agent should have. In a two-alternative decision and in planning in a Markov decision process, the best number depends on the variability of the environment and the cost of metareasoning. When two systems are optimal, one is fast and error-prone and the other slow and accurate.
- Sezener, C. E., Dezfouli, A., and Keramati, M. (2019). Optimizing the depth and the direction of prospective planning using information values. PLOS Computational Biology 15(3), e1006827. In plan-until-habit, a search tree is simulated to limited depth and habitual values stand in for what lies beyond. Chooses the direction and depth of expansion by a speed/accuracy tradeoff based on information values, shown in a grid world. The model accounts for the effect of time pressure on planning depth, of reward magnitude on planning direction, and the shift from goal-directed to habitual behavior with training.
- Adriaensen, S., Biedenkapp, A., Shala, G., et al. (2022). Automated Dynamic Algorithm Configuration. arXiv:2205.13881; Journal of Artificial Intelligence Research 75, 1633–1699. Learning policies that adjust an algorithm’s parameters while it runs, rather than fixing a configuration in advance. Formalises dynamic algorithm configuration as a computational problem, surveys the methods used for it, and reports case studies in evolutionary optimisation, AI planning and machine learning.
- Godara, P., and Alemán, T. D. (2024). Metareasoning in uncertain environments: a meta-BAMDP framework. arXiv:2408.01253. Extends models of metareasoning to environments with unknown rewards and transitions by embedding the reasoning problem in a Bayes-adaptive MDP. Applied to Bernoulli bandit tasks, with two theorems that make approximate solutions tractable and give testable predictions about human exploration under cognitive limits.
- Russek, E. M., Acosta-Kane, D., van Opheusden, B., Mattar, M. G., and Griffiths, T. L. (2025). Time Spent Thinking in Online Chess Reflects the Value of Computation. Cognitive Science 49(10), e70119. Compares thinking time in over 12 million online chess games with the value of computation in each position. Players think longer where more computation is more beneficial, the relation is stronger in stronger players, and a model built on the measured cost of time captures its qualitative features.
Information acquisition
- Frazier, P. I., Powell, W. B., and Dayanik, S. (2008). A Knowledge-Gradient Policy for Sequential Information Collection. SIAM Journal on Control and Optimization 47(5), 2410–2439. The one-step expected improvement in the value of the best alternative.
- Russo, D. and Van Roy, B. (2014). Learning to Optimize via Information-Directed Sampling. arXiv:1403.5556. Acts to minimise squared expected regret per unit of information about the optimal action. A regret bound that scales with the entropy of the optimal action.
- Frazier, P. I. (2018). A Tutorial on Bayesian Optimization. arXiv:1807.02811. Expected improvement, entropy search and the knowledge gradient, with multi-fidelity and multi-source variants.
- Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., and Talwalkar, A. (2016). Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. arXiv:1603.06560. Adaptive resource allocation with early stopping over random configurations, reported an order of magnitude faster than Bayesian optimisation on its benchmarks. The non-Bayesian counterpoint.
- Li, Y. and Oliva, J. B. (2020). Active Feature Acquisition with Generative Surrogate Models. arXiv:2010.02433. Which missing features to pay for at prediction time, with a generative surrogate estimating the information gain of each.
- Wan, B., Liu, M., Grigas, P., and Shen, Z.-J. M. (2026). Decision-Focused Sequential Experimental Design: A Directional Uncertainty-Guided Approach. arXiv:2602.05340. Sequential experimental design aimed at decision loss rather than prediction error. Stops earlier than decision-blind designs.
- Maron, O., and Moore, A. W. (1994). Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation. Advances in Neural Information Processing Systems 6 (NIPS 1993). Model selection by cross-validation that discards models as soon as Hoeffding bounds show them inferior and concentrates evaluation on the better ones. The paper studies leave-one-out cross-validation for memory-based learners and argues the method applies to model selection generally.
- Heckerman, D., Breese, J. S., and Rommelse, K. (1995). Decision-Theoretic Troubleshooting. Communications of the ACM 38(3), 49–57. Troubleshooting a failed device with a Bayesian network over its components, choosing observations and repairs by expected cost. Under a single-fault assumption with fixed costs, the optimal sequence inspects components in decreasing order of fault probability to observation cost.
- Breese, J. S., and Heckerman, D. (1996). Decision-Theoretic Troubleshooting: A Framework for Repair and Experiment. arXiv:1302.3563; UAI 1996. Extends troubleshooting from observations to repairs and reconfigurations of the device, each followed by observation of its behaviour. Reasoning about actions that change the system needs persistence networks, Bayesian networks that encode what stays the same across an intervention.
- Birattari, M., Stützle, T., Paquete, L., and Varrentrapp, K. (2002). A Racing Algorithm for Configuring Metaheuristics. GECCO 2002, 11–18. F-Race: runs every surviving configuration of a metaheuristic on the same problem instance and drops those a Friedman test finds worse. Demonstrated on an ant colony optimisation algorithm for the travelling salesman problem.
- Krause, A., McMahan, H. B., Guestrin, C., and Gupta, A. (2008). Robust Submodular Observation Selection. Journal of Machine Learning Research 9(93), 2761–2801. Choosing observations that perform well against the worst of several submodular objectives. The Submodular Saturation algorithm carries approximation guarantees, and the paper proves that better approximations are impossible unless NP-complete problems admit efficient algorithms. Extensions handle non-unit observation costs and trade expected-case against worst-case performance.
- Mnih, V., Szepesvári, C., and Audibert, J.-Y. (2008). Empirical Bernstein Stopping. ICML 2008, 672–679. Adaptive stopping rules for sampling with probabilistic accuracy guarantees, built on empirical Bernstein bounds so that the number of samples adapts to the observed variance. Upper bounds on sample complexity, and experiments on model selection and on boosting in the filtering setting.
- Frazier, P., Powell, W., and Dayanik, S. (2009). The Knowledge-Gradient Policy for Correlated Normal Beliefs. INFORMS Journal on Computing 21(4), 599–613. Bayesian ranking and selection with independent normal sampling noise and a correlated multivariate normal belief on the alternatives&rsquo. Means, so one measurement updates beliefs about many alternatives. The knowledge-gradient policy is provably optimal in some special cases and has bounded suboptimality in all others, and it extends to maximising a continuous function with a fixed number of noisy measurements.
- Golovin, D. and Krause, A. (2010). Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization. arXiv:1003.3967. Adaptive submodularity extends submodular set functions to adaptive policies, and under it a greedy adaptive policy is competitive with the optimal policy for stochastic maximisation and coverage. Version 5 addresses a flaw in the proof of Theorem 13 identified by Nan and Saligrama (2017) and gives a weaker squared-logarithmic bound under an additional strong adaptive submodularity condition.
- Houlsby, N., Huszár, F., Ghahramani, Z., and Lengyel, M. (2011). Bayesian Active Learning for Classification and Preference Learning. arXiv:1112.5745. Information-theoretic active learning with the information gain about parameters written in terms of predictive entropies (BALD), applied to Gaussian-process classification with minimal approximation and extended to preference learning.
- Naghshvar, M., and Javidi, T. (2013). Active Sequential Hypothesis Testing. arXiv:1203.4626; Annals of Statistics 41(6), 2703–2738. Lower bounds on the optimal total cost of adaptively choosing sensing actions before declaring a hypothesis, and two heuristic policies. The first is asymptotically optimal in Chernoff’s sense as the penalty for a wrong declaration grows.
- Javdani, S., Chen, Y., Karbasi, A., Krause, A., Bagnell, J. A., and Srinivasa, S. (2014). Near Optimal Bayesian Active Learning for Decision Making. arXiv:1402.5886; AISTATS 2014. Tests are chosen to drive the posterior into a single decision region, rather than to reduce uncertainty about the hypothesis itself. Necessary and sufficient conditions for identifying such a region, and a Hyperedge Cutting algorithm proved competitive with the intractable optimal policy.
- Swersky, K., Snoek, J., and Adams, R. P. (2014). Freeze-Thaw Bayesian Optimization. arXiv:1406.3896. Bayesian optimisation that uses partially observed training curves to decide whether to pause a model, resume one, or start a new one. A covariance kernel for training curves, a Gaussian process prior that scales with additional temporal observations, and an information-theoretic rule for the choice.
- Chen, Y., Javdani, S., Karbasi, A., Bagnell, J. A., Srinivasa, S., and Krause, A. (2015). Submodular Surrogates for Value of Information. AAAI 2015. Optimising decision-theoretic value of information is intractable, and greedy approximations of it are known to perform poorly. DiRECt replaces it with a surrogate that is aligned with value of information, cheap to evaluate and adaptive submodular, so that greedy selection carries an approximation guarantee. Case studies in robotic localisation, preference learning, conservation management and behavioural economics.
- Hernández-Lobato, J. M., Gelbart, M. A., Adams, R. P., Hoffman, M. W., and Ghahramani, Z. (2016). A General Framework for Constrained Bayesian Optimization using Information-based Search. arXiv:1511.09422; Journal of Machine Learning Research 17(160), 1–53. Predictive entropy search with constraints (PESC), an approximation to expected information gain that separates into the contributions of the objective and of each black-box constraint, so that each can be evaluated on its own. For mixes of fast and slow functions, a partial update trades the accuracy of the acquisition computation against its own run time, and an adaptive rule switches between partial and full updates.
- Jamieson, K., and Talwalkar, A. (2016). Non-stochastic Best Arm Identification and Hyperparameter Optimization. arXiv:1502.07943; AISTATS 2016. Hyperparameter optimisation cast as best-arm identification without stochastic assumptions, using successive halving to give more resource to promising settings. Reported comparable test accuracy an order of magnitude faster than baseline methods.
- Chen, Y., Hassani, S. H., and Krause, A. (2017). Near-optimal Bayesian Active Learning with Correlated and Noisy Tests. arXiv:1605.07334; AISTATS 2017; Electronic Journal of Statistics 11(2), 4969–5017. When test outcomes are conditionally dependent given the hidden target, greedily maximising uncertainty reduction can perform poorly. ECED instead greedily maximises a surrogate objective that accounts for dependence between tests, and adaptive submodularity gives a near-optimality guarantee.
- Chen, Y., Renders, J.-M., Chehreghani, M. H., and Krause, A. (2017). Efficient Online Learning for Optimizing Value of Information: Theory and Application to Interactive Troubleshooting. arXiv:1703.05452; UAI 2017. Choosing low-cost tests that let the best decision be made. Dynamic hypothesis enumeration gives guarantees when the hypothesis distribution is known, and posterior sampling learns its parameters online when it is not, with a bound on expected regret. Applied to interactive troubleshooting.
- Kandasamy, K., Dasarathy, G., Schneider, J., and Póczos, B. (2017). Multi-fidelity Bayesian Optimisation with Continuous Approximations. arXiv:1703.06240; ICML 2017. Multi-fidelity Bayesian optimisation over a continuous space of cheap approximations, such as training on less data or for fewer iterations, rather than a finite set of fidelities. The method, BOCA, has better regret than strategies that ignore the approximations.
- Bogunovic, I., Zhao, J., and Cevher, V. (2018). Robust Maximization of Non-Submodular Objectives. arXiv:1802.07073; AISTATS 2018, PMLR 84:890–899. Maximizing a monotone set function under a cardinality constraint when an adversary deletes some of the chosen elements. Oblivious-Greedy gives the first constant-factor guarantees for a class of non-submodular objectives, including when the number of deletions grows linearly with the set size, with bounds stated through the submodularity ratio and an inverse curvature.
- Kanagawa, M., and Hennig, P. (2019). Convergence Guarantees for Adaptive Bayesian Quadrature Methods. NeurIPS 2019. Consistency and convergence rates for a broad class of adaptive Bayesian quadrature methods, whose acquisition functions depend on earlier integrand values, via a condition the authors call weak adaptivity. The authors describe the rates as non-tight but informative.
- Gessner, A., González, J., and Mahsereci, M. (2020). Active Multi-Information Source Bayesian Quadrature. arXiv:1903.11331; UAI 2019, PMLR 115, 712–721. Bayesian quadrature with several related information sources of different cost, such as cheaper approximations of an expensive simulation. Cost-sensitive acquisition functions choose both the point and the source, and vanilla Bayesian quadrature is the one-source, constant-cost case. On an epidemiological model the method allocates budget more efficiently than vanilla Bayesian quadrature.
- Astudillo, R., and Frazier, P. I. (2021). Bayesian Optimization of Function Networks. arXiv:2112.15311; NeurIPS 2021. Bayesian optimization of the output of a network of functions, modelling each node with a Gaussian process and using expected improvement under the implied posterior, maximized by sample average approximation. The method is asymptotically consistent without evaluating the domain densely and outperforms standard Bayesian optimization on synthetic and real problems.
- Astudillo, R., Jiang, D. R., Balandat, M., Bakshy, E., and Frazier, P. I. (2021). Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs. arXiv:2111.06537; NeurIPS 2021. Bayesian optimisation under a total budget when the cost of each evaluation is unknown until it is paid. Expected improvement, and expected improvement per unit cost, can be arbitrarily worse than the optimal non-myopic policy. The paper proposes a budgeted multi-step expected improvement in their place.
- Foster, A., Ivanova, D. R., Malik, I., and Rainforth, T. (2021). Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design. arXiv:2103.02438; ICML 2021, PMLR 139:3384–3395. Trains a design network before the experiment, so that each adaptive design at deployment is a single forward pass. Contrastive information bounds serve as the training objective for the sequential setting.
- Hellerstein, L., Kletenik, D., and Parthasarathy, S. (2021). A Tight Bound for Stochastic Submodular Cover. arXiv:2102.01149; Journal of Artificial Intelligence Research 71, 347–370. The adaptive greedy algorithm of Golovin and Krause achieves a (ln(Q/η) + 1) approximation for stochastic submodular cover, where Q is the goal value and &eta. The smallest nonzero marginal gain. This restores the bound Golovin and Krause originally claimed, whose first proof was shown to be incorrect.
- Ivanova, D. R., Foster, A., Kleinegesse, S., Gutmann, M. U., and Rainforth, T. (2021). Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods. arXiv:2111.02329; NeurIPS 2021. Extends amortised, policy-based experimental design to implicit models that can simulate differentiable samples, removing the need for a closed-form likelihood or conditionally independent experiments.
- Neiswanger, W., Wang, K. A., and Ermon, S. (2021). Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information. arXiv:2104.09460; ICML 2021, PMLR 139:8005–8015. Infers the output of an algorithm that would need far more function evaluations than the budget allows, by choosing queries that maximise mutual information with that output (InfoBAX). Up to 500 times fewer queries than running the algorithm itself, on shortest paths and local optimisation.
- Ding, L., Hong, L. J., Shen, H., and Zhang, X. (2022). Technical Note—Knowledge Gradient for Selection with Covariates: Consistency and Computation. arXiv:1906.05098; Naval Research Logistics 69(3), 496–507. Ranking and selection when the best alternative depends on covariates. Proves that the knowledge-gradient sampling policy is consistent under minimal assumptions, so that the best alternative as a function of the covariates is identified almost surely as samples grow, and gives a stochastic gradient ascent algorithm for computing the policy.
- Sloman, S. J., Oppenheimer, D. M., Broomell, S. B., and Shalizi, C. R. (2022). Characterizing the robustness of Bayesian adaptive experimental designs to active learning bias. arXiv:2205.13698. Bayesian adaptive experimental design can suffer active learning bias, with unrepresentative sampling giving inconsistent estimates, depending on model misspecification. For a linear model, worse misspecification means more bias, and models that assume more observation noise suffer less.
- Aushev, A., Putkonen, A., Clarté, G., et al. (2023). Online Simulator-Based Experimental Design for Cognitive Model Selection. arXiv:2303.02227; Computational Brain & Behavior 6(4), 719–737. Sequential experimental design for choosing between simulator models whose likelihoods are intractable, using a simulator-based design utility and an approximate model likelihood. In simulated cognitive-science tasks it selects models accurately in up to two orders of magnitude less time than likelihood-free alternatives.
- Bickford Smith, F., Kirsch, A., Farquhar, S., Gal, Y., Foster, A., and Rainforth, T. (2023). Prediction-Oriented Bayesian Active Learning. arXiv:2304.08151; AISTATS 2023. BALD seeks information about parameters and has no notion of an input distribution, so it can favour irrelevant data. Expected predictive information gain (EPIG) measures information about predictions at target inputs instead and gives better predictive performance than BALD across a range of datasets and models.
- Daulton, S., Balandat, M., and Bakshy, E. (2023). Hypervolume Knowledge Gradient: A Lookahead Approach for Multi-Objective Bayesian Optimization with Partial Information. ICML 2023, PMLR 202:7167–7204. A one-step lookahead acquisition based on the knowledge gradient that decides what to evaluate, when and where, when objectives can be queried separately at different costs or through cheaper lower-fidelity proxies. It is by construction the one-step Bayes-optimal policy for hypervolume maximization and improves sample efficiency on synthetic and real problems.
- Saad, E. M., Blanchard, G., and Verzelen, N. (2023). Covariance Adaptive Best Arm Identification. arXiv:2306.02630; NeurIPS 2023. Fixed-confidence best-arm identification when arms are sampled together and may be correlated. Algorithms that adapt to the unknown covariance improve on the independent-arm setting, with new lower bounds for this setting.
- Verma, R., Barrejón, D., and Nalisnick, E. (2023). Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles. arXiv:2210.16955; AISTATS 2023. Consistent surrogate losses, softmax and one-vs-all, for a classifier that may defer to one of several experts, and estimates of each expert’s probability of being correct. In theory mis-calibration propagates between experts under softmax but not one-vs-all, though in practice there are trade-offs. A conformal method chooses which subset of experts to query, tested on galaxy, skin-lesion and hate-speech classification.
- Xu, R., Zariphopoulou, T., and Zhang, L. (2023). Decision Making under Costly Sequential Information Acquisition: the Paradigm of Reversible and Irreversible Decisions. arXiv:2401.00569. A decision maker chooses between a product of known value and a new product of unknown value while observing signals about the new one, and may exchange the new product for the old after choosing it. This gives sequential optimal stopping problems with a coarser signal before the purchase and a more accurate one after, analyzed with viscosity solution methods as nested obstacle problems.
- Buathong, P., Wan, J., Astudillo, R., Daulton, S., Balandat, M., and Frazier, P. I. (2024). Bayesian Optimization of Function Networks with Partial Evaluations. arXiv:2311.02146; ICML 2024. Bayesian optimization of an objective computed by a network of functions whose nodes can be evaluated individually at different costs. A knowledge-gradient acquisition chooses which node and which inputs to evaluate, so each step may run only part of the network, and it outperforms existing methods on synthetic and real problems.
- Cheng, C. X., Astudillo, R., Desautels, T., and Yue, Y. (2024). Practical Bayesian Algorithm Execution via Posterior Sampling. arXiv:2410.20596; NeurIPS 2024. Replaces the information-gain criterion of Bayesian algorithm execution with posterior sampling when the property of interest is a set of points defined by the function. Competitive with existing methods at much lower computational cost, with conditions for asymptotic convergence.
- Harris, B. and Nagarajan, V. (2024). Lower Bound on the Greedy Approximation Ratio for Adaptive Submodular Cover. arXiv:2405.14995. An instance with Q = 1 on which the greedy algorithm for adaptive submodular cover has approximation ratio at least 1.3(1 + ln Q), which invalidates the (1 + ln Q)2 ratio claimed by Golovin and Krause.
- Huang, D., Guo, Y., Acerbi, L., and Kaski, S. (2024). Amortized Bayesian Experimental Design for Decision-Making. arXiv:2411.02064; NeurIPS 2024. Experimental design aimed at the downstream decision rather than at information about model parameters. A transformer (TNDP) proposes the next design and infers the decision within one amortised model.
- Poiani, R., Degenne, R., Kaufmann, E., Metelli, A. M., and Restelli, M. (2024). Optimal Multi-Fidelity Best-Arm Identification. arXiv:2406.03033; NeurIPS 2024. Best-arm identification when each arm can be sampled at a cheaper, less accurate fidelity. A tight instance-dependent lower bound on the cost of identifying the best arm, and a gradient-based algorithm whose cost complexity is asymptotically optimal. The analysis yields an optimal fidelity for each arm.
- Barlas, Y. Z., Sloman, S. J., and Kaski, S. (2025). Robust Experimental Design via Generalised Bayesian Inference. arXiv:2511.07671. Expected information gain depends on a correctly specified model. Replaces the likelihood with a loss function, in the manner of Gibbs inference, and derives a Gibbs expected information gain for choosing designs. More robust to outliers and to wrong assumptions about the noise distribution in its experiments.
- Buathong, P., and Frazier, P. I. (2025). Fast Bayesian Optimization of Function Networks with Partial Evaluations. arXiv:2506.11456. Cuts the computational overhead of partial-evaluation knowledge gradient by generating node-specific candidate inputs from one global Monte Carlo simulation, with a modest loss in query efficiency and up to a 16×. Speedup over the original algorithm.
- Hedman, M., Ivanova, D. R., Guan, C., and Rainforth, T. (2025). Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design. arXiv:2507.14057; ICML 2025, PMLR 267:22904–22923. A design policy trained before the experiment is periodically refined on the data gathered so far, rather than held fixed. Reported to improve decisions and robustness over fully amortised methods.
- Lee, D. B., Zhang, A. S., Kim, B., et al. (2025). Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning. arXiv:2510.21379; NeurIPS 2025. Freeze-thaw hyperparameter optimisation with a utility that trades cost against performance, estimated from user preference data. The acquisition function chooses which paused configuration to continue, and a stopping criterion ends the whole search near maximum utility.
- Song, E., and Kim, T. (2025). Optimizing Input Data Collection for Ranking and Selection. arXiv:2502.16659. Ranking and selection where all solutions share Bayesian input models updated from several data sources, and a single budget buys either input data or simulation runs. The paper characterises near-optimal static ratios for the two, proposes OSAR to reach them in the limit, and extends it with kernel ridge regression to continuous input parameters.
- Vershinin, G., Cohen, A., and Gurewitz, O. (2025). Active Sequential Hypothesis Testing with Non-Homogeneous Costs. arXiv:2509.11632; ICASSP 2026. Active hypothesis testing when actions have different costs. Expected total cost factors into the mean number of samples times the mean per-action cost, so a policy should maximise expected information gain per expected cost rather than the average of per-step information per cost. In simulation an adapted Chernoff scheme cuts mean cost by up to 50% against the classic Chernoff policy.
- Xie, Q., Cai, L., Terenin, A., Frazier, P. I., and Scully, Z. (2025). Cost-aware Stopping for Bayesian Optimization. arXiv:2507.12453; ICML 2026. A stopping rule for Bayesian optimisation that weighs solution quality against cumulative evaluation cost, derived from the Pandora’s Box Gittins index and log expected improvement per cost. Paired with either acquisition function, the policy has a bound on expected cost-adjusted simple regret.
- Yang, L., Tan, V. Y. F., and Cheung, W. C. (2025). Best Arm Identification with Possibly Biased Offline Data. arXiv:2505.23165; UAI 2025, PMLR 286:4715–4730. Best-arm identification that can draw on offline data whose distribution may differ from the online arms. Without prior knowledge of a bias bound no adaptive algorithm can exploit the offline data. LUCB-H, given such a bound, matches LUCB when the offline data mislead and improves on it when they help.
- Al-Thani, H., Cui, Y., Harris, B., and Nagarajan, V. (2026). Minimum Cost Adaptive Submodular Cover. Mathematics of Operations Research; preprint arXiv:2208.08351. Covering an adaptive submodular function at minimum expected cost when item realisations may be correlated. The greedy policy’s approximation ratio is logarithmic in the goal value Q and best possible up to constant factors, and the guarantees extend to the p-th moment of the cover cost and to covering several functions at once.
- Chen, Z., Ament, S., Eriksson, D., et al. (2026). Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search. arXiv:2609.01431. A cost-aware multi-fidelity acquisition function that picks the training run which most reduces uncertainty in the fitted scaling law per unit compute, rather than optimising a single objective. It therefore favours informative small-scale runs. Reaches accurate hyperparameter scaling laws with under a tenth of the budget of grid search on synthetic, surrogate and real pre-training settings.
- Dinkel, M., Ana, D. C., Goderbauer, B., and Wall, W. A. (2026). Efficient Bayesian Optimal Experimental Design for Expensive Computational Models over Finite Design Sets. arXiv:2607.16933. Bayesian optimal experimental design over a finite set of designs when each model evaluation is expensive. Pairwise expected-information-gain comparisons use an accelerated nested Monte Carlo estimator with common random numbers and Rao–Blackwellization. Bootstrap estimates of the probability that one design beats another drive successive elimination, so further computation goes only to designs still in contention.
- Go, J., Qian, X., and Yoon, B.-J. (2026). Goal-driven Bayesian Optimal Experimental Design for Robust Decision-Making Under Model Uncertainty. arXiv:2605.26093. Experimental design aimed at a downstream decision rather than at information about parameters, using an amortized variational posterior and a differentiable convex decision layer. The design gradients are shown to be insensitive to parameter directions irrelevant to the decision. In source localization, epidemic management and pharmacokinetic control, near-optimal design windows are wider than goal-agnostic design suggests.
- Guo, Y., Huang, D., Zhang, X., Katt, S., Kaski, S., and Bharti, A. (2026). Constrained Bayesian Experimental Design via Online Planning. arXiv:2605.26990; ICML 2026. Sequential Bayesian experimental design under budgets, varying costs and constraints on how designs may change over time. Offline pre-training of an amortised policy and posterior network is combined with online multi-step lookahead over scenario trees, giving more informative design sequences than existing methods for a modest additional computational cost.
- Ham, D. W., Zhao, X., Jasin, S., and Yang, F. (2026). Human–AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation. arXiv:2609.28859. Runs a hypothesis test with prescribed type-I and type-II errors at minimum cost by scoring items with an AI judge and escalating selectively to human verification. An information-theoretic lower bound on cost, and a sequential policy, SCALE, that is valid in finite samples and matches the bound to first order as the target errors vanish.
- Harikumar, H., Katt, S., Barlas, Y. Z., and Kaski, S. (2026). Robust Bayesian Decision Making under Adversarial Uncertainty. arXiv:2607.08590; UAI 2026, PMLR 337:1983–2007. Sequential experimental design for a decision that must hold up against worst-case values of adversarial variables. The design criterion targets stability of the robust optimal decision rather than nominal optimality. On synthetic and scientific datasets, conventional decision-aware design converges quickly to confident but fragile decisions.
- Jha, A. K., Onuţu, D. A., Mallik, N., et al. (2026). Amortizing Scaling Law Construction Costs. arXiv:2609.05016. Fitting a scaling law needs only the best-loss frontier, so data collection is posed as Bayesian optimisation over the training grid. Expanding the compute budget progressively during acquisition, and filling the grid with surrogate-fantasised evaluations, matches fits from a full dense grid at reported savings of 10–100×. A five-page workshop paper.
- Kobayashi, Y., Jing, Z., Yao, J., Namkoong, H., and Joshi, S. (2026). Learning-To-Measure: In-Context Active Feature Acquisition. arXiv:2510.12624; ICML 2026. Active feature acquisition learned across tasks rather than per task. A sequence model pre-trained to give uncertainty estimates under arbitrary missingness drives a greedy agent that acquires the feature with the largest conditional mutual information, in context and without per-task retraining. Matches or beats task-specific baselines on tabular benchmarks, most clearly when labels are scarce and missingness is high.
- Ma, T., Qin, H., Zhu, R., and Zuo, J. (2026). Best-Arm Identification with Generative Proxy. arXiv:2607.06879. Best-arm identification where each costly reward comes with a cheap correlated proxy whose correlation must be learned online. A plug-in residual variance is anti-conservative, so PROBE keeps an upper certificate on it from an ordinary least squares fit, is δ-PAC, and attains the known-correlation sample complexity up to constants.
- Riegler, B., Odgers, J. A. C., and Fortuin, V. (2026). Standard Acquisition Is Sufficient for Asynchronous Bayesian Optimization. arXiv:2603.13501; AISTATS 2026. In asynchronous Bayesian optimization, standard acquisitions such as the upper confidence bound attain guarantees essentially equivalent to sequential Thompson sampling, because posterior updates from completed runs are enough to avoid redundant queries. Penalizing busy locations can over-explore, and standard acquisitions match or beat purpose-built asynchronous methods in experiments.
Metareasoning in language models
- Graves, A. (2016). Adaptive Computation Time for Recurrent Neural Networks. arXiv:1603.08983. A network learns how many internal steps to take. On text it spends more computation on harder transitions.
- Snell, C., Lee, J., Xu, K., and Kumar, A. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. arXiv:2408.03314. Compute-optimal test-time scaling depends on prompt difficulty. More than four times the efficiency of best-of-N.
- Chen, X., Xu, J., Liang, T., He, Z., Pang, J., et al. (2024). Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs. arXiv:2412.21187. Reasoning models spend heavily on easy problems for little benefit.
- De Sabbata, C. N., Sumers, T. R., AlKhamissi, B., Bosselut, A., and Griffiths, T. L. (2024). Rational Metareasoning for Large Language Models. arXiv:2410.05563. A reward built on the value of computation, trained by expert iteration. 20–37% fewer tokens at unchanged performance.
- Sui, Y., He, Y., Cao, T., Han, S., Chen, Y., and Hooi, B. (2025). Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models. arXiv:2502.19918; ACL 2026. A contextual bandit chooses between continuing, backtracking, switching strategy and restarting.
- Wang, Z., Tsang, I., and Qian, H. (2026). AERA: Adaptive Evidence Residual Allocation for Efficient Test-Time Reasoning. arXiv:2608.27964. Present confidence is a poor guide to whether more computation helps. Learns the future value of computation instead, cutting completion tokens by 95.99% for a 0.4-point accuracy cost on held-out GSM8K.
- Fang, H. and Wang, B. (2026). Exploit More, Explore Smarter for Budget-Constrained Agentic Search. arXiv:2608.23848. Tree expansion as a value-of-information decision under a small evaluation budget.
- Aggarwal, P., Madaan, A., Anand, A., et al. (2023). AutoMix: Automatically Mixing Language Models. arXiv:2310.12963; NeurIPS 2024. Routes a query to a larger model according to few-shot self-verification of a smaller model’s answer, with a POMDP router because self-verification is noisy. Over 50% lower computational cost at comparable performance across five language models and five datasets.
- Aggarwal, P., Madaan, A., Yang, Y., and Mausam (2023). Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs. arXiv:2305.11860; EMNLP 2023. Replaces the fixed sample count of self-consistency with a lightweight stopping rule based on agreement among the samples drawn so far. Across 17 reasoning and code-generation datasets and three LLMs it reduces the sample budget by up to 7.9 times with an average accuracy drop below 0.1%.
- Damani, M., Shenfeld, I., Peng, A., Bobu, A., and Andreas, J. (2024). Learning How Hard to Think: Input-Adaptive Allocation of LM Computation. arXiv:2410.04707; ICLR 2025. Predicts, for each input, the distribution of rewards as a function of the computation budget, and spends extra decoding computation where it is predicted to help most. Applied to adaptive best-of-k and to routing between a cheap and an expensive decoding procedure on programming, mathematics and dialog tasks, it cuts computation by up to 50% at unchanged quality or raises quality by up to 10% at a fixed budget.
- Manvi, R., Singh, A., and Ermon, S. (2024). Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation. arXiv:2410.02725. Trains the model to predict mid-generation, through a single predefined token, the probability that restarting would yield a better response. The prediction decides whether to draw more samples and which to prune early. 74% of the improvement from 16 samples is obtained with 1.2 samples on average, and 50–75% of samples can be pruned early with little loss.
- Stroebl, B., Kapoor, S., and Narayanan, A. (2024). The Limits of Inference Scaling Through Resampling. arXiv:2411.17501; ICLR 2026. Resampling until a candidate passes an imperfect verifier cannot lower the verifier’s false-positive rate, which caps accuracy whatever the compute budget. On HumanEval and MBPP, single-sample accuracy and false-positive rate are strongly correlated, so resampling a weaker model does not reach the single-sample accuracy of a sufficiently strong one. When false positives carry negative utility, the optimal number of attempts is often below 10.
- Chen, H. M., Lu, G., Okoshi, Y., Mo, Z., Motomura, M., and Fan, H. (2025). Rethinking Optimal Verification Granularity for Compute-Efficient Test-Time Scaling. arXiv:2505.11730; NeurIPS 2025. Varies how often the verifier is called during generation through a granularity parameter whose extremes are beam search and best-of-N. Adaptive choices of granularity gave accuracy gains of up to 3.1% over beam search and 3.6% over best-of-N while reducing FLOPs by more than 52%.
- Jung, D., Shi, P., Chen, M., and Zhang, Y. (2025). FutureWeaver: Planning Test-Time Compute for Multi-Agent Systems with Modularized Collaboration. arXiv:2512.11213; COLM 2026. Plans test-time compute for multi-agent language-model systems under a fixed budget. Reusable collaboration modules are induced by self-play reflection, and a two-level planner combines short-horizon action selection with abstract lookahead.
- Kalayci, Y., Raman, V., and Dughmi, S. (2025). Optimal Stopping vs Best-of-N for Inference Time Optimization. arXiv:2510.01394. Each generation is a Pandora’s box opened at a cost to reveal a random reward. A UCB-style rule that does not know the reward distribution comes provably close to Weitzman’s optimal rule, and with a Bradley–Terry style normalisation of rewards across prompts it matches Best-of-N with 15–35% fewer generations on AlpacaFarm and HH-RLHF.
- Liu, T., Wang, Z., Miao, J., et al. (2025). Budget-Aware Tool Use Enables Effective Agent Scaling. arXiv:2511.17006; COLM 2026. Web search agents under explicit tool-call budgets. Raising the budget alone hits a ceiling because agents lack budget awareness, and a Budget Tracker plug-in and the BATS framework, which adapts planning and verification to the remaining budget, give better cost–performance scaling under a unified token-and-tool cost metric.
- Singhi, N., Bansal, H., Hosseini, A., et al. (2025). When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning. arXiv:2504.01005; COLM 2025. Under a fixed inference budget, should compute go to more candidate solutions with majority voting or to generative verification of fewer solutions? Self-consistency is the more compute-efficient choice for most practical budgets in its experiments: generative verification first matches it only after up to 8×. The compute. Fitted scaling laws favour scaling the number of solutions faster than the number of verifications.
- Tong, J., Fan, Y., Zhao, A., Ma, Y., and Shen, X. (2025). StreamingThinker: Large Language Models Can Think While Reading. arXiv:2510.17238; ICLR 2026. Reasoning that proceeds in the order the input arrives and deepens once reading is complete, using streaming attention masks and parallel KV caches. On Qwen3 models it keeps accuracy comparable to batch thinking, with an 80% reduction in token waiting before reasoning starts and over 60% lower time to the final answer.
- Zuo, B. and Zhu, Y. (2025). Strategic Scaling of Test-Time Compute: A Bandit Learning Approach. arXiv:2506.12721; ICLR 2026. Test-time compute allocation posed as a bandit problem: query difficulty is estimated on the fly, more compute goes to hard queries, and among those to the solvable ones. Proved more compute-efficient than uniform allocation, with gains of up to about 11 points on MATH-500, AIME25 and LiveCodeBench.
- Bilal, A., Mohsin, M. A., Umer, M., et al. (2026). What If We Allocate Test-Time Compute Adaptively?. arXiv:2602.01070; ICML 2026. A verifier-guided framework that, in each iteration, picks reasoning tools, a compute strategy and an exploration parameter, with a process reward model guiding pruning and expansion within iterations and final selection across them. It reports gains over direct test-time scaling on MATH-500, AIME24 and AMO-Bench, with efficiency measured in theoretical FLOPs and a compute-intensity metric that penalises wasted generation and tool overhead.
- Chen, W.-L., Peng, L., Tan, T., et al. (2026). Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens. arXiv:2602.13517; ICML 2026. Measures reasoning effort by deep-thinking tokens, where the model’s internal predictions are revised substantially in deeper layers before settling. Their share correlates with accuracy better than length or confidence, and Think@n uses it on short prefixes to reject unpromising samples early, matching or exceeding self-consistency at lower cost.
- Davidov, H., Cohen, N., Kalinsky, O., et al. (2026). Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning. arXiv:2604.18419; ICML 2026. Mid-generation abstention treated as an action in regularised reinforcement learning, with an abstention reward that prices compute against information. Abstaining when the value function falls below that reward beats natural baselines under general conditions. Experiments on mathematical reasoning and toxicity avoidance.
- Dong, Z., Qin, F., and Shah, M. (2026). When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models. arXiv:2606.30852. Early-exit rules for reasoning models compared at a matched risk of losing correct answers, across 18 task-model settings. A learned stopper saves the most tokens on free-form math, calibrated scalar exits win on multiple choice, and small hard benchmarks admit no certifiable aggressive policy. The serving regime matters: the same policy saves 32% of tokens with KV-cache forking and costs 121% extra under repeated black-box prefilling.
- Dughmi, S., Haghifam, M., and Kalayci, Y. H. (2026). Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification. arXiv:2605.17609. A cheap reward score and an expensive verifier, formalised as a cost-sensitive search for the first verified positive. With the score distribution and success function known, dynamic programming gives the optimal policy. When both are unknown, ADAP grows the numbers of samples and of top-ranked verifications in shells and stays within a constant factor of the optimal expected cost, assuming higher scores are no less likely to pass. Lower bounds show that some such assumption on the score–label relationship is necessary.
- He, L., Wen, J., Wang, H., et al. (2026). Not All Errors Are Equal: Consequence-Aware Reasoning Compute Allocation. arXiv:2606.04402. Allocates premium test-time compute by the cost of failure rather than by difficulty alone. The formal priority combines consequence with the marginal gain from compute, but because marginal gain is hard to predict before execution the deployed scheduler routes on predicted consequence. On SWE-bench Lite under a fixed budget it gives the best success on high-consequence tasks with competitive overall accuracy.
- He, F., Wang, H., Meng, L., Gao, M., and Li, Q. (2026). Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation. arXiv:2610.05170; EMNLP 2026. When a code generator and a test-writing verifier share a false task premise, selection among sampled programs can pick one that fails hidden tests even when a correct program is in the pool. Across three benchmarks and five code models, false premises lower the correctness of the program selected from 64 samples, and a predictor using only verifier-visible features detects the failure with 0.846 AUROC. Evidence that does not depend on the premise recovers more than scaling the coupled verifier.
- Huang, J., Zhang, Y., Ma, W., Zhou, W., and Zhou, Z. (2026). Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback. arXiv:2609.38931. Self-consistency when each sampled trajectory reveals the model’s whole answer distribution, read from log-probabilities, rather than one answer. Stopping as soon as the modal answer is identified at a set confidence has an asymptotic rate never worse than the answer-only rate. The betting rule ASC-D uses 46.4–95.6% fewer trajectories than answer-only adaptive self-consistency on MMLU-Redux.
- Jha, A. (2026). Does Fault Localization Beat a Fresh Attempt? A Placebo-Controlled Study of Test-Guided Code Repair. arXiv:2609.00854. Applies three repairs to the same failing program: blind resampling of the whole solution, spectrum-based fault localization followed by infilling of the suspect span, and same-length infilling at a random span as a placebo. Localization was available for only 9.0% of 488 failing candidates, and where it was available localized infilling lost to blind resampling at a matched attempt count. Its lead over the random-span placebo is reported as suggestive rather than established, and conclusions are restricted to the 24–32B models tested.
- Jiang, Z., Ramos, G., Suh, J., et al. (2026). FrankenReport: Early Exiting in Long-Form Generation Using Expected Value of Computation. arXiv:2609.06320. Long-form report generation with early exit per section: intermediate drafts are used to predict whether further targeted computation will improve quality materially. In simulation it beats random allocation by up to four times at low budgets and recovers full-pipeline quality as the budget grows.
- Kassenaar, G., Yang, Z., and François-Lavet, V. (2026). Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation. arXiv:2608.20256. A reasoning model chooses with its first output token between no thinking, brief reasoning and extended reasoning, learned within GRPO through a shaped reward and per-mode token caps. On a 1.5B model, held-out MATH500 accuracy is 0.782 against the base model’s 0.796 while mean response length falls by 41%.
- Li, X., Wang, H., Yuan, G., et al. (2026). GraphAHA: Graph-Based Adaptive Search with Heterogeneous Actions for Test-Time Code Generation. arXiv:2609.12757. Test-time code generation as search on a typed graph in which equivalent programs are merged so their statistics are shared. Hierarchical Thompson sampling chooses among sampling, reasoning, implementation and repair. Best score in 18 of 20 cases on LiveCodeBench and CodeContests.
- Luan, Z., Zhang, X., Hu, M., Yang, Y., Yu, J., and Chen, X. (2026). Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systems. arXiv:2608.25920. Asks whether repair methods for multi-agent LLM systems fix failures causally or by resampling. SymTrace replays the recorded trajectory up to an intervention anchor and regenerates only what follows, and SymFail supplies 536 annotated failure trajectories. Unguided reruns reproduce 67.97% of failures and repair 6.90%, while a symptom-driven intervention repairs 20.15%.
- Ma, S., Gao, B., Xiao, Z., et al. (2026). CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning. arXiv:2603.28135. A training-free meta-controller over partial reasoning trajectories that decides when to expand, prune, repair, stop or fall back, guided by an online step-level process oracle. Reports gains over single-path, sampling and search baselines at matched inference budgets.
- Manvi, R., Hong, J., Seyde, T., Labonne, M., Lechner, M., and Levine, S. (2026). Zero-Overhead Introspection for Adaptive Test-Time Compute. arXiv:2512.01457; ICLR 2026. ZIP-RC reuses reserved or unused logits in the ordinary forward pass to output, at every token, a joint distribution over final reward and remaining length. Meta-actions choose which prefixes to continue or start sampling from, maximising a utility that combines expected maximum reward, compute and latency. On mixed-difficulty mathematics benchmarks accuracy improves by up to 12% over majority voting at equal or lower average cost.
- Papamarkou, T., Smirnov, V., Mazanov, V., et al. (2026). Bayesian Control for Coding Agents. arXiv:2606.24453. Coding-agent orchestration posed as cost-sensitive sequential hypothesis testing. A controller keeps a belief that the candidate is correct and decides whether to gather more evidence, refine the candidate, verify it or stop. Across six generators and nine coding benchmarks it helps most when verification is costly and critics are informative but imperfect.
- Qu, S. (2026). Adaptive Test-Time Compute Allocation via Learned Heuristics over Categorical Structure. arXiv:2602.03975. Allocates a limited budget of verifier calls across intermediate reasoning states, using feasibility gating, learned ranking and allocation by local uncertainty. On MATH it reports higher accuracy than best-of-N, majority voting and beam search with 44% fewer verifier calls.
- Singh, H., Li, X., Sareen, K., et al. (2026). V1: Unifying Generation and Self-Verification for Parallel Reasoners. arXiv:2603.04304. Models judge pairs of candidate solutions better than they score them one at a time. V1-Infer spends verification calls on the pairs whose order is most uncertain in a tournament, improving Pass@1 by up to 10% over pointwise verification on code and math benchmarks.
- Tang, Y., and Zhan, J. (2026). Verify What Matters: Budgeted Verification for Tool-Using Agents under Counterfactual Downstream Harm. Transactions on Machine Learning Research. Allocates a fixed verification budget across a tool-using agent’s proposed actions using downstream harm as well as local uncertainty. In the authors&rsquo. Controlled sandbox, downstream consequence is the strongest single signal, and the improvement of harm-aware routing is directional, with outcome intervals crossing zero.
- Tong, J., Zhang, Y., Zhao, A., Fan, Y., Ma, Y., and Shen, X. (2026). ProactiveLLM: Learning Active Interaction for Streaming Large Language Models. arXiv:2606.00523; ICML 2026, PMLR 306:121773–121791. A streaming language model learns from its own internal states when a partial input is sufficient to act on, trained by masked streaming modelling and self-distillation from a full-context view of the same model. Lower interaction latency at maintained quality on text and speech streaming tasks.
- Wan, X., Zhu, S., Cai, J., et al. (2026). The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs. arXiv:2606.03092. Inference budget allocation across queries posed as a global constrained optimisation. A shifted-surge utility for each query gives a policy set by a global shadow price that equalises marginal utility, and the resulting method, CLEAR, abandons queries it judges unsolvable to fund those near their emergence thresholds. Up to 3x global accuracy over uniform allocation when resources are scarce.
- Wang, X., Suresh, A., Zhang, A., et al. (2026). Conformal Thinking: Risk Control for Reasoning on a Compute Budget. arXiv:2602.03814; ICML 2026, PMLR 306:130032–130046. Token budgets for reasoning models set as a risk-control problem. An upper threshold stops when the model is confident and a parametric lower threshold stops instances that look unsolvable. Both are calibrated on a validation set with distribution-free risk control to meet a target error rate while minimizing compute.
- Xie, Y., Wang, T., Mallick, S., et al. (2026). Statistical Early Stopping for Reasoning Models. arXiv:2602.13935; ICML 2026. Stops a reasoning trace early using uncertainty signals in the generated text. One rule models the arrival of uncertainty keywords as a renewal process and tests sequentially. A nonparametric rule bounds, in finite samples, the probability of halting too early on well-posed queries. The gains are largest on mathematical reasoning.
- Yang, B., Li, J., Fan, J., et al. (2026). How Much Can Language Models Gain from Test-Time Computation?. arXiv:2610.01110. SELF-POT compares direct inference with parallel sampling and self-revision at fixed multiples of the direct budget, across mathematics, programming and agentic tasks, and charges every model call, including selection and critique, in dollars. The gains depend on the domain, the selection rule and failure handling. On replayed programming candidate pools, public-example selection raises correct submissions from 376 to 453 of 500, and keeping an available candidate when judging fails recovers 61 more at no extra cost.
- Zhai, Z., Li, B., Xiao, B., Li, M., and Wang, X. (2026). Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimization. arXiv:2604.14853. Decides which inputs get more test-time compute by maximising expected accuracy under an average compute budget. Lagrangian relaxation gives a per-instance oracle action, and a lightweight classifier trained to imitate it is deployed, with regret bounded by imitation error times the worst per-instance gap. Up to 12.8% relative accuracy improvement on MATH at matched budget.
- Zhang, X. (2026). Intelligence Under Time Constraints: Rethinking Test-Time Compute. arXiv:2609.14995. A position paper on computing while evidence streams in and may be revised: starting early leaves more time but rests on less evidence, which the paper calls the information–slack dilemma. Advance work is valuable only if its benefit survives the costs of verification, invalidation and recovery, and the paper proposes a research and evaluation agenda rather than experiments.
- Zhou, X., Nguyen, H., Yu, B., Liu, C., and Cheng, L. (2026). Adaptive Stopping for Multi-Turn LLM Reasoning. arXiv:2604.01413. Conformal prediction for multi-turn pipelines such as adaptive retrieval-augmented generation and ReAct. MiCP splits the error budget across turns so the model can stop early while keeping an overall coverage guarantee, and reaches the target coverage on single-hop and multi-hop question answering with fewer turns and smaller prediction sets.
- Zhou, Y., Ouyang, Y., Zheng, K., and Xiang, S. (2026). CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing. arXiv:2608.07424. Test-time reasoning as a choice of whether the next unit of compute goes to generation, verification or stopping. CoBa verifies cheaply across a small candidate set and sends uncertain or high-value candidates to a stronger verifier, matching a self-evaluation weighted-voting proxy (85.13% against 85.20% macro accuracy) with 49.1% fewer parameter-weighted tokens. Best-of-16 majority voting keeps a small edge at much higher cost.
- Zuo, B., Zhou, D., and Zhu, Y. (2026). Adaptive Test-Time Compute Allocation with Evolving In-Context Demonstrations. arXiv:2604.21018; Findings of ACL 2026. A warm-up phase identifies easy queries and collects solved question-response pairs from the test set. Further compute then goes to unresolved queries, each generation conditioned on successful responses to related queries rather than resampled from a fixed distribution. Beats baselines on math, coding and reasoning benchmarks with less inference compute.
Learning from resolved outcomes
- Cotton, P. (2022). Microprediction: Building an Open AI Network. MIT Press. Chapter 5, “Micromanagers” (pp. 81–120), considers agents that solicit, compensate and combine the predictions of other agents.
- Brier, G. W. (1950). Verification of Forecasts Expressed in Terms of Probability. Monthly Weather Review 78(1), 1–3. The squared-error score for probability forecasts.
- Gneiting, T., and Raftery, A. E. (2007). Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association 102, 359–378. The general theory of proper scoring rules, and the reason they can serve as rewards.
- Zou, A., Xiao, T., Jia, R., Kwon, J., Mazeika, M., et al. (2022). Forecasting Future World Events with Neural Networks. arXiv:2206.15474; NeurIPS 2022. Tournament questions paired with a date-ordered news corpus, so a model sees only what a forecaster could have seen.
- Halawi, D., Zhang, F., Chen, Y.-H., and Steinhardt, J. (2024). Approaching Human-Level Forecasting with Language Models. arXiv:2402.18563. A retrieval-augmented forecasting system that nears the crowd aggregate on post-cutoff questions.
- Schoenegger, P., Tuminauskaite, I., Park, P. S., and Tetlock, P. E. (2024). Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy. arXiv:2402.19379. A twelve-model ensemble is statistically indistinguishable from a 925-person crowd on 31 questions.
- Karger, E., Bastani, H., Chen, Y.-H., Jacobs, Z., Halawi, D., et al. (2024). ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities. arXiv:2409.19839. A dynamic benchmark of questions about unresolved events. Experts beat the best language model, $p < 0.001$.
- Bosse, N. I., Mühlbacher, P., Wildman, J., Phillips, L., and Schwarz, D. (2026). Automating Forecasting Question Generation and Resolution for AI Evaluation. arXiv:2601.22444. Agents write and later resolve forecasting questions. About 96% verifiable and 95% correctly resolved.
- Turtel, B., Franklin, D., and Schoenegger, P. (2025). LLMs Can Teach Themselves to Better Predict the Future. arXiv:2502.05253. Self-play forecasts ranked by distance to the outcome, then DPO. 7–10% accuracy gains over base and randomised-label controls.
- Turtel, B., Franklin, D., Skotheim, K., Hewitt, L., and Schoenegger, P. (2025). Outcome-based Reinforcement Learning to Predict the Future. arXiv:2505.17989. Reinforcement learning with verifiable rewards on prediction-market questions. A 14B model matches frontier accuracy with better calibration.
- Joulani, P., György, A., and Szepesvári, C. (2013). Online Learning under Delayed Feedback. arXiv:1306.0686; ICML 2013. Delay multiplies regret in adversarial problems and adds to it in stochastic ones.
- Cesa-Bianchi, N., and Lugosi, G. (2006). Prediction, Learning, and Games. Cambridge University Press. The reference for online combination of expert predictions.
- Gibbs, I. and Candès, E. (2021). Adaptive Conformal Inference Under Distribution Shift. arXiv:2106.00170. Long-run coverage of prediction sets under arbitrary drift. Coverage is not conditional accuracy.
- Arnold, S., Henzi, A., and Ziegel, J. F. (2023). Sequentially Valid Tests for Forecast Calibration. arXiv:2109.11761; Annals of Applied Statistics 17(3), 1909–1935. E-values for testing probabilistic calibration of forecasts that stay valid under optional stopping, so calibration can be monitored continuously. Power competitive with fixed-sample tests, illustrated on probabilistic weather forecasts.
- Han, E., Huang, C., and Wang, K. (2024). Model Assessment and Selection under Temporal Distribution Shift. arXiv:2402.08672; ICML 2024, PMLR 235:17374–17392. Model assessment and selection under unknown temporal distribution shift. An adaptive rolling window decides how much historical data to use when estimating a model’s error, or the difference in error between two models, and a single-elimination tournament over pairwise comparisons gives near-optimal selection.
- Grob, A., Daniele, M., and Ziegel, J. (2026). Sequentially Valid Inference for Probabilistic Inflation Forecasts. arXiv:2608.23064. Applies e-value calibration tests to probabilistic inflation forecasts for the United States, the Euro Area and Switzerland. The sequential tests find evidence against calibration around structural breaks that static full-sample tests often miss.
- Reyzin, L., Saha, A., and Wu, S. (2026). Expert Advice with Costly Observations. UAI 2026, PMLR 337:5700–5715. Prediction with expert advice when each expert’s prediction must be paid for. Tight minimax regret for adversarial losses that interpolates between bandit and full-information feedback and identifies the best number of experts to observe, and an elimination algorithm with instance-dependent regret for stochastic losses, with matching lower bounds.
- Wu, Y., Liang, J., Wang, S., et al. (2026). RAP: Research Attention Prediction Reveals Target-Conditioned Evidence Acquisition Biases. arXiv:2609.10092. A rolling benchmark in which an LLM agent searches a time-restricted arXiv corpus and predicts the next six months&rsquo. Paper shares across research directions, over 278 AI/ML fields and 1,390 episodes. All four diagnostic models do worse in compositional accuracy than an exponentially weighted moving average of exact counts. Carrying forward an estimate of the current state beats direct forecasting, and forecast-oriented policies retrieve a smaller share of recent evidence.
Autonomous research loops
- Lu, C., Lu, C., Lange, R. T., Foerster, J., Clune, J., and Ha, D. (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv:2408.06292. Ideas, code, experiments, paper and automated review, under $15 per paper.
- Beel, J., Kan, M.-Y., and Baumgart, M. (2025). Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?. arXiv:2502.14297. Independent evaluation of the AI Scientist: 42% of experiments failed on coding errors, novelty checks misfired, some papers had hallucinated numbers.
- Yamada, Y., Lange, R. T., Lu, C., Hu, S., Lu, C., et al. (2025). The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search. arXiv:2504.08066. Agentic tree search without human code templates. One of three workshop submissions scored above the average acceptance threshold.
- Jiang, Z., Schmidt, D., Srikanth, D., Xu, D., Kaplan, I., et al. (2025). AIDE: AI-Driven Exploration in the Space of Code. arXiv:2502.13138. Machine-learning engineering as tree search over code.
- Chan, J. S., Chowdhury, N., Jaffe, O., Aung, J., Sherburn, D., et al. (2024). MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. arXiv:2410.07095. 75 Kaggle competitions. o1-preview with AIDE reached bronze or better in 16.9%.
- Huang, Q., Vora, J., Liang, P., and Leskovec, J. (2023). MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation. arXiv:2310.03302. Success from 100% on old datasets to 0% on post-training Kaggle challenges.
- Zhao, M., Dong, J., Xu, K., Hasan, Z., Fan, C., et al. (2026). ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond. arXiv:2608.14354. Long-horizon research with recoverable executable states and value-driven compute allocation. 70.22% any-medal on MLE-bench within 24 hours.
- Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., et al. (2023). Mathematical Discoveries from Program Search with Large Language Models. Nature 625, 468–475. FunSearch. Evolves programs against an evaluator.
- Novikov, A., Vũ, N., Eisenberger, M., Dupont, E., Huang, P.-S., et al. (2025). AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv:2506.13131. Evolutionary code editing with evaluator feedback. A 48-multiplication procedure for $4 \times 4$ complex matrices.
- Lu, C., Holt, S., Fanconi, C., Chan, A. J., Foerster, J., et al. (2024). Discovering Preference Optimization Algorithms with and for Large Language Models. arXiv:2406.08414. A model proposes preference-optimisation losses from the scores of earlier ones.
- Agarwal, D., Majumder, B. P., Adamson, R., et al. (2025). AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise. arXiv:2507.00310; NeurIPS 2025. Open-ended, data-driven discovery that chooses hypotheses by Bayesian surprise, the shift between a language model’s prior and posterior beliefs after an experiment. Monte Carlo tree search with progressive widening uses surprisal as its reward. Across 21 datasets it yields 5–29% more discoveries the model finds surprising than competing methods under a fixed budget.
- Agarwal, D., Adamson, R., McCallum, A., Clark, P., Sabharwal, A., and Majumder, B. P. (2026). Evidence-Informed LLM Beliefs for Continual Scientific Discovery. arXiv:2606.29182. Surprisal measured against a static prior rewards rediscovery. Updates the model’s priors with evidence from earlier hypotheses, finding retrieval over prior discoveries the best of the updating mechanisms tested, and flags 37.5% of static surprisals as spurious. Adding belief-update filtering and diversity maximisation to the search raises accumulated non-stationary surprisal by 30.62% on average across five domains.
- Brilliantov, K., Hernández-Cano, A., and Abbé, E. (2026). How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?. arXiv:2609.40303. Under an equal time budget and the same frontier backbone, open-source machine-learning-engineering harnesses give no advantage over a single session of a minimal coding agent with read, write and bash access. Ablations attribute performance mainly to the backbone model.
- Chen, J., Dalvi Mishra, B., Nam, J., Meng, R., Pfister, T., and Yoon, J. (2026). MARS: Modular Agent with Reflective Search for Automated AI Research. arXiv:2602.02660; ICML 2026. An agent for machine-learning engineering tasks that plans with cost-constrained Monte Carlo tree search, builds modular repositories, and distills lessons by comparing solutions. It reports state-of-the-art results among open-source frameworks on MLE-Bench under comparable settings, and 63% of the lessons it uses come from other branches of the search.
- Foster, T. S., Al Omari, B., Fu, T., et al. (2026). AI Research Preference Models. arXiv:2608.13940. Models that predict which candidate solutions a research agent should run when it can propose more than it can afford to execute. Inside the AIRA-dojo agent on AIRS-Bench they raise the average normalised score from 0.684 to 0.711 (inference only) and 0.729 (with small pilot experiments), and reach the unguided agent’s 24-hour performance in roughly 15 hours.
- Hassoon, A., and Dredze, M. (2026). Capability-Gated Planning: Cost-to-Goal Discovery and the Limits of Myopic Experiment Selection. arXiv:2608.05085. Some experiments build a capability whose value lies in the actions it makes available later, not in the information it returns. For every lookahead depth there is an instance on which every myopic information-maximising planner has an unbounded approximation ratio, and one on which it never reaches the goal. The proposed replanner is tested in a controlled testbed.
- Hong, H., Li, Y., Huy, L. G., Yang, J., Zhou, M., and Lin, C. (2026). Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Papers. arXiv:2609.11117; NLPCC 2026 shared task overview. AgentActionBench evaluates agent-based reproduction of experiments from 150 papers (120 machine learning, 30 AI4Science) by recording the agent’s actions throughout the process and scoring the traces against paper-specific rubrics. Current systems remain limited, with execution the main bottleneck.
- Liu, Z., Wu, J., Peng, R., et al. (2026). Can LLM Design High-Quality Experiments? A Comprehensive and Systematic Benchmark on Autonomous Experimental Design. arXiv:2608.03501. SCOPE, a benchmark built from 300 recent papers in 19 domains, scores LLM experimental plans on high-level completeness (main, ablation and analysis experiments) and low-level configuration (datasets, baselines and metrics). Most LLMs cannot design high-quality experiments directly, all struggle with configuration, and search mode does not help. OptED, an agentic workflow with stage isolation, tools and rule-based constraints, eases the configuration bottleneck.
- Murphy, K. (2026). Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models. arXiv:2608.09696. Combines language-model proposals of model structures, experiment design by value of information, and approximate Bayesian inference over structures, parameters and latent trajectories. Better sample efficiency than baselines on chemical rate laws, a glucose simulator and a new stochastic neuron simulator.
- Philippov, V., Katunin, P., Andreev, D., Ostanin, I., and Nikolaev, A. (2026). Glite ARF: Verifier-Driven Research with Parallel LLM Coding Agents. arXiv:2606.27416. Many LLM coding agents work in parallel on one research repository while deterministic verifier scripts enforce task isolation, immutability of completed work and a corrections overlay. In a shared-task campaign, per-fold provenance caught four target-leaking feature sets, and across three campaigns the structural machinery added about 1% of wall-clock time.
- Xing, S., He, H., Wu, K., et al. (2026). Compute Allocation for Self-Evolving LLMs: From Depth–Breadth to Multi-Armed Bandits. arXiv:2605.29268. Sweeps the depth–breadth grid of LLM-guided evolutionary search over five models and three tasks, and finds a fitness–compute envelope and a bilinear depth–breadth fit. BaSE, a multi-armed bandit that allocates LLM calls across parallel trajectories, raises mean fitness by 12.3% over the strongest island-protocol baseline across eight model–task cells, with the largest gains where run-to-run variance is high.
- Yu, X., Bai, F., Sun, K., et al. (2026). PrimeScientist: Strategic Allocation of Research Effort in Autonomous Research. arXiv:2609.17846. Treats the allocation of research effort across successive attempts as a sequential decision problem in which remaining resources guide the policy. An executable plan tree keeps competing plans and their outcomes, and an MCTS-based policy balances exploration and exploitation using experimental feedback and the remaining budget. Across 12 AI research tasks it raises average reward by 10.3% with 50.6% fewer attempts than AutoResearch under the same budget.
Quantitative research and trading agents
- Yang, X., Liu, W., Zhou, D., Bian, J., and Liu, T.-Y. (2020). Qlib: An AI-oriented Quantitative Investment Platform. arXiv:2009.11189. Infrastructure for AI-driven quantitative research.
- Tang, Z., Chen, Z., Yang, J., Mai, J., Zheng, Y., et al. (2025). AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay. arXiv:2502.16789. Factor mining with originality, hypothesis alignment and complexity constraints to resist decay.
- Mao, J., Li, C., Li, X., Duan, Q., Yuan, J., et al. (2026). EVOQUANT: Self-Evolving Verifier-Guided Strategy Optimization for Robust Quantitative Trading. arXiv:2607.12455. Verified strategy edits distilled into reusable knowledge.
- Yu, Y., Li, H., Chen, Z., Jiang, Y., Li, Y., et al. (2023). FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design. arXiv:2311.13743. Layered memory for a trading agent.
- Zhang, W., Zhao, L., Xia, H., Sun, S., Sun, J., et al. (2024). A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist. arXiv:2402.18485. FinAgent. Multimodal market intelligence with dual-level reflection.
- Yu, Y., Yao, Z., Li, H., Deng, Z., Cao, Y., et al. (2024). FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making. arXiv:2407.06567. Manager-analyst hierarchy with conceptual verbal reinforcement.
- Xiao, Y., Sun, E., Luo, D., and Wang, W. (2024). TradingAgents: Multi-Agents LLM Financial Trading Framework. arXiv:2412.20138. A simulated trading firm of specialised agents.
- Nguyen, P. and Pham, T. (2026). Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness. arXiv:2603.27539. Five evaluation failures in multi-agent trading systems that can reverse the sign of reported returns.
- Li, W. W., Kim, H., Cucuringu, M., and Ma, T. (2025). Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?. arXiv:2505.07078. FINSABER. Reported LLM timing advantages deteriorate over two decades and 100+ symbols.
Self-improvement mechanisms
- Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., et al. (2023). DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. arXiv:2310.03714. Pipelines as declarative modules compiled against a metric.
- Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., et al. (2023). Large Language Models as Optimizers. arXiv:2309.03409; ICLR 2024. OPRO. Past solutions and scores in the prompt.
- Fernando, C., Banarse, D., Michalewski, H., Osindero, S., and Rocktäschel, T. (2023). Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution. arXiv:2309.16797. Evolves task prompts and the mutation prompts that produce them.
- Yuksekgonul, M., Bianchi, F., Boen, J., Liu, S., Huang, Z., et al. (2024). TextGrad: Automatic "Differentiation" via Text. arXiv:2406.07496. Textual feedback propagated backwards through a computation graph.
- Agrawal, L. A., Tan, S., Soylu, D., Ziems, N., Khare, R., et al. (2025). GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457; ICLR 2026. Reflective prompt evolution over execution traces. Beats GRPO by 6% on average with up to 35 times fewer rollouts.
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., and Yao, S. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366. Verbal reflections kept in episodic memory across trials.
- Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., and Huang, G. (2023). ExpeL: LLM Agents Are Experiential Learners. arXiv:2308.10144; AAAI 2024. Insights extracted across tasks and recalled at inference.
- Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., et al. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291. A growing library of executable skills.
- Zhang, S., Wang, J., Zhou, R., Liao, J., Feng, Y., et al. (2026). MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory. arXiv:2601.03192. Learns which episodic memories are useful from environmental feedback, with weights fixed.
- Shao, R., Shen, S. Z., Yin, J. O., Li, Y., Wang, M., et al. (2026). Context Language Models. arXiv:2609.37725. Context as a file the model rewrites without restriction, trained in context and by RL. The latent counterparts are surveyed on the latent reasoning site.
- Li, M. Y., Hamid, J. I., Fox, E. B., and Goodman, N. D. (2026). Neural Garbage Collection: Learning to Forget while Learning to Reason. arXiv:2604.18002. A model trained from outcome reward alone to evict entries from its own KV cache while it reasons.
- Schmidhuber, J. (2003). Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements. arXiv:cs/0309048. Provably optimal self-rewrites, conditional on finding the proof.
- Zelikman, E., Lorch, E., Mackey, L., and Kalai, A. T. (2023). Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. arXiv:2310.02304; COLM 2024. A scaffold that improves itself, with sandbox-bypass rates measured.
- Hu, S., Lu, C., and Clune, J. (2024). Automated Design of Agentic Systems. arXiv:2408.08435. A meta agent programs new agents from an archive of discoveries.
- Yin, X., Wang, X., Pan, L., Lin, L., Wan, X., and Wang, W. Y. (2024). Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement. arXiv:2410.04444; ACL 2025. An agent that modifies its own logic under a high-level objective.
- Zhang, J., Hu, S., Lu, C., Lange, R., and Clune, J. (2025). Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. arXiv:2505.22954. Empirical validation replaces proof. SWE-bench 20.0% to 50.0%.
- Luo, X., Zhang, Y., He, Z., Wang, Z., Zhao, S., et al. (2025). Agent Lightning: Train ANY AI Agents with Reinforcement Learning. arXiv:2508.03680. Agent execution decoupled from RL training.
- He, Z., Zhang, S., Zhou, Z., Yang, Y., Kang, Y., et al. (2026). Agent Lightning v1.0: Towards Harnessed Agentic RL. arXiv:2608.17528. The deployment harness owns the loop. SWE-bench Verified 41.8% to 56.4% for Qwen3.5-9B.
- Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., et al. (2023). Large Language Models Cannot Self-Correct Reasoning Yet. arXiv:2310.01798. Without external feedback, models struggle to correct their reasoning and sometimes get worse.
- Hospedales, T., Antoniou, A., Micaelli, P., and Storkey, A. (2020). Meta-Learning in Neural Networks: A Survey. arXiv:2004.05439. The meta-learning literature before language agents.
- Wang, W., Piękos, P., Li, N., et al. (2025). Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine. arXiv:2510.21614; ICLR 2026. In a tree of self-modifying coding agents, an agent’s own benchmark score is a poor guide to how good its descendants will be. Scores each node by the aggregated performance of its clade instead, and uses an estimate of that score to decide which node to expand. Outperforms earlier self-improving coding-agent methods on SWE-bench Verified and Polyglot while using fewer CPU hours.
- Li, X., Li, H., Zhou, Y., Liu, Y., and Ai, Q. (2026). Beyond Experience Retrieval: Learning to Generate Utility-Optimized Structured Experience for Frozen LLMs. arXiv:2602.02556; ACL 2026. SEAM is a small executor-specific module that stores experience in its parameters and generates a structured experience entry in one forward pass to guide a frozen LLM, in place of similarity-based retrieval. It is trained for utility with executor rollouts and GRPO, and can be refined after deployment by supervised fine-tuning on logged successful trajectories. Evaluated on mathematical reasoning benchmarks.
Relational features and weak labels
- Kanter, J. M., and Veeramachaneni, K. (2015). Deep Feature Synthesis: Towards Automating Data Science Endeavors. IEEE DSAA 2015. Stacked aggregations across related tables. The explicit baseline.
- Fey, M., Hu, W., Huang, K., Lenssen, J. E., Ranjan, R., et al. (2023). Relational Deep Learning: Graph Representation Learning on Relational Databases. arXiv:2312.04615. A database as a temporal heterogeneous graph, one node per row.
- Robinson, J., Ranjan, R., Hu, W., Huang, K., Han, J., et al. (2024). RelBench: A Benchmark for Deep Learning on Relational Databases. arXiv:2407.20060. The benchmark, with a user study against a data scientist’s hand-built features.
- Gu, J., Ranjan, R., Kanatsoulis, C., Tang, H., Jurkovic, M., et al. (2026). RelBench v2: A Large-Scale Benchmark and Repository for Relational Data. arXiv:2602.12606; ICLR 2026. 11 datasets, 22 million rows, autocomplete tasks under temporal constraints.
- Zhang, Y., Xu, L., Gan, Q., Wipf, D., and Wang, M. (2026). RDBLearn: Simple In-Context Prediction Over Relational Databases. arXiv:2602.18495. Relational aggregations plus a tabular foundation model, competitive with supervised baselines.
- Gentzkow, M., Kelly, B., and Taddy, M. (2019). Text as Data. Journal of Economic Literature 57(3), 535–574. Statistical methods for text in economics.
- Ke, Z. T., Kelly, B., and Xiu, D. (2019). Predicting Returns with Text Data. NBER Working Paper 26186. Sentiment learned from subsequent returns rather than from a dictionary.
- Lopez-Lira, A. and Tang, Y. (2023). Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models. arXiv:2304.07619. GPT-4 scores capture initial reactions to post-cutoff headlines. Returns decline as adoption rises.
- Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., and Ré, C. (2017). Snorkel: Rapid Training Data Creation with Weak Supervision. arXiv:1711.10160. Labelling functions of unknown accuracy, denoised without ground truth.
From forecasts to decisions
- Elmachtoub, A. N., and Grigas, P. (2022). Smart “Predict, then Optimize”. Management Science 68(1), 9–26; arXiv:1710.08005. Prediction models trained on decision error rather than prediction error. SPO+ is a convex, statistically consistent surrogate.
- Gârleanu, N., and Pedersen, L. H. (2013). Dynamic Trading with Predictable Returns and Transaction Costs. Journal of Finance 68(6), 2309–2340. The optimal portfolio trades partway toward an aim portfolio, and persistent signals get more weight.
- Dudik, M., Langford, J., and Li, L. (2011). Doubly Robust Policy Evaluation and Learning. arXiv:1103.4601; ICML 2011. Accurate policy values if either the reward model or the logging-policy model is good.
- Swaminathan, A. and Joachims, T. (2015). Counterfactual Risk Minimization: Learning from Logged Bandit Feedback. arXiv:1502.02362. Propensity-weighted learning with a variance penalty.
- Levine, S., Kumar, A., Tucker, G., and Fu, J. (2020). Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv:2005.01643. The tutorial on learning policies from fixed data.
- Kumar, A., Zhou, A., Tucker, G., and Levine, S. (2020). Conservative Q-Learning for Offline Reinforcement Learning. arXiv:2006.04779. A value function that lower-bounds the true value.
- Kostrikov, I., Nair, A., and Levine, S. (2021). Offline Reinforcement Learning with Implicit Q-Learning. arXiv:2110.06169. Never queries actions outside the dataset.
- Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018). Double/Debiased Machine Learning for Treatment and Structural Parameters. The Econometrics Journal 21(1), C1–C68. Orthogonal scores and cross-fitting. Estimation under stated identification assumptions, not a substitute for them.
- Laroche, R., Trichelair, P., and Tachet des Combes, R. (2019). Safe Policy Improvement with Baseline Bootstrapping. arXiv:1712.06924; ICML 2019. Batch reinforcement learning from a fixed dataset, with a guarantee to perform at least as well as the baseline policy that collected the data. SPIBB copies the baseline on state-action pairs where uncertainty is high, has safe-improvement guarantees, and beats competing algorithms in safety and mean performance on gridworld and random MDPs, with a deep variant (SPIBB-DQN).
- Simão, T. D., Laroche, R., and Tachet des Combes, R. (2020). Safe Policy Improvement with an Estimated Baseline Policy. arXiv:1909.05236; AAMAS 2020. SPIBB when the baseline policy is unknown, as with data collected under human supervision. A baseline estimated from the data still yields safe-improvement guarantees over the true baseline, with little loss relative to SPIBB given the true baseline.
- Kuzborskij, I., Vernade, C., György, A., and Szepesvári, C. (2021). Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting. arXiv:2006.10460; AISTATS 2021. A lower confidence bound on the value of a target policy from logged contextual-bandit data, built on the self-normalized importance-weighting estimator, used to select among a set of candidate policies. On synthetic and real datasets it gives tighter intervals and better chosen policies than its main competitors.
Evaluating a search
- White, H. (2000). A Reality Check for Data Snooping. Econometrica 68(5), 1097–1126. Tests whether the best model from a search beats a benchmark, accounting for the search.
- Hansen, P. R. (2005). A Test for Superior Predictive Ability. Journal of Business & Economic Statistics 23(4), 365–380. More power than the reality check, less sensitive to poor alternatives.
- Romano, J. P., and Wolf, M. (2005). Stepwise Multiple Testing as Formalized Data Snooping. Econometrica 73(4), 1237–1282. Which of many strategies beat the benchmark.
- Harvey, C. R., Liu, Y., and Zhu, H. (2016). … and the Cross-Section of Expected Returns. Review of Financial Studies 29(1), 5–68. Multiple testing applied to hundreds of published factors.
- Bailey, D. H., and López de Prado, M. (2014). The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality. Journal of Portfolio Management 40(5), 94–107. A Sharpe ratio corrected for the number of trials.
- Bailey, D. H., Borwein, J., López de Prado, M., and Zhu, Q. J. (2016). The Probability of Backtest Overfitting. Journal of Computational Finance. The chance that the in-sample winner is below median out of sample.
- Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., and Roth, A. (2014). Preserving Statistical Validity in Adaptive Data Analysis. arXiv:1411.2664. Holdout reuse for adaptively chosen queries, via differential privacy.
- Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., and Roth, A. (2015). The Reusable Holdout: Preserving Validity in Adaptive Data Analysis. Science 349(6248), 636–638. The practical version. Assumes independent samples from a fixed distribution.
- Kapoor, S. and Narayanan, A. (2022). Leakage and the Reproducibility Crisis in ML-based Science. arXiv:2207.07048. Leakage in 17 fields and 329 papers, and a taxonomy of eight types.
- Paleka, D., Goel, S., Geiping, J., and Tramèr, F. (2025). Pitfalls in Evaluating Language Model Forecasters. arXiv:2506.00723. Temporal leakage and weak extrapolation in evaluations of LLM forecasters.
- Glasserman, P. and Lin, C. (2023). Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis. arXiv:2309.17322. Distraction by general company knowledge outweighs look-ahead in-sample.
- He, S., Lv, L., Manela, A., and Wu, J. (2025). Chronologically Consistent Large Language Models. arXiv:2502.21206. ChronoBERT and ChronoGPT, trained only on text available at each date.
- Yan, Y., Tang, R., Gao, Z., Jiang, W., and Lu, Y. (2026). DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining. arXiv:2603.11838. Twelve 1.3B models with annual cutoffs. A look-ahead premium of 26.4 basis points per standard deviation.
- Gao, Z., Jiang, W., and Yan, Y. (2025). Detecting Lookahead Bias in LLM Forecasts. arXiv:2512.23847. A look-ahead propensity estimated from date-only recall queries.
- Benhenda, M. (2026). Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance. arXiv:2601.13770. Look-ahead measured by performance decay across regimes.
- Li, W. W., Wang, M., and Ma, T. (2026). Summoning the Oracle to Slay It: Mitigating Look-Ahead Bias in Financial Backtesting with Large Language Models. arXiv:2605.24564; EMNLP 2026. FinCAD. Attenuates memorised outcomes at inference.
- Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., and Roth, A. (2015). Generalization in Adaptive Data Analysis and Holdout Reuse. arXiv:1506.02629. An algorithm for reusing a holdout set to validate many adaptively chosen hypotheses without overfitting to it, illustrated on a synthetic experiment. Guarantees come from differential privacy and, separately, from description length, unified through approximate max-information.
- Hardt, M. (2017). Climbing a Shaky Ladder: Better Adaptive Risk Estimation. arXiv:1706.02733. Revisits the leaderboard problem behind the Ladder algorithm of Blum and Hardt (2015). A randomized Ladder attains leaderboard error O(1/n0.4), improving the earlier O(1/n1/3), and any further improvement would imply progress on general adaptive estimation.
- Jamieson, K. and Jain, L. (2018). A Bandit Approach to Multiple Testing with False Discovery Control. arXiv:1809.02235; NeurIPS 2018. Adaptive sampling of n distributions to find those whose means exceed a baseline, using as few samples as possible to reach a target true positive proportion with anytime control of the false discovery proportion. Sample complexity matches information-theoretic lower bounds.
- Corso, A., Du, P., Driggs-Campbell, K., and Kochenderfer, M. J. (2019). Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validation. arXiv:1908.01046; IEEE ITSC 2019, 163–168. Plain adaptive stress testing tends to find failures that are unavoidable and to rediscover the same kinds of failure. Adding domain information to the search finds a larger and more varied set of failure scenarios for an autonomous vehicle policy.
- Lee, R., Mengshoel, O. J., Saksena, A., et al. (2020). Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning. arXiv:1811.02188; Journal of Artificial Intelligence Research 69, 1165–1201. Searches for the most likely path to a failure in simulation, treating the system as a black box and posing the search as a Markov decision process solved by reinforcement learning. A Monte Carlo tree search variant needs only the simulator’s pseudorandom number generator, and a differential variant finds failures one system has and another lacks. Demonstrated on aircraft collision avoidance.
- Balcan, M.-F., Sandholm, T., and Vitercik, E. (2021). Generalization in Portfolio-Based Algorithm Selection. arXiv:2012.13315; Proceedings of the AAAI Conference on Artificial Intelligence 35(14), 12225–12232. Generalisation guarantees when a portfolio of algorithm parameter settings and a per-instance selector are both learned from the same training instances. A large portfolio overfits even with a very simple selector, so portfolio size trades coverage of instances against overfitting.
- Kossen, J., Farquhar, S., Gal, Y., and Rainforth, T. (2021). Active Testing: Sample-Efficient Model Evaluation. arXiv:2103.05331; ICML 2021. Chooses which test points to label so that model evaluation needs fewer labels. Acquisition strategies designed for testing rather than training, and a correction for the bias active selection introduces that also reduces the estimator’s variance.
- Witzany, J. (2021). A Bayesian Approach to Measurement of Backtest Overfitting. Risks 9(1), 18. When a strategy is chosen from many models tested on the same holdout, out-of-sample backtesting is unreliable. A Bayesian approach fitted by Markov chain Monte Carlo estimates the probability of backtest overfitting and adjusts performance measures such as the Sharpe ratio, tested on a class of technical trading strategies.
- Xu, Z., Wang, R., and Ramdas, A. (2021). A unified framework for bandit multiple testing. arXiv:2107.07322; NeurIPS 2021. Bandit multiple testing with exploration kept separate from the summary of evidence. Per-arm e-processes combined by e-BH control the false discovery rate for arbitrary exploration rules, stopping times and composite nulls, even with dependent arms or several arms queried at once. Matching sample complexity is recovered in the independent sub-Gaussian case, and other settings are not analysed.
- Chen, J. and Andrews, I. (2023). Optimal Conditional Inference in Adaptive Experiments. arXiv:2309.12162. Inference in batched bandit experiments conditional on the realized stopping time, assignment probabilities and target parameter, all of which may be chosen adaptively. Without further restrictions, using only the last batch is optimal. Location invariance or polyhedral selection events allow more of the data to be used, with tractable optimal conditional procedures in the polyhedral case.
- Andrews, I., Kitagawa, T., and McCloskey, A. (2024). Inference on Winners. Quarterly Journal of Economics 139(1), 305–358. Choosing among options by their estimates creates a winner’s curse: the selected option’s estimate is biased upward and conventional confidence intervals are invalid. Gives estimators and confidence intervals that remove it, illustrated on job-training programs and neighborhoods selected by estimated effects.
- Neufeld, A., Dharamshi, A., Gao, L. L., and Witten, D. (2024). Data Thinning for Convolution-Closed Distributions. arXiv:2301.07276; Journal of Machine Learning Research 25(57), 1–35. Splits one observation into independent parts that sum to it, each from the same family up to a known scaling of a parameter, for any convolution-closed distribution, including the Gaussian, Poisson, negative binomial, gamma and binomial. This allows cross-validation, and validation of unsupervised methods such as k-means and principal components, where sample splitting is unattractive or unavailable.
- Dharamshi, A., Neufeld, A., Motwani, K., Gao, L. L., Witten, D., and Bien, J. (2025). Generalized Data Thinning Using Sufficient Statistics. arXiv:2303.12931; Journal of the American Statistical Association 120(549), 511–523. Generalizes data thinning so that the independent parts need only reconstruct the observation through a known function rather than a sum. This extends thinning to more families of distributions and shows that sample splitting and data thinning are both applications of sufficiency.
- Zrnic, T. and Fithian, W. (2025). A Flexible Defense Against the Winner’s Curse. arXiv:2411.18569; Annals of Statistics 53(6), 2516–2535. The zoom correction gives valid inference on the empirically best candidate. It works in parametric and nonparametric settings, allows arbitrary dependence between candidates, adapts to the amount of selection bias, and extends to the top k winners, the identity of the population winner and near-winners.
- Arnold, S., Gavrilopoulos, G., Schulz, B., and Ziegel, J. (2026). Sequential Model Confidence Sets. arXiv:2404.18678; Journal of the Royal Statistical Society, Series B. Extends the model confidence set of Hansen et al. to continuous monitoring using e-processes and confidence sequences. The set of retained models covers the best models with time-uniform, nonasymptotic guarantees.
- Ashouritaklimi, K., Kilian, V., Huang, D., Rainforth, T., and Caron, F. (2026). Prediction-Powered Active Testing. arXiv:2607.08347. Active testing in which a black-box model’s predictions enter the unbiased LURE estimator as a control variate rather than as pseudo-labels, keeping the risk estimate unbiased while reducing its variance. Acquisition rules aimed at that variance, asymptotic normality for confidence intervals, and target coverage with fewer labels on tabular regression and image classification tasks.
- Li, Y., Zhou, Y., Teng, Y. L., et al. (2026). Finding the Right Fit: Model–Harness Interactions across Agent Tasks. arXiv:2610.00917. Evaluates 66 agent configurations: five models paired with four configurable harnesses on three agent benchmarks, plus two native vendor pairings. Model rankings reverse across harnesses, the best harness for a model changes between benchmarks for four of the five models, and higher cost does not reliably buy a higher score. The adapters, evaluation code and 6,204 scored trajectories are released.
- Li, P., Tang, X., Chen, S., et al. (2026). Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks. arXiv:2511.04689; ICML 2026. ATLAS estimates an LLM’s ability with item response theory, choosing benchmark items adaptively by Fisher information. On HellaSwag it matches whole-bank ability estimates with 41 of 5,600 items (0.157 MAE). Ability estimates separate models with identical accuracy, and among more than 3,000 evaluated models 23–31% shift by more than ten rank positions.
- Pilditch, T. D. (2026). Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations. arXiv:2608.14425. Language model evaluation treated as sequential measurement: hierarchical Bayesian inference decides which benchmark items to keep sampling and stops where estimates are precise or stable enough. In an illustrative 200-item, 10-epoch evaluation it removes 57–97% of planned trials across nine validation settings with conclusions equivalent to the full run, and the paper notes that the savings depend on evaluation design.
Surveys
- Gao, H., Geng, J., Hua, W., Hu, M., Juan, X., et al. (2025). A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence. arXiv:2507.21046; TMLR 2026. What, when and how agents evolve.
- Ren, Z., Chen, Y., Guo, D., Rong, G., Li, T., et al. (2026). Self-Improvements in Modern Agentic Systems: A Survey. arXiv:2607.13104. Self-improvement as an update operator on model parameters or scaffold components.
- Tayal, K., Sharma, A., Winata, G. I., Das, A., and Sahu, S. (2026). The Rise of Verbal Reinforcement Learning. arXiv:2609.01597. Natural-language feedback as grounding, deliberation and learning signal.
- Corso, A., Moss, R., Koren, M., Lee, R., and Kochenderfer, M. (2021). A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems. Journal of Artificial Intelligence Research 72. Simulation-based safety validation of black-box systems: falsification, finding the most likely failure, and estimating failure probability, with algorithms from optimisation, path planning, reinforcement learning and importance sampling.
- Mahsereci, M., and Karvonen, T. (2026). Bayesian Quadrature. arXiv:2602.16218; Foundations and Trends in Machine Learning 19(2–3), 121–240. A survey of Bayesian quadrature: foundations, a taxonomy along modelling, inference and sampling, general theoretical guarantees, a controlled numerical study, and an assessment of practical limitations.