This post gives some details of my speed talk at the SCB Oceania conference, which is in room P9 on Thursday 7 July at 11:50 as part of a session on conservation planning and adaptive management. We have submitted this work to Ecological Applications – a copy is available here, so please add to the peer review by giving us comments.
Every natural resource management agency seems to do (or at least claims to do) adaptive management, which seeks to use management and monitoring to learn about the system being managed, thereby improving future management. It is sometimes referred to as “learning by doing”.
Active adaptive management seeks to explicitly design management like an experiment, and entails extra costs. The experimental design and monitoring requires more resources than simply just managing the system. Further, if two management strategies are implemented at the same time, then inevitably one of them will be inferior.
The benefits of improved management in future can be weighed against these extra costs. And the optimal balance between these costs and benefits can be determined, thereby optimizing the design of adaptive management programs to maximize performance.
In my opinion, attempts to optimize adaptive management programs have been overwhelmingly disappointing. Firstly, the optimizations seem to only work on relatively small problems (but see Nicol and Chadès 2012). Secondly, each published optimization is different in fundamental ways from others, making it difficult to derive generalities across studies. And perhaps most depressingly, the benefits of optimizing adaptive management seem small – the optimizations typically only increase expected performance by a few percent at most.
The apparently minor benefits of adaptive management make me worry that science might be impotent; we go to all that effort to optimize the design, yet only get tiny improvements. Surely science is better than that!
So with Moore and a few other colleagues, we decided to examine optimal adaptive management to ask a few fundamental questions:
When is adaptive management most useful?
What drives optimal experimentation?
How much should be invested in experimentation?
How big are the benefits of adaptive management
To answer these questions, we set up the simplest possible adaptive management problem. We considertwo possible management options and two time steps. The first time step allows for possible experimentation, and the apparently best option is applied exclusively in the second time step.
Each option has an expected level of performance (which was uncertain), and we need to determine how much effort to expend on each option in the first time step. Each unit of management effort in the first time step is monitored so that its performance can be assessed. Each option has a per unit cost of implementation and a per unit cost of monitoring.
The monitoring data will be uncertain, so we will be more certain about the performance of each option as investment in each option in the first time period effort increases. However, increasing the level of effort allocated to each option in the first time period will decrease the resources available to spend in the second time step. Thus, when we invest more in the first time period, we can more reliably choose between options in the second time period, but we will have fewer resources to spend on the apparently best option. We face a trade-off!
While this formulation of adaptive management is as simple as we could devise, it is still somewhat complex. In total the model has 11 parameters plus the two control variables (the control variables were how much to allocate to each option in the first time period).
We show how the trade-off between learning and saving resources for acting later can be optimized, and the results have some interesting features. Firstly, various thresholds exist. For example, as the expected difference in performance of the two options increases, the optimal effort to spend on experimentation increases, but only up to a point.
Once the threshold difference in performance is sufficiently large, the optimal level of experimentation declines to zero. This threshold makes some intuitive sense; once we are relatively sure of the difference in performance, then we shouldn’t bother with an experiment to evaluate that. However, prior to reaching that threshold, the optimal effort to spend on the experiment increases with the expected difference in performance; that is somewhat counter intuitive.
Similar thresholds exist for the budget and the prior level of uncertainty in performance. However, while the optimal level of experimentation increases with the expected difference in performance, the biggest benefits of experimentation are realized when the expected performance of the two strategies are the same. Thus, the greatest benefits of experimentation are realized under conditions that differ from when the optimal level of experimentation is greatest.
This work helps to illustrate some fundamental features of adaptive management. We also tie the results explicitly to the notion of expected value of sample information. And we derive analytical solutions for the optimal level of experimentation for some special cases of parameter values.
While the paper is quite mathematically involved, the concept itself is quite straight-forward, and the results are very interpretable. I think it is a very interesting study – see what you think. Please send us comments to help us improve the paper while it is being peer reviewed.
Moore AL, Walker L, Runge MC, McDonald-Madden E & McCarthy MA (in review) Two-step adaptive management for choosing between two management actions.