I’m speaking tomorrow on the last afternoon of the International Statistical Ecology Conference in Montpellier. I’ll be arguing that usual metrics (e.g., AIC) to measure the performance of species distribution models (SDMs) might not actually be relevant for selecting models that are to be used for management. There are two reasons for this.
The first reason is the metrics are usually based on the ability to fit the calibration data, not the quality of their predictions where they will be used.
Secondly, most metrics of SDMs consider statistical fit (e.g., AIC can be thought of as a bias-corrected estimate of deviance), which also might not be relevant to management. Mangers will want to select the model that will provide the best management outcome. The link between the usual metrics and management outcomes is indirect at best.
Instead, I develop a metric that is relevant when using SDMs to optimize eradication or searches of species. This metric aims to reflect the number of occurrences of a species that will remain undetected when using a species distribution model for spatial allocation of search effort.
The paper by Lawson et al. (2014) suggests that metrics need to reflect the use, but I think falls short of stating that the metrics that are used are actually not as relevant as they could be.
Slides from my talk are available here.
Relevant references that I build on are:
- Guillera-Arroita, G., Hauser, C. E. and McCarthy, M. A. (2009) Optimal surveillance strategy for invasive species management when surveys stop after detection. Ecology and Evolution 2014; 4(10):1751–1760. doi: 10.1002/ece3.1056
- Hauser, C. E. and McCarthy, M. A. (2009) Streamlining ‘search and destroy’: cost-effective surveillance for invasive species management. Ecology Letters, 12: 683–692. doi: 10.1111/j.1461-0248.2009.01323.x
- Lawson, C. R., Hodgson, J. A., Wilson, R. J., Richards, S. A. (2014) Prevalence, thresholds and the performance of presence–absence models. Methods in Ecology and Evolution, 5: 54–64. doi: 10.1111/2041-210X.12123
- McCarthy, M. A., Thompson, C. J., Hauser, C., Burgman, M. A., Possingham, H. P., Moir, M. L., Tiensin, T. and Gilbert, M. (2010) Resource allocation for efficient environmental management. Ecology Letters, 13: 1280–1289. doi: 10.1111/j.1461-0248.2010.01522.x