COVID counterfactuals

So, there is yet another media article pondering whether Victoria’s lockdown is warranted… These articles invariably mention various countries, the level of restrictions, number of deaths and sometimes the current trajectory of cases. We’ve read it all before.

The latest article I read* notes that fewer than 900 Australians have died due to COVID, a small fraction of the approximate 160,000 Australians who die each year. The question posed, and answered by implication, is whether Victoria’s lockdown has saved lives or stifled them? But a proper counterfactual is missing – how many lives has lockdown saved?

Sweden provides a useful point of reference, because rather than having very strict mandates from government, it relies more on individual actions and choices of people. If Australia had the same per-capita death rate as Sweden, approximately 15,000 Australians would have died. We’d also have a similar number of deaths if Australia’s per-capita death rate had been the same as in Italy, Spain, UK, or USA (remembering these other countries have had various lockdowns too). So restrictions in Australia have seemingly been effective, and compared to some other countries, might have saved at least 14,000 lives so far.

And let’s emphasise so far. The COVID pandemic has a fair way to run – likely well into 2021 if effective vaccines are not available within the next few months. And we should remember that the number of deaths is only one measure of the impacts: long-term health effects are also prominent.

The other counterfactual to consider is how much better our lives would be with laxer restrictions. Sitting in Melbourne, I have not left my suburb for a couple of months, I’ve been working from home for six months, and I’m unlikely to return to my office until sometime next year. And I could live without any more zoom/teams/meet/webex meetings for a while**. I would love to have fewer restrictions. But would I love the increase in cases that would arise if restrictions were lifted?

While I recognise that the social and economic impacts of the pandemic are multi-faceted, change in GDP for each country provides one measure of this. Let’s consider the G20 countries. We can compare the current per-capita death rate from COVID to the OECD’s forecast reduction in GDP for each country in 2020. That relationship indicates (somewhat crudely) what might have happened to Australia’s economy in the presence of more deaths (which would occur with laxer restrictions).

OECD forecast percentage reduction in GDP for 2020 compared to per capita death rate from COVID (log-scale) for G20 countries (as at 25 September 2020).

Contrary to the view that saving lives via restrictions reduces economic outcomes, COVID death rates are positively correlated with economic harm. The G20 countries with lower COVID death rates have experienced a smaller reduction in GDP than those countries with more deaths. While this is merely correlative, these data show that had we implemented softer restrictions, Australia could have had worse economic outcomes – the issue is the degree to which the disease (rather than the restrictions) harms the economy.

We can also look at the trajectory of the disease in each of these countries: the number of new cases per capita so far in September (up to 25 Sept) also correlates with the current COVID death rate. So those countries that have limited COVID deaths are on track to have fewer new COVID cases (and likely fewer deaths in the future, and potentially better economic conditions).

New COVID cases in September 2020 per capita compared to COVID deaths per capita for G20 countries (as at 25 September 2020; both axes on log scales).

The COVID pandemic still has months to run; in the absence of a vaccine, it will be many months. But so far, countries have had more than 100-fold differences in per-capita deaths from the disease and quite different economic impacts. All countries have imposed various restrictions. The comparatively onerous restrictions in Australia have likely saved up to 14,000 lives when compared to the worst-hit G20 countries. And the OECD forecasts that the economic impact on GDP will comparable or better in Australia than most other G20 countries.

Clearly, Australia’s economic and health outcomes would have been better without Victoria’s spike in cases that originated from a breach of hotel quarantine. However, once the breach had occurred and cases were increasing rapidly from June, Victoria’s subsequent lockdown reduced the number of deaths and long-term illnesses that would have otherwise occurred.

Consider Arizona, a US state with a similar population size to Victoria. Arizona had a large spike in cases that peaked in early July at around 4000 cases per day, forcing the implementation of more restrictions. With about 10 times as many infections as Victoria, Arizona has also experienced about 10 times as many COVID deaths.

Now with around 500 cases per day in Arizona (close to Victoria’s peak in early August), they have recorded infections in about 3% of the population compared to Victoria’s 0.3%. With the vast majority of the both populations yet to be infected, the number of cases and deaths could rise substantially in both states in the absence of control measures. But cases in Arizona remain high – much higher than in Victoria, and the health (and economic) prospects seem much worse in Arizona than Victoria.

We are seeing alarming increases in new cases across much of Europe since the start of July, an indication of what can happen if the disease is not controlled: exponential growth. The feature of exponential growth is that cases accelerate rapidly from seemingly low levels. These other locations provide the counterfactual of what happens to the number of cases if COVID is not controlled. Counterfactuals for the economic impacts suggest that controlling the disease can also lead to better economic impacts, something that a range of economists predicted at the start of the epidemic.

The next time someone muses about whether controlling COVID is worse than the disease, I sincerely hope that they make a decent fist of trying to answer that question. Because when I look at the evidence, the answer seems to be a resounding “no”.

————————————————-

* I am not going to link to the article – it really seems to be click bait.

** I have about 20 meetings already in the calendar for next week, which could be worse now that the bulk of my teaching has just finished for the year. And while I might moan, virtual meetings are so much better than no interactions.

Where is COVID-19 heading in Australia?

Update: 18 April 2020, p.m.

The relatively slow decline in active Australian cases (see original post below) reflects some regional variation in the progression of the epidemic. For example, the number of active cases has increased in Tasmania. In Victoria, active cases have declined much faster than the national aggregate. If you head over to Ben Phillip’s COVID-19 forecaster and select Victoria, you will see that the number of cases has been halving ever 4-5 days. One gets a similar rate of decline if examining the number of new cases in Victoria when excluding imports (both new cases and the number of active cases should decline exponentially).

Number of active cases in Victoria with a fitted exponential curve (from au.covid19forecast.science.unimelb.edu.au). This rate of decline is substantially faster than for the aggregated Australian data.

With that trend, the expected number of active cases in Victoria will equal 0.5 by mid May. That is a little more encouraging than when examining the aggregated Australian data. The data for New South Wales is unreliable because the number of recoveries has not been updated consistently over time in the JHU data repository. However, rates of decline in some other Australian states and territories have been similar to Victoria (e.g., the number of active cases have halved in less than a week in the ACT and South Australia).

Original post

In Australia and a few other countries*, COVID-19 cases are declining. But where are we heading? Here I look at the data to answer that question.

In one of my previous posts, I mentioned that typical epidemiological model predict exponential growth in the early phases of an epidemic (in the absence of further importation of new cases and if the transmission rate remains constant). The previous posts aimed to investigate the degree to which transmission rates were changing in Australia (and elsewhere) as control measures were implemented.

Now that Australia has a relatively stable (and low) rate of transmission, we still expect the number of active cases to change exponentially but now it will be exponential decline because physical distancing is helping to control transmission of coronavirus.

When cases increase exponentially, the rate of increase gets faster with time. In contrast, the rate of decline gets slower with time under exponential decline. Over the last week or so, the number of active cases in Australia has declined at a rate of about 15% per week. That is, the number of active cases now is about 85% of the number that existed one week ago.

If that rate of decline continues, then in two weeks we would expect the number of cases to decline to 85% × 85% = 73% of the number now. Over another two weeks (four weeks in total), we would expect the number of active cases to decline to 53% of the number now (85% × 85% × 85% × 85%). You will note how the rate of decline gets slower and slower.

Change in the number of cases when increasing exponentially at a rate of 15% per week (red), and when decreasing exponentially at 15% per week (blue). Increases accelerate, while decreases decelerate with exponential dynamics.

 

It takes about 19 weeks (almost 4.5 months – i.e., late August) to get the number of active cases to one twentieth of the number now with a decline rate of 15% per week. We currently have about 2600 cases in Australia, so this projection would suggest we will have about 130 cases by late August. That is approximately the number of cases that existed a little over a month ago.

This might seem a little disheartening. The increase in occurrence that we have seen in the virus in about a month (even with effective physical distancing for much of that time) will take almost 5 months to eliminate. This emphasises the difficulties faced when managing coronavirus in Australia.

Of course, we don’t have much data to estimate the long-term rate of decline in the number of active cases; the number of active cases in Australia has only been declining for a couple of weeks, so our estimate of the rate of decline is uncertain. It is possible the incidence of COVID-19 cases might decline faster than 15% per week. Or cases might decline less quickly.

Nevertheless, this analysis suggests to me (and as foreshadowed by the Federal and State Governments of Australia), that management of coronavirus in Australia (and elsewhere) is a long-term proposition.

*very few other countries.

Local transmission rates

Last update (9 April):

In my previous post, I examined country-level trends in transmission rates of the coronavirus. Most of Australia’s cases have been imported from international arrivals (typically returning travellers). Notwithstanding some notable bloopers, controlling importation of coronavirus is relatively straightforward, especially with enforced quarantine of returning citizens and residents. Control of imported cases of COVID-19 are likely responsible for much of the decline in Australian infection rates seen in my previous post.

From here, control of COVID-19 in Australia will depend on limiting local transmission. The data from covid19data.com.au distinguishes between sources of infection, and also allows us to compare states. If we exclude international sources, and look at new infections, the local transmission rates appear to be declining in New South Wales and Victoria (Australia’s largest states with the most cases). In this, I again account for an incubation period by calculating the rate of new infections as a proportion of cases 5 days previously (excluding international sources, so the “under-investigation” cases are counted as local sources).

We see an apparent drop in local transmission around 26 March, and possibly a further one around 30 March. The drop around 26 March most likely reflects policy changes implemented from 16 March (no gatherings of more than 500 people).

The next major policy changes occurred on 19 March (indoor spaces with at most 1 person per 4 square metres), and 22 March (closure of restaurants, pubs, cafes, etc). The drop at around 30 March might reflect these changes, or it might simply be noise in the data.

Evidence of the impact of further restrictions (no more than two people together in public, 29 March) probably won’t be clear for at least another week or so.

Also, while the new infection rate has declined, the total number of cases has increased, such that the two roughly balance – the number of new cases per day has not been trending up or down over the last week, albeit with some variation.

To get on top of the epidemic, this number of new cases would ideally decline over the next week or two. Let’s keep an eye on it.

 

Progress on COVID-19

Update 7 April:

New data on recoveries for Australia – we now have more recoveries than new cases.

Spain’s rate of new infections is almost below the rate of recoveries.

Update 4 April (latest graphs):

Note: There was an error in the calculation of recovery rates. This is fixed in this latest update (the original post retains the errors).

USA remains a major worry, with the new infection rate remaining well above 0.2; this probably needs to be needs to be well below 0.1 for the number of existing cases to decline. There are further good signs in Australia. Spain is better than it was, but still a worry.

 

 

 

Original post:

You must be tired of exponential trajectories of COVID-19 by now. Well, this blog post isn’t about that directly, but it addresses the same topic – the trajectory of COVID-19. But I hope this is a little more useful for illustrating progress (or lack thereof) in managing the epidemic in each country.

The exponential trajectory of the number of cases is useful as a point of comparison. It shows what would happen in the early stages of a COVID-19 epidemic with only local transmission at a constant rate per infected person (ignoring importation, no controls on spread, and with even mixing of people in a population). We could compare an observed trajectory with that, and hope to see the “curve flattening”. This is the basis of Ben Phillips’ “coronavirus forecaster“.

However, spotting small bends in exponential curves is difficult to do by eye – is that ever-increasing curve starting to flatten? Further, the data that typically get plotted are the number of existing cases, the growth of which includes new cases minus recoveries.

A useful approach is to examine the new cases each day. For COVID-19, the number of local transmissions should be proportional to the number of infected people (in the early stages of the epidemic, when the vast majority of people are still susceptible).  It is this proportionality that leads to the expected exponential growth (when the ratio of new to infected cases remains constant).

Further, given an incubation period of five days (roughly that of COVID-19), the number of new cases should be proportional to the number of cases that existed five days ago. In this situation, the ratio of new cases to existing cases from five days ago is the key parameter. I will call this ratio the “rate of new cases”. If we look at that rate, you can get a better picture of the trajectory of the epidemic.

In an epidemic, there will be new cases each day until the disease is almost eradicated. Progress in controlling the disease will be seen by reducing the number of new cases until they are fewer than the number of recovering cases. Therefore, a useful benchmark for assessing new cases is the number of recovering cases per day. To make the number of recovering cases comparable to new cases, we should also scale recoveries by the same factor – the number of existing cases from five days ago.

So, what do these figures look like for some countries?

Australia’s rate of new cases (per existing case) reduced noticeably around 25 March. But more work needs to be done to get the rate of new cases below the rate of recoveries. In South Korea, for example, where the number of existing cases is declining, the rate of new cases is below 0.05. Australia is a long way from that at the moment.

A reduced rate of new cases will arise from changes in behaviour of people and control of importation through measures such as physical distancing and quarantining recent arrivals. Responses in the data will lag these initiatives, with the lag corresponding to the incubation period (about 5 days for COVID-19) plus the time it takes to process tests (which varies from less than a day to several days).

Progressively more aggressive approaches to controlling importation and spread of COVID-19 commenced in Australia from 15 March. It seems likely that the reduction in the rate of infection around 25 March reflects those actions.

The USA is in a worse position than Australia. While the rate of new cases has declined, it remains much higher than the recovery rate, and much higher than Australia’s rate. This suggests the epidemic in the USA will continue to worsen for some time yet.

The situation in Spain has improved, but it is still bad. The rate of new infections has dipped to be similar to Australia, but with many more existing cases, the number of new cases per day remains extremely high (and much higher than in Australia).

This analysis has a few idiosyncrasies. Ideally it would distinguish between local transmissions and importation – the former is much more of a worry if importation can be controlled. However, the daily data I used (from the Wikipedia pages of each country, in case you are interested) only publishes total cases.

Secondly, the analysis doesn’t account for undetected cases. If the proportion of undetected cases remains constant, then this isn’t a problem when calculating the rates (the detection rates cancel when dividing). However, the spike in rates in the USA up until 20 March most likely reflects an increased rate of detection of cases through increased testing rather than an increase in transmission.

So this tells us the USA has hard times ahead, and further limits to transmission seem required to control the epidemic. The rate of infection in Spain and Australia has declined, and it will be interesting to see the extent to which these rates decline further in response to the measures put in place by governments (and hopefully followed by their citizens – people, it’s not time to go to the beach!!!)

But in all three countries examined, the trajectory of the epidemic is clearly still increasing. Please stay at home as much as possible.

Edit: I also did a separate analysis for California and NY State. The big increase in the rate of new infection in NY State is probably a testing artifact. Rates in both states need to reduce substantially.

 

 

Wentworth 2018 – Phelps dominates the preference flows

Update (23 Oct): I’ve included a graph of the estimated preference flows (after a slight tweak to the analysis).

With the Australian Electoral Commission (AEC) counting of Wentworth 2018 nearing its end, we can look at the preference flows to David Sharma and Kerryn Phelps. For this analysis, I took the data from the 35 regular polling places (booths; not pre-polls, hospital teams, or postals), and examined how many of the primary votes for the other candidates went to Sharma versus Phelps when the preferences were distributed.

For example, for the Bondi Surf booth, Sharma got 375 primary votes and 449 votes in total after the flow of preferences (i.e, 74 preferences from voters for the other candidates). For the same booth, Phelps got 459 primary votes and 779 votes in total (320 preferences other voters – more than 4 times the number of preferences as Sharma).

The flow of preferences to Phelps was not as strong in other booths. For example, in Rose Bay Central, Phelps only got about twice as many preferences as Sharma. The differences in the flow of preferences can be largely explained by voters tending to have different voting patterns in the different booths. Voters at the Bondi Surf booth had a greater propensity to vote Green and Labor as their first preference (15% and 10.5% of voters). In Rose Bay Central, candidates for these parties only got 5% of  the primary vote.

In a single electorate, it might be reasonable to assume that voters who preference a particular candidate first will have a similar tendency to preference the two leading candidates. That is, Greens voters might tend to preference Phelps over Sharma. While voters for another candidate might tend to preference in a different way.

With the AEC data available, we can build a statistical model to estimate the degree to which voters for each of the candidates preferenced Sharma ahead of Phelps. This can be analysed as a basic regression model. We use the number of primary votes to each candidate in each booth as the explanatory variable (ignoring Sharma and Phelps because they don’t receive preferences from their primary votes), and the number of preferences received by Sharma as the response variable. The coefficients for this regression estimate the proportion of voters for each candidate who preferenced Sharma over Phelps.

Some candidates received very few votes, so it is difficult to estimate the preference flows from voters for those candidates using this method. However, it is clear that voters for the Greens, Labor and the independent candidate Licia Heath tended to preference Phelps (Phelps was estimated to receive about 80-100% of preferences from these voters).

 

 

Estimated preference flows to Sharma versus Phelps for voters for other Candidates in the Wentworth by-election. The dot is the estimate, with the bars representing the 95% credible estimate.

In contrast, voters for the other independent candidate Angela Vithoulkas appeared to flow towards Sharma; the analysis estimated Sharma won most of the preferences from Vithoulkas voters.

However, the Greens, Labor and Heath won the vast majority of primary votes that did not go to Sharma or Phelps, so with those voters preferencing Phelps over Sharma, Phelps dominated the preference battle, winning by 4 to 1. At this point it seems to be enough to get her over the line.

Finally as an aside, the fit of the model is quite good. The correlation between the number of preferences received by Sharma and the fitted value in the statistical model is 0.99.

---

For those interested, here are the data and BUGS code that I used in the analysis:

model
{
  for (i in 1:35) # for each booth
  {
    m[i] <- b[1]*CALLANAN[i] + b[2]*KANAK[i] + b[3]*HIGSON[i] + b[4]*GEORGANTIS[i] + b[5]*MURRAY[i] + b[6]*FORSYTH[i] + b[7]*ROBINSON[i] + b[8]*GUNNING[i] + b[9]*VITHOULKAS[i] + b[10]*DOYLE[i] + b[11]*LEONG[i] + b[12]*HEATH[i] + b[13]*KELDOULIS[i] + b[14]*DUNNE[i]

    Sharma[i] ~ dnorm(m[i], p)
  }


  for (i in 1:14)
  {
    logit(b[i]) <- d[i]
    d[i] ~ dnorm(0, 0.1) # I(0,1)
  }
  p ~ dgamma(0.001, 0.001)
}

#Initial values for the MCMC
list(p=1, d=c(0,0,0,0,0,0,0,0,0,0,0,0,0,0))

#Data
Total[] Sharma[] CALLANAN[] KANAK[] HIGSON[] GEORGANTIS[] MURRAY[] FORSYTH[] ROBINSON[] GUNNING[] SHARMA[] VITHOULKAS[] DOYLE[] LEONG[] HEATH[] KELDOULIS[] PHELPS[] DUNNE[]
515 120 10 169 12 3 192 5 2 13 1032 36 15 10 29 7 724 12
165 37 7 40 5 1 71 0 1 7 635 11 3 6 9 2 282 2
399 76 4 161 5 3 147 0 1 5 333 15 10 13 20 6 360 9
1370 243 17 614 15 4 468 4 14 14 1087 42 26 32 75 18 1300 27
182 35 6 82 3 0 64 1 1 3 188 5 1 4 10 2 256 0
514 106 12 150 12 5 228 2 5 6 657 16 10 20 31 4 534 13
473 104 11 158 10 6 192 2 4 15 557 12 11 16 18 10 418 8
370 81 8 127 8 3 128 3 5 4 400 15 8 9 36 6 380 10
248 49 1 89 3 0 96 1 2 7 257 7 5 5 17 7 248 8
394 74 7 183 4 1 129 1 0 5 375 12 3 10 25 9 459 5
596 95 7 204 12 0 230 0 3 3 430 6 4 12 100 4 489 11
539 83 5 175 12 3 237 2 1 1 480 7 15 10 62 3 599 6
268 38 6 99 4 1 112 0 1 3 189 3 1 11 14 6 294 7
365 69 9 110 4 0 170 0 1 9 404 8 2 9 37 1 355 5
261 72 5 82 13 0 90 4 5 10 887 15 6 3 15 7 487 6
245 36 3 86 3 0 97 0 2 1 162 12 3 4 27 4 320 3
507 125 9 138 16 0 209 4 4 15 1665 31 22 6 34 5 835 14
78 14 1 28 1 1 30 0 1 0 160 4 2 2 4 2 100 2
224 72 4 63 5 5 80 2 0 6 881 13 8 12 16 3 344 7
170 37 2 49 3 1 75 1 1 2 351 2 1 5 21 7 200 0
539 77 5 176 6 1 217 2 2 8 440 8 7 18 69 18 783 2
333 43 3 109 5 1 133 1 1 1 260 13 8 13 34 7 440 4
611 118 17 189 16 3 229 3 4 11 771 13 11 13 82 11 907 9
615 107 14 192 10 0 237 4 4 9 573 26 20 13 60 13 756 13
350 66 4 99 7 2 151 0 2 3 306 13 10 19 28 1 366 11
750 148 11 197 16 2 375 3 6 8 766 25 15 23 44 9 708 16
357 97 10 99 10 1 125 1 5 6 876 32 7 5 38 6 547 12
196 60 7 56 10 2 54 3 4 8 579 20 6 4 16 2 304 4
84 26 5 21 1 0 30 0 1 4 317 13 1 1 5 1 99 1
310 68 5 115 8 1 106 5 2 6 1007 19 6 8 20 3 437 6
178 36 4 65 10 0 60 4 1 2 352 4 3 3 16 3 234 3
625 138 8 199 15 4 260 5 4 9 545 27 12 11 51 7 583 13
268 54 6 90 1 0 103 1 1 6 344 11 8 8 24 5 325 4
341 71 6 92 11 2 127 3 4 5 584 16 3 8 48 7 456 9
216 34 4 50 9 0 101 0 2 1 340 7 0 2 31 6 327 3
END

Batman By-Election 2018

It’s on in Batman. And the result might well depend on what happens north of the Hipster-proof Fence, a term coined (by my wife) to help describe the voting patterns that flipped in the vicinity of Bell St.

With David Feeney resigning from Federal Parliament due to unresolved issues regarding his citizenship, a by-election for the federal seat of Batman will be held. Batman was an interesting race in 2016, with the ALP narrowly beating the Greens. But with the Greens winning a recent state by-election in Northcote, which covers the southern half of the Batman electorate (south of the Hipster-proof Fence), the 2018 by-election promises to be even more interesting.

One feature of the 2016 federal election was the north-south gradient in votes, both in terms of the 2 candidate-preferred vote, and the swing from the 2013 election. In both cases, the ALP did much better north of the Hipster-proof Fence. Indeed, the ALP had swings toward it in some of the northern-most booths. If the ALP had suffered the same swings north of Bell St as they did further south, the Greens would have won comfortably in 2016.

The result of the Northcote 2017 state by-election closely matched the outcome of the 2016 federal election if one examines the outcomes at individual booths. The consistent swing to Greens in 2017 simply mirrored what had occurred a year before. This is seen in both the 2-candidate-preferred vote, and the swing from the previous election.

Two-candidate preferred vote to the ALP in each booth for the 2016 federal election in the seat of Batman and the 2017 Northcote by-election as a function of latitude. The 2017 result matches that seen in 2016.

Swing to the ALP in each booth for the 2016 federal election in the seat of Batman and the 2017 Northcote by-election as a function of latitude. The 2017 result matches that seen in 2016, with a solid win to the Greens in the south.

While the swings in 2017 and 2016 were quite similar for corresponding booths (above), you might notice that the three northern-most booths in the 2107 state by-election had larger swings away from the ALP than the same booths in 2016 federal election. That will make the ALP nervous, and the Greens hopeful.

While both parties will aim to sway voters in the south, the outcome of the 2018 federal by-election most likely hinges on voting patterns north of Bell St. If the Greens can win back northern voters who apparently turned away from them in 2016 while retaining voters in the south, the Greens might be one of the few winners out of the citizenship saga that has engulf federal parliament.

Interesting times!

When does research help environmental management?

Think of the case where a manager needs to decide which action to take to stop a species  declining, or to eradicate a pest, or to increase sustainable harvest levels. It is rare in environmental management to know, with certainty, which action to take.

In response to such uncertainty, a scientist might recommend that the manager should trial different management actions, and use the results of that trial to decide on the best course of action. Such trials can certainly improve subsequent management.

But research costs money – money that might have been better put toward management. Further, even trialling two options means that, almost inevitably, one of the trialled actions will be inferior to the other. So opportunity costs are likely to exist in almost any trial, even if the research itself were cheap.

The trade-off between learning and doing lies at the heart of adaptive management. My recent paper led by Alana Moore addresses this trade-off, using the simplest formulation of the problem that we could muster. In that case we only considered resolving a choice between two management options. Our hope was to gain greater insight into the question of the circumstances in which research assists environmental management.

The answers surprised us in several instances. One surprise was the threshold behaviour that existed in many parameters. For example, as the expected difference in performance of the two management options increases, the optimal effort to spend on experimentation increases, but only up to a point. Once the threshold difference in performance is sufficiently large, the optimal level of experimentation declines to zero.

This threshold makes some intuitive sense; once we are relatively sure of the difference in performance, then we shouldn’t bother with an experiment to evaluate that. However, prior to reaching that threshold, the optimal effort to spend on the experiment increases with the expected difference in performance; that is somewhat counter intuitive. Other thresholds also exist.

Another surprise is that circumstances in which the investment in management trials is greatest do not necessarily correspond to the circumstances in which the benefit of trials is the greatest. Cases exist when relatively modest investment in trials can lead to large expected management gains. And counter-cases exist in which large investments in trialling options is the best thing to do, but the benefits of those trials are quite small.

I love this sort of modelling – simple models leading to somewhat counter-intuitive insights. You can read about them more in the paper, or in a previous blog post I wrote regarding a talk I did on this topic.

The paper is:

Moore, A. L., Walker, L., Runge, M. C., McDonald-Madden, E. and McCarthy, M. A. (2017). Two-step Adaptive Management for choosing between two management actions. Ecological Applications. doi:10.1002/eap.1515

Swinging on the Hipster-proof Fence

The “Hipster-proof Fence” is an evocative name for Bell St, which tended to divide booths in Batman that were won by the ALP in the 2016 federal election from those won by the Greens. A similar pattern was seen in neighbouring Wills electorate.

While I like the name “Hipster-proof Fence”, “The Tofu Curtain” is probably more accurate because Bell St is not a sharp barrier to the voting trend; the two-candidate-preferred vote trends across the entire north-south gradient. Bell St just happens to be where the vote approximately flips from one party to the other.

However, the geographic gradient in the swings is quite different between Wills and Batman. In Batman, Bhathal actually had some swings against her in the northern booths. In Batman, Bell St approximates the location where the swings change.

Swings to Feeney (ALP) for each booth in Batman. Negative swings represent swings to Bhathal (Greens). Booths are colour coded by the party that won the booth. The black line shows the latitude of Bell St at Merri Creek.

Bhathal had consistently strong swings in booths south of Bell St. Swings were much more variable north of Bell St, with strong swings to her at some booths, and strong swings away at others.

In contrast, Ratnam had consistently strong swings across the entire electorate of Wills. Her smallest swing occurred in the far north of the electorate, but so did her largest swing.

Swings to Khalil (ALP) for each booth in Wills. Negative values represent swings to Ratnam (Greens). Booths are colour-coded by the party that won the booth. The black line shows the latitude of Bell St at Merri Creek.

If Bhathal had extended her SoBe (South of Bell St) swing to NoBe, she would have won Batman. In contrast, Ratnam achieved large and consistent swings throughout Wills, but was simply coming from too far behind to win.

 

Simple Adaptive Management

This post gives some details of my speed talk at the SCB Oceania conference, which is in room P9 on Thursday 7 July at 11:50 as part of a session on conservation planning and adaptive management. We have submitted this work to Ecological Applications – a copy is available here, so please add to the peer review by giving us comments.

Every natural resource management agency seems to do (or at least claims to do) adaptive management, which seeks to use management and monitoring to learn about the system being managed, thereby improving future management. It is sometimes referred to as “learning by doing”.

Active adaptive management seeks to explicitly design management like an experiment, and entails extra costs. The experimental design and monitoring requires more resources than simply just managing the system. Further, if two management strategies are implemented at the same time, then inevitably one of them will be inferior.

The benefits of improved management in future can be weighed against these extra costs. And the optimal balance between these costs and benefits can be determined, thereby optimizing the design of adaptive management programs to maximize performance.

In my opinion, attempts to optimize adaptive management programs have been overwhelmingly disappointing. Firstly, the optimizations seem to only work on relatively small problems (but see Nicol and Chadès 2012). Secondly, each published optimization is different in fundamental ways from others, making it difficult to derive generalities across studies. And perhaps most depressingly, the benefits of optimizing adaptive management seem small – the optimizations typically only increase expected performance by a few percent at most.

The apparently minor benefits of adaptive management make me worry that science might be impotent; we go to all that effort to optimize the design, yet only get tiny improvements. Surely science is better than that!

So with Moore and a few other colleagues, we decided to examine optimal adaptive management to ask a few fundamental questions:

When is adaptive management most useful?
What drives optimal experimentation?
How much should be invested in experimentation?
How big are the benefits of adaptive management

To answer these questions, we set up the simplest possible adaptive management problem. We considertwo possible management options and two time steps. The first time step allows for possible experimentation, and the apparently best option is applied exclusively in the second time step.

Each option has an expected level of performance (which was uncertain), and we need to determine how much effort to expend on each option in the first time step. Each unit of management effort in the first time step is monitored so that its performance can be assessed. Each option has a per unit cost of implementation and a per unit cost of monitoring.

The monitoring data will be uncertain, so we will be more certain about the performance of each option as investment in each option in the first time period effort increases. However, increasing the level of effort allocated to each option in the first time period will decrease the resources available to spend in the second time step. Thus, when we invest more in the first time period, we can more reliably choose between options in the second time period, but we will have fewer resources to spend on the apparently best option. We face a trade-off!

While this formulation of adaptive management is as simple as we could devise, it is still somewhat complex. In total the model has 11 parameters plus the two control variables (the control variables were how much to allocate to each option in the first time period).

We show how the trade-off between learning and saving resources for acting later can be optimized, and the results have some interesting features. Firstly, various thresholds exist. For example, as the expected difference in performance of the two options increases, the optimal effort to spend on experimentation increases, but only up to a point.

Once the threshold difference in performance is sufficiently large, the optimal level of experimentation declines to zero. This threshold makes some intuitive sense; once we are relatively sure of the difference in performance, then we shouldn’t bother with an experiment to evaluate that. However, prior to reaching that threshold, the optimal effort to spend on the experiment increases with the expected difference in performance; that is somewhat counter intuitive.

Optimal level of experimentation for a particular set of parameter values. The optimal level of experimentation increases with the difference in the expected benefit of each option, but only up to a point after which it is best not to experiment at all.

Similar thresholds exist for the budget and the prior level of uncertainty in performance. However, while the optimal level of experimentation increases with the expected difference in performance, the biggest benefits of experimentation are realized when the expected performance of the two strategies are the same. Thus, the  greatest benefits of experimentation are realized under conditions that differ from when the optimal level of experimentation is greatest.

This work helps to illustrate some fundamental features of adaptive management. We also tie the results explicitly to the notion of expected value of sample information. And we  derive analytical solutions for the optimal level of experimentation for some special cases of parameter values.

While the paper is quite mathematically involved, the concept itself is quite straight-forward, and the results are very interpretable. I think it  is a very interesting study – see what you think. Please send us comments to help us improve the paper while it is being peer reviewed.

Moore AL, Walker L, Runge MC, McDonald-Madden E & McCarthy MA (in review) Two-step adaptive management for choosing between two management actions.

Preference flows in #IndiVotes 2106

Update (6 July 2016, 8:00 a.m.)

Kevin Bonham pointed me to this AEC table that indicates the flow of preferences for Nationals voters in three-cornered contests in 2013. It was 75.45% to the Liberals and 24.55% to the ALP. So the Liberals should not be surprised with the preference flow of 3/4 from Nationals to Mirabella in Indi.

The flow of preferences from Liberals voters to Nationals was 90.79% in 2013 – higher than the flow the other way, but even if the flow of Nationals preferences to Mirabella had been that high, McGowan would still be leading in Indi.

Original post

Liberal federal electorate chairman for Indi, Tony Schneider, reportedly said that many Nationals voters preferenced Cathy McGowan ahead of Sophie Mirabella in 2016. He said “The reason we have a Coalition is to put a Coalition member in Parliament, whether that’s Sophie or Marty, but now we don’t have either. A lot of National Party people need to have a good look at that.”

Well, we have a preferential voting system so voters can preference whomever they like. But that aside, we can look at the booth data to estimate the proportion of Nationals voters who preferenced McGowan, an independent, ahead of Mirabella, the Liberal candidate.

Using the method I described last week, I estimate the following flow of preferences to Mirabella from each of the other candidates:

0%  of  1498 votes from  LAPPIN, Alan James ( Independent )
0%  of  2724 votes from  O’CONNOR, Jenny ( The Greens )
64.5%  of  697 votes from  QUILTY, Tim ( Liberal Democrats )
28.1%  of  7404 votes from  KERR, Eric ( Australian Labor Party )
41.2%  of  376 votes from  DYER, Ray ( Independent )
73.6% of  13822 votes from  CORBOY, Marty ( The Nationals )
0 % of  1538 votes from  FIDGE, Julian ( Australian Country Party )
92.1%  of  937 votes from  FERRANDO, Vincent ( Rise Up Australia Party )

This model provides a good fit to the data:

Observed versus fitted number of preferences flowing to Mirabella in each of Indi’s booths.

Based on that analysis, I estimate that three quarters of Nationals voters preferenced Mirabella ahead of McGowan – and a total of approximately 3650 National voters preferred McGowan. Counting is still continuing, but McGowan leads with 41,548 votes to 34,489. If all the Nationals voters had preferenced Mirabella, we would now have another neck-and-neck race rather than a safe win to McGowan.

I’m sure the Liberals will be disappointed they couldn’t inspire the Nationals voters to preference their candidate. But I’m not sure that castigating them will help win them over for the next election, however.