Policy evaluation · University of Bath · 2025
School Closures and the Attendance Gap
A critical evaluation of Tomaszewski et al. (2022) on COVID-19 and attendance in Tasmania
Summary
For the Policy Evaluation unit at Bath, I critically evaluated a quasi-experimental study asking whether Tasmania's 2020 school closures widened the attendance gap between students from high and low socioeconomic backgrounds. I unpicked the study's causal design, tested its assumptions against the evidence, and set out what its findings can and can't support for policy. It received a mark of 90, my highest at university.
The study
School closures during the pandemic affected nearly 1.6 billion students worldwide, and early evidence suggested disadvantaged students faced the steepest challenges. In Australia, students from low and high socioeconomic backgrounds were already about three years apart in attainment.
Tasmania closed its schools for only 30 to 40 days. The authors argue this makes it a useful test case: if even a short closure widened the gap, it gives a minimum, lower-bound estimate of the impact of longer closures elsewhere. The authors used attendance, a key precondition for learning, as their outcome, and administrative data on 14,312 students in Years 10 to 12.
| Method | Difference-in-difference-in-differences (DDD), comparing the change in the high–low SES attendance gap across two cohorts |
|---|---|
| Covid cohort | Attendance in June–July 2019, then June–July 2020, straight after schools reopened |
| Pre-Covid cohort | The same periods one year earlier, 2018 and 2019, never exposed to closures |
| Finding | The gap widened for Years 10 and 11 (in Year 10, from 0.14 to 0.21), driven by falling attendance among low-SES students and those from jobless households. Year 12 showed no significant change. |
My evaluation
The design is a sensible choice: closures applied to everyone at once, so there was no randomisation, cut-off or untreated group to exploit. I compared it against five alternative methods before turning to four problems that limit how far its findings can be trusted.
-
Untested parallel trends
Risk: the effect may be overstated
- The method assumes SES groups' attendance would have moved in step without Covid, but the authors never checked
- Their longitudinal data made pre-trend comparisons and placebo tests on earlier cohorts straightforward
- Without them, the widening gap could reflect trends that predated the pandemic
-
Shocks that came with Covid
Risk: the effect may be overstated
- Job losses, financial stress and higher infection rates all fell hardest on disadvantaged households
- The design can't separate these from the closures themselves
- No synthetic control was possible, because no group had the shocks without the closures
-
SES treated as fixed
Risk: the effect may be understated
- 870,000 Australians lost their jobs in the first month of the pandemic
- Families who moved down the scale stayed in higher SES groups, shrinking the measured gap
- Policymakers could underestimate how much support disadvantaged students need
-
Mechanisms left untested
Risk: policy can't be targeted
- Digital exclusion, family stress and reduced academic support were proposed but never measured
- Each points to a different fix, from broadband access to counselling
- Causal mediation analysis or qualitative work with families could show which matters most
Smaller points I raised
Peer spillovers break the assumption that one student's outcome is unaffected by others', so the estimate captures the total effect of closures, not just the direct one. Up to 21% of baseline cases were missing, with no test of whether they were missing at random. And because Year 10 had 40 days of remote learning against 30 for Years 11 and 12, the authors could have tested a dose-response effect.
What it means for policy
Even a short closure widened the gap for disadvantaged students. The lack of an effect in Year 12 suggests exam stakes kept students engaged, which points to stronger incentives for attendance in earlier years.
Recovery shouldn't stop at academic catch-up. Rebuilding routines, relationships with teachers and a sense of belonging matters most for students at risk of disengaging, alongside digital access and digital skills for families. Future evaluations should pair administrative data with surveys and qualitative research, so policy can respond to why gaps open, not just that they do.