By Thomas Appleyard
Program evaluation informs practitioners, funders and researchers on the efficacy of interventions. Standard program evaluation methods emphasise desired outcomes and pre-identified criteria for success. Program evaluators stress establishing SMART goals: specific, measureable, agreed upon, realistic and time-bound.
These standard program evaluation methods do not apply well for emergency responders who must thrive in a volatile, unpredictable, complex and ambiguous (VUCA) environment.
What would SMART goals have looked like on 9/11? Was undertaking an unplanned maritime evacuation of 500,000 people off Manhattan Island within nine hours a realistic goal? Was establishing Union Square as the site for New Yorkers to grieve and to share information and food part of an agreed-upon goal? (No – the City shut this operation down after several days).
A problem with SMART goals for emergency managers is they limit possibilities. Here is the goal-setting language effective emergency responders used on 9/11:
- “Please do what you can for these people. Come out, lend a hand, anything you can do will be accepted.” – Gander Newfoundland Mayor Claude Elliott
- “We don’t know what we’re gonna see, we don’t know what we’re gonna do, but we’re gonna go.” – tugboatman Ken Peterson
- “You ready? Okay. Let’s roll.”– United 93 passenger Todd Beamer
Instead of squeezing emergency response efforts into standard program evaluation methods, here are two approaches to program evaluation emergency responders can take:
- Know what you are good at and evaluate that.
- Know the competencies required to thrive in a VUCA world (e.g., initiative, improvisation, empathy, creativity) and evaluate them.