Assessment Toolkit

The Assessment Toolkit enables you to capture observed behaviours during a live crisis exercise and translate them into a structured assessment of capabilities. It works by defining capabilities and observable behaviours in advance.

During the exercise, observers record those behaviours as and when they occur. The results are then mapped and visualised in a bar chart to support debrief and feedback.

To find out more about Observers, visit the page below:

Some definitions of who does what

Concept

The Assessment Toolkit allows multiple Observers to make quantitative assessments about the behaviour of the training audience. Beforehand, the exercise designer creates the assessment schema which defines what will be measured and how within an exercise:

	Description	Example
Capabilities	The skills you want players to have	Good communication, strategic foresight, leadership
Observations	The specific, observable behaviours that demonstrate a capability	Empathy (may demonstrate effective communication), clear delegation (may demonstrate leadership)

Therefore, the designing phase of the Assessment Toolkit determines how to manage multiple real-time observers scoring players differently.

Designing

How-to

To get started, name your Assessment Schema and then click Add Capability.

1

Define capabilities

Each capability becomes a category that Observers will assess players against during the exercise e.g. communication.

Calculation Method

1. Average (AVG) (default)

Calculates the average of all observer ratings.

1. Sum (SUM)

Calculates the sum of all observer ratings.

1. Maximum (MAX)

Calculates the maximum observer rating.

1. Minimum (MIN)

Calculates the minimum observer rating.

2

Add observations

Now you can add observations to your capability:

In the example below, empathy is an indicator of/evidence for effective communication skills.

The Assessment Toolkit lets you decide which inputs to use and how they should be combined to calculate the player's capability:

1. Star rating (default)

Allows observers to rate performance on a 3-star or 5-star scale.

1. Checkbox

Allows observers to confirm whether a specific behaviour occurred.

1. Open Text

Allows observers to leave written comments.

As you build your schema, observations appear as cards. You can toggle between capabilities in the right-hand menu to see which observations are linked to each one (highlighted by a black border):

When your schema is ready, get in touch with us at [e-mail] to upload it to your space.

---

Running

Once your schema has been uploaded, observers can begin assessing participants during the live exercise.

1

Facilitator Dashboard

In the Facilitator Dashboard, navigate to Review > Assess. Here you will find your participants where observers can rate them against the capabilities defined in your schema.

If multiple observers are contributing, their ratings will be combined using the selected calculation method.

It's important to mention that observers might not be observing all players at once. This is why you'll find a toggle under Observing, as shown in the screenshot below. Untoggle to remove the player from the observer's assessment.

Observers should remember to press 'Save changes' in the right-hand corner to submit their player capability ratings.

2

View capability results

Navigate to the Capabilities tab to view results.

Capability performance is displayed in a visual bar chart, allowing you to:

• See strengths and gaps at a glance

• Compare participants

• Support structured debrief

You can also export your observations as an Excel document for reporting: