Assessment Toolkit

The Assessment Toolkit enables you to capture observed behaviours during a live crisis exercise and translate them into a structured assessment of capabilities. It works by defining capabilities and observable behaviours in advance.

During the exercise, observers record those behaviours as and when they occur. The toolkit maps and visualises the results in a bar chart to support debrief and feedback.

Concept

The Assessment Toolkit allows multiple observers to make quantitative assessments about the behaviour of the training audience. Beforehand, the exercise designer creates the assessment schema which defines what will be measured and how within an exercise:

What is measured	Description	Example
Capabilities	The skills you want players to have	Good communication, strategic foresight, leadership
Observations	The specific, observable behaviours that demonstrate a capability	Empathy (may demonstrate effective communication), clear delegation (may demonstrate leadership)

Therefore, the designing phase is where you decide how multiple observers' scores get combined.

Designing

How-to

To get started, name your Assessment Schema and then click Add Capability.

1

Define capabilities

Each capability becomes a category that observers will assess players against during the exercise e.g. communication.

Calculation Method

1. Average (AVG) (default)

Calculates the average of all observer ratings.

1. Sum (SUM)

Calculates the sum of all observer ratings.

1. Maximum (MAX)

Calculates the maximum observer rating.

1. Minimum (MIN)

Calculates the minimum observer rating.

2

Add observations

Now you can add observations to your capability:

In the example below, empathy is an indicator of/evidence for effective communication skills.

The Assessment Toolkit lets you decide which inputs to use and how they should be combined to calculate the player's capability:

1. Star rating (default)

Allows observers to rate performance on a 3-star or 5-star scale.

1. Checkbox

Allows observers to confirm whether a specific behaviour occurred.

1. Open Text

Allows observers to leave written comments.

Reviewing and editing your schema

As you build your schema, observations appear as cards. You can toggle between capabilities in the left-hand menu to see which observations are linked to each one (highlighted by a black border).

Click the ✏️pencil icon to make any changes to observations.

If you want to change which capability an observations belongs to:

1. Unlink by clicking the slashed 🔗link icon on the observation card

2. Select the intended new capability in the left-hand menu

3. Link the observation by clicking the 🔌plug icon

---

Running

Uploading your schema

Find all saved schemas under My Assessments. Then click Upload and select your space.

Now from your Editor, publish your exercise to the correlating space. Once the exercise starts, observers can begin assessing participants.

Running your schema

In the Facilitator Dashboard, navigate to Review > Assess. Here you will find your participants where they can be rated against the capabilities defined in the schema.

1

Submit observations

If multiple observers contribute, the toolkit combines their ratings using the selected calculation method.

⚡It's important to mention that observers might not be observing all players at once. This is why you'll find a toggle under Observing.

Observers should remember to press 'Save changes' in the right-hand corner to submit their player capability ratings.

2

View capability results

Navigate to Capabilities to view results.

Capability performance is displayed in a visual bar chart, allowing you to:

• See strengths and gaps at a glance

• Compare participants

• Support structured debrief

You can also export your observations as an Excel document for reporting which contains each players' scores and open text comments.