Scorecards

​

Scorecard Structure

• Scorecards are a list of criteria and metrics that add up to 100%
• A criteria is an AI prompt that analyzes the roleplay transcript to provide a score
• A metric measures paraverbal metrics like word speed, filler words, with special analysis tools
• Both criteria and metrics can be re-used across multiple scorecards

​

Building a Scorecard

You have three options to build a roleplay:

• Build from scratch - Self-select your own criteria and metrics
• Start with a template - Use one of our pre-built scorecards
• Build with AI - Use AI to generate new criteria from a text prompt

​

Editing a Scorecard

Once your scorecard is made, you can add, edit, and remove criteria and metrics to improve scorecard reliability and accuracy.

​

Adding Criteria

You can add criteria and metrics with the ”+/- criteria” and ”+/- metrics” options at the bottom of the scorecard.

​

Creating New Criteria

There are a couple core parts to the criteria: the criteria title, folder, question type, and prompt.

The question types ordered according to most commonly used include:

• Yes/No Questions: Simple binary evaluation
• Compound Questions: Combination of multiple yes/no criteria
• Range Questions: Score on a scale of 1-10
• Open-ended Questions: For gathering general feedback

Here are a couple tips for writing reliable and accurate criteria:

1. Use the AI generator to the right of the “Prompt” field to help with generation
2. Avoid general statements like “speak with confidence,” “ask good questions,” etc.
3. Be hyper-specific in what you are looking for
4. Provide passing examples to help the AI understand

The general structure for a well built criteria is:

• “Did the rep… [SPECIFIC ACTION]? For example, did they say something along the lines of [DIRECT QUOTE]“

​

Removing and Adjusting Criteria Weights

• You can remove criteria with the red trashcan icon when you hover over the criteria
• You can adjust the criteria weights by changing the number next to the title of the criteria