Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.replay.sale/llms.txt

Use this file to discover all available pages before exploring further.

​
Scorecard Structure

  • Scorecards are a list of criteria and metrics that add up to 100%
  • A criteria is an AI prompt that analyzes the roleplay transcript to provide a score
  • A metric measures paraverbal metrics like word speed, filler words, with special analysis tools
  • Both criteria and metrics can be re-used across multiple scorecards
Every new company comes with 100 pre-built criteria

​
Building a Scorecard

You have three options to build a roleplay:
  • Build from scratch - Self-select your own criteria and metrics
  • Start with a template - Use one of our pre-built scorecards
  • Build with AI - Use AI to generate new criteria from a text prompt

​
Editing a Scorecard

Once your scorecard is made, you can add, edit, and remove criteria and metrics to improve scorecard reliability and accuracy.

​
Adding Criteria

You can add criteria and metrics with the ”+/- criteria” and ”+/- metrics” options at the bottom of the scorecard.

​
Creating New Criteria

There are a couple core parts to the criteria: the criteria title, folder, question type, and prompt. The question types ordered according to most commonly used include:
  • Yes/No Questions: Simple binary evaluation
  • Compound Questions: Combination of multiple yes/no criteria
  • Range Questions: Score on a scale of 1-10
  • Open-ended Questions: For gathering general feedback
Here are a couple tips for writing reliable and accurate criteria:
  1. Use the AI generator to the right of the “Prompt” field to help with generation
  2. Avoid general statements like “speak with confidence,” “ask good questions,” etc.
  3. Be hyper-specific in what you are looking for
  4. Provide passing examples to help the AI understand
The general structure for a well built criteria is:
  • “Did the rep… [SPECIFIC ACTION]? For example, did they say something along the lines of [DIRECT QUOTE]“

​
Removing and Adjusting Criteria Weights

  • You can remove criteria with the red trashcan icon when you hover over the criteria
  • You can adjust the criteria weights by changing the number next to the title of the criteria
Use the “Distribute Weights” button to automatically balance the scoring criteria evenly.