Skip to main content

Building your first roleplay bot in Verity

Written by Richard Lombos

A bot is the AI character your participants practise a conversation with. It could be a sceptical procurement manager pushing back on price, an underperforming team member receiving feedback, or a customer threatening to churn. You describe who that character is and what situation they're in, and Verity brings them to life in text, voice or video.

A scorecard is the list of things you want the participant to actually do in that conversation. It's how Verity grades the practice and gives feedback.

Together they make one practice experience. This guide takes you through both, from an empty screen to a bot you've tested yourself. You don't need any technical background — if you can describe a difficult conversation out loud, you can build a bot.

Time needed: about 60-90 minutes for your first bot, not including testing. Later ones take 20-30 minutes.

What you need before you start

  • Access to Bot builder. If you can't see it in the left-hand menu, you don't have builder permissions yet — ask your Verity admin.

  • A clear idea of the conversation. One specific moment, not a whole meeting. "Handling a pricing objection" works. "Being a better salesperson" doesn't.

  • A scorecard — either one that already exists, or one you'll create. You can start a bot without one and add it later.

  • Optionally, a knowledge repository. This is a folder of your own documents (product guides, playbooks, pricing sheets) that keeps the bot factually consistent with your business. Without one, the bot will invent plausible-sounding details.

Which should I build first, the bot or the scorecard?

Most experienced builders decide the scoring criteria first, because "what should the participant do?" is the same question as "what should the bot make them do?". But you can build in either order — a bot will happily sit in draft until you attach a scorecard.

If this is your first time, we suggest building the bot first (so you learn the interface on something concrete) and then the scorecard, which is covered in the second half of this article.


Part 1 — Building the bot

Finding your way around the bot library

Open Bot builder from the left-hand menu. This is the library of every bot in your organisation.

Each row tells you who created the bot, which scorecard it uses, how many people have practised with it, the average score, and whether it's live. Use the filters across the top when the list gets long:

  • Show (type) — narrow to Text, Voice or Avatar bots

  • Statuses — Live or Draft

  • Scorecards — only bots using a particular scorecard

  • Time — last 7, 30 or 90 days, or all time

  • Standard bots toggle — switches between your own bots and the BTS standard bot library

A bot stays in Draft until someone is assigned to it. After that it shows as Live. Draft doesn't mean unfinished — it just means not enrolled to anyone

.

Tip: before building anything, browse the standard bots. Click the ⋮ menu on one and choose Duplicate to get a fully built bot you can edit. Starting from a working example is much faster than starting from a blank page, and it shows you what "good" looks like.

The ⋮ menu on any row also lets you Edit or Delete a bot.

Step 1 — Set up the basics

Click Create new in the top right. You'll see a short "Before we start" screen. Nothing here is permanent — you can change all of it later.

What would you like to call this roleplay

Up to 60 characters. Name it after the situation, not the skill, so people recognise it in a list. "Pricing objection — procurement manager" is clearer than "Negotiation practice v2".

What scorecard would you like to use

Pick an existing scorecard, or leave it blank and come back once you've built one.

Select knowledge you'd like to use to create the bot

Click Select repository and choose one or more knowledge repositories. This is required before you can continue.

A knowledge repository is a set of documents your organisation has uploaded — product information, pricing, methodology, policies. The bot reads from these so that when a participant asks "what's the renewal discount?", the answer matches reality instead of being made up.

Repositories are managed under Organization Setup → Knowledge. If nothing useful exists yet, ask your admin to create one, or pick the closest available and add the specifics by hand later in World Knowledge.

Would you like to score this roleplay?

On by default. When it's on, participants see ratings and an overall score. When it's off, they still get written feedback but no headline number — which is the better choice for a first, low-pressure practice where you don't want people fixating on the score.

Click Continue. Your bot now exists as a draft and you're inside the builder.

Step 2 — Build the character and the scene

The bot editor has two tabs: Create and Configuration. They are two routes to the same destination, and you can move between them freely.

Use the Create tab if…

Use the Configuration tab if…

This is your first bot, or you'd rather describe what you want in plain language and let Verity draft it

You know exactly what the character should say and do and have pre-written prompts, or you're fine-tuning something that already exists

Most people use both: build with Create, then polish in Configuration.

Route A — The Create tab (let the AI draft it)

The Create tab is a chat with Verity's building assistant. Describe your bot the way you'd describe it to a colleague — who the character is, what the situation is, how hard they should be to deal with — and the assistant asks follow-up questions, then writes the whole configuration for you.

For example, type:

"Sales reps practising how to handle a pricing objection from a sceptical procurement manager."

The assistant will come back asking what you're selling, roughly what it costs, and how tough the buyer should be. Answer in normal sentences. It then writes the backstory, the character's personality and the opening scene, and you'll see the

Configuration tab fill in live as it works.

Be specific in your answers. "A tough buyer" gives you a generic bot. "A procurement manager who has already been quoted 20% less by a competitor and has been told by their CFO to cut supplier spend this quarter" gives you a bot with something real to push back with.

Type / in the message box to see shortcut commands:

  • /clear — clears the chat history but keeps everything you've built. Useful when the conversation has wandered.

  • /compact — summarises the conversation so far so the assistant has room to keep going. Use it on long building sessions.

  • /grill-me — the assistant interviews you one question at a time about your bot. The fastest way to build if you're not sure where to start.

  • /sweep — reviews your current bot for contradictions, vague wording and unclear instructions. Worth running before you test.

Route B — The Configuration tab (write it yourself)

The Configuration tab breaks the bot into sections you fill in directly. A pill next to the tab name shows Ready when everything required is complete, or tells you how many sections are still missing.

Here's what each section is for, and what a good answer looks like.

Overall

Details — the name, scorecard, knowledge repositories and scoring toggle you set at the start. Change any of them here.

Results experience — what the participant sees when they finish:

  • Full debrief — detailed scorecard, communication insights, a performance breakdown, recommendations for what to practise next, and a report by email. Best for self-directed learning and anything people do on their own time.

  • Scorecard only — the scorecard and key insights immediately, with no debrief and no follow-up email. Best for live sessions where a facilitator is doing the debrief in the room.

Conversation

Backstory — the history between the participant and the character. Who are they to each other, and why are they talking at all? Have they met? Is there a contract in place? Did something go wrong last month?

Example: "You have been a customer for three years. The participant is your account manager. Two months ago a system outage on their side cost your team a full day of work, and you felt the response was slow. Your renewal is in six weeks."

Current Scene — the specific moment the practice starts in. Where are you, what does the character want out of this conversation, and what's the tension?

Example: "You're on a video call the account manager requested. You have 20 minutes. You want a concrete commitment on service levels before you'll discuss renewing, and you've been asked by your director to get a discount."

These two fields do most of the work. If a bot feels flat or generic, it's almost always because the backstory and scene are thin.

Background

World Knowledge — a free-text field for facts, rules and context the bot must stay consistent with, on top of the knowledge repositories. Prices, names, dates, internal policies, competitor details. Anything the character would know and shouldn't get wrong.

Character

Profile — the character's name, gender, job title and company, plus their avatar. Small detail, but a named character with a real job title feels considerably more real to practise against than "the customer".

Guidelines — how this person behaves and communicates. Are they blunt or polite? Do they interrupt? Do they go quiet when challenged? This is also where you set difficulty.

The rule of thumb: the character should push back, and then give ground when the participant does the right thing. If they yield too easily, everyone passes without earning it. If they never yield, nobody can recover from a shaky start and the practice just feels unfair.

There's also a Bot opens conversation toggle. Turn it on and the character speaks first, which suits scenarios where they called the meeting. Leave it off and the participant has to open, which is itself a skill worth practising.

Participant

Instructions — notes shown to the participant before they begin. Use this for their side of the story: their role, their objective, and anything they'd realistically have known walking in. Don't give away how the character will behave — discovering that is the point.

User Experience:

Step 3 — Choose an avatar and a practice format

In the Profile section, click Select Avatar. The picker has three tabs — Video, Voice and Type — matching the three ways a participant can practise. Choose a tab, pick a face, and click Choose.

Choosing the tab is how you set the format, so pick deliberately:

  • Type (text chat) — lowest pressure, works on any connection, easiest for large groups. Good for a first exposure.

  • Voice — much closer to a real conversation, and a big step up in realism. Participants need a quiet space and a headset.

  • Video — the most realistic and the most demanding, both for the participant and for bandwidth.

Step 4 — Test it yourself before anyone else does

When the Configuration tab shows Ready, the Preview button in the top right becomes active. Click it and a live conversation with your bot opens in a side panel. Type or speak to it exactly as a participant would.

Don't just check that it replies. Run it three times, playing a different person each time — this is the single most useful thing you can do before release:

  1. The ideal participant. Do everything right. Can you actually score well, or does the bot block you no matter what? If a strong performance can't win, the bot is too hard.

  2. The typical participant. Do a reasonable job with a few gaps. Does the conversation give you genuine openings to demonstrate each scorecard item? Does it feel like a useful struggle rather than a wall?

  3. The off-track participant. Go off topic, be unprofessional, ask irrelevant questions. Does the bot steer you back while staying in character, without lecturing you or handing you the answer?

If you test only one of these, you'll release a bot that works for one kind of learner and fails everyone else.

Two more things to watch for while testing:

  • Invented facts. If the bot makes up a price, a name or a policy, add the real detail to World Knowledge.

  • A conversation that won't end. The character should reach a natural close once the point of the practice has been covered. If it drags, say so in Guidelines.

When something's wrong, change one thing at a time and test again. Changing three things at once tells you nothing about which one helped.

Step 5 — Save (and roll back if you need to)

Click Save in the top right whenever you like. Every save creates a new version, and the History dropdown next to it lists them all, so you can always go back to an earlier draft. It also shows changes made by colleagues, which matters when several people are building the same bot.

Save early and often — it costs nothing, and it means you can experiment freely.


Part 2 — Building the scorecard

A scorecard defines what "good" looks like in this conversation. It's a list of specific, observable things the participant either did or didn't do, and it's what Verity uses to grade the practice and generate feedback.

Think of it as the answer to: if I watched a recording of this conversation, what would tell me it went well?

Finding your existing scorecards

Click Scorecards in the left-hand menu.

The dropdown in the top right switches between My Scorecards, All Scorecards, Company Scorecards and Templates. Start with Templates — there's a good chance something close to what you need already exists, and duplicating it is faster than writing from scratch.

The ⋮ menu on any scorecard lets you View its criteria, Download PDF, Duplicate or Delete it.

Step 1 — Name it

Click Create new, type a name, and click Continue.

Name it after the methodology or the situation — "MEDDIC", "Pricing objection", "Difficult feedback conversation" — so it's recognisable when you're attaching it to a bot months from now.

Step 2 — Add sections and criteria

You'll land on an empty builder. Use the Add bar at the bottom to add two kinds of thing.

Sections

Headings that group related criteria — "Discovery", "Objection handling", "Closing". Keep them short. They shape how feedback is presented back to the participant, so they should match the phases of the conversation as people actually experience it.

Items

The individual criteria. This is where scorecards are made or ruined, so it's worth getting right.

Write each item as a clear yes/no question that someone could answer by reading the transcript. If two colleagues could read the same conversation and disagree about whether the participant did it, the item needs rewriting.

Write this

Not this

Why

Did the seller acknowledge the customer's concern before responding?

Was the seller empathetic?

"Empathetic" is a judgement. "Acknowledged the concern" is something you can point at in the transcript.

Did the seller ask about the decision-making process?

Did the seller ask about the decision process and timeline and budget?

Bundled items can't be answered yes or no. Split them into separate criteria.

Did the seller propose a specific next step with a date?

Did the seller close well?

"Well" means something different to everyone. A date is a fact.

Two more rules:

  • Each item should be unique. Overlapping criteria punish or reward the same behaviour twice.

  • The bot must give a fair chance to do it. If the conversation never raises budget, don't score the participant on discussing budget. A missing opportunity is a design problem, not a participant failure — fix it in the bot's scene or guidelines.

Weighting each item

Click the impact dropdown on any item to set how much it counts:

  • High impact — 3 points — miss this and the conversation fails

  • Medium impact — 2 points — makes a real difference to how the conversation goes

  • Low impact — 1 point — good practice, but minor

  • Not scored — 0 points — tracked and shown in feedback, but doesn't affect the score

Resist the urge to mark everything high impact. If everything is critical, the score stops distinguishing a good conversation from a weak one. Most scorecards have two or three genuinely high-impact items.

Not scored is useful for behaviours you want to give feedback on without penalising people while they're still learning.

You need a minimum of four criteria to save a scorecard. In practice, six to twelve works well — enough to cover the conversation, few enough that the feedback stays digestible.

Step 3 — Save

Click Details in the top right to rename the scorecard at any point. Once you have at least four criteria, Save and exit becomes active. Click it to finish and return to your list.

Your new scorecard is now available to attach to any bot, in the Details section of the Configuration tab.


Common problems and what causes them

What you're seeing

Usually because

Fix

The bot makes up facts about your product or prices

No knowledge repository connected, or the detail isn't in it

Connect a repository in Details, or add the facts to World Knowledge

The bot is too easy — everyone scores highly

The character yields without the participant earning it

In Guidelines, state what the character needs to hear before they give ground

The bot is too hard — nobody can recover

The character never softens, or the scene gives no way forward

Give the character a condition under which they become more open, and check the ideal-participant run can actually succeed

Difficulty varies wildly between runs

Guidelines are vague, so the AI improvises differently each time

Be concrete about behaviour and about what changes the character's mind

Scoring feels unfair

Criteria need interpretation, or the conversation never offers a chance to demonstrate them

Rewrite items as observable yes/no questions, and check each one has an opening in the scene

The bot drifts off the scenario

The scene doesn't pin down the moment tightly enough

Tighten Current Scene: where, why now, what the character wants, and how long they have

The conversation never ends

Nothing tells the character what "done" looks like

Add a closing condition to Guidelines

A checklist before you release a bot

  • Configuration shows Ready

  • You've run the ideal, typical and off-track tests in Preview

  • Every scorecard item had a fair chance to come up in at least one of those runs

  • A strong participant can score highly, and a weak one clearly can't

  • The bot stayed in character and didn't invent facts

  • The conversation ended cleanly

  • Participant instructions explain their role without giving away the bot's behaviour

That's the full loop: build the character and the scene, test it yourself as three different people, then build the scorecard that grades participants against it. Your first bot will take the longest. After that it's mostly duplicating something that already works and changing the details.

Did this answer your question?