CoT defines how your agent thinks and acts during the conversation, it’s like training a new human agent to make calls.
A good CoT doesn’t just give instructions; it guides behavior, tone, and decision-making.
17. Best practices for an efficient CoT
17.1. Clearly define context and role
The agent must know:
- Who it is
- Who it represents
- The purpose of the call
Correct example
You work as a software consulting representative at Rootstack, making professional prospecting calls.
Incorrect
You are an assistant that makes calls.
Problem: too generic → inconsistent responses
17.2. Maintain a clear and predictable structure
A good CoT always follows a logical structure:
- Context
- Capabilities
- Personality
- Rules
- Flow (states/conversation)
This reduces ambiguity and improves consistency.
17.3. Be specific about what the agent SHOULD and SHOULD NOT do
Correct
Do not attempt to sell services during the call.
Incorrect
Be professional and respectful.
Problem: too open-ended → the model improvises
17.4. Use response examples
Examples help to:
- Define tone
- Avoid robotic responses
- Guide real conversation structure
Correct
"Hello, I’m calling from Rootstack. I wanted to briefly introduce our services..."
Incorrect
Introduce the company in a professional way.
Problem: no examples → inconsistent responses
"id": "2_value_proposition",
"description": "Position Rootstack consultatively.",
"instructions": [
"Explain Rootstack clearly and concisely.",
"Position the conversation as consultative, not sales-driven.",
"Briefly explain core services without listing excessive features."
],
"examples": [
"At Rootstack, we specialize in helping companies scale their engineering capacity and modernize their technology ecosystems. We support clients through custom software development, dedicated engineering teams, and ongoing managed services, depending on their needs."
17.5. Design the conversational flow (states)
This allows you to:
- Control the direction of the conversation
- Avoid out-of-context responses
- Handle real scenarios (gatekeepers, rejection, etc.)
17.6. Define critical rules explicitly
Especially for:
- Call termination
- Interruptions
- Silence
- IVR
17.7. Keep the CoT in English even if the user speaks Spanish:
- Improves model consistency
- Reduces technical ambiguity
You can use examples in both languages to guide tone.
18. How to design an effective conversational flow (CoT)
The conversational flow defines how the conversation progresses step by step, allowing the agent to make clear decisions based on each scenario.
A good flow prevents your agent from giving out-of-context responses, having disorganized conversations, or behaving inconsistently.
Base structure of a conversational flow
A well-designed flow is composed of states, where each state represents a specific moment in the conversation. Therefore, each state must include:
- ID → Unique identifier
- Description → What is happening at that moment
- Instructions → What the agent should do
- Examples → How it should respond
- Transitions → Where to go next
Recommended template
You can use this structure as a base:
{
"id": "state_name",
"description": "What is happening in this step",
"instructions": [
"What the agent must do",
"How it should behave",
"Constraints"
],
"examples": [
"Example response 1",
"Example response 2"
],
"transitions": [
{ "condition": "user_response_type", "next_id": "next_state" }
]
}
18.1. How to build the flow step by step
Define the goal of the conversation
Before creating states, answer:
- What do you want to achieve?
- What is the “ideal outcome”?
Examples:
- Schedule a meeting
- Obtain an email
- Validate information
- Transfer to a human agent
This defines your final state.
Identify key moments in the conversation
Break the conversation into simple steps:
Example (outbound call):
- Intro / Gatekeeper
- Brief explanation
- Contact confirmation
- Introduction
- Offer (info / transfer)
- Closing
Create one state for each moment
Example:
- 1_gatekeeper_intro
- 2_gatekeeper_reason
- 3_identity_confirmation
- 5_intro_rootstack
Define transitions
Transitions are what make the flow intelligent.
Each state should answer:
“What can happen next?”
Example:
"transitions": [
{ "condition": "user_shows_interest", "next_id": "offer_info" },