Welcome to AskRAI

Documentation for the Responsible AI platform — architecture, admin console, integrations, and guardrails

AskRAI is a Responsible AI platform that brings intelligent automation to your organization — without stretching budgets or adding complexity. With configurable safety and compliance controls, an always-on audit trail, and transparent logic, AskRAI helps teams Resolve issues, Inform decisions, Engage users, and Support operations.

How RAI works

Platform capabilities

Concepts

Understand how AskRAI delivers responsible AI — from knowledge management and guardrails to confidence scoring and governance

AskRAI is a Responsible AI platform that delivers intelligent automation with built-in safety, compliance, and accountability. Every response is grounded in curated knowledge, evaluated by the guardrails you have attached, scored for confidence, and recorded in a full audit trail.

How a query flows through AskRAI

When a user asks a question, the platform runs a multi-step pipeline designed to keep the response grounded, governed, and traceable.

1. The user sends a question through Microsoft Teams — the one human channel available today — or a machine calls through a programmable interface (API, CLI, or MCP) authenticated with a credential.
2. Active AI Guardrails evaluate the question against your safety, compliance, and quality rules. A guardrail set to Block stops the request here and no answer is produced; one set to Warn or Redact records or rewrites and lets the turn continue. The same rules can instead be set to read the generated answer, or the arguments an agent sends to a tool.
3. AskRAI searches the curated knowledge base for relevant content and composes a grounded answer.
4. The platform assigns a confidence score reflecting how well the knowledge base supports the answer.
5. The response is delivered. Afterwards, your escalation rules are evaluated against the confidence band and can open a support ticket for human follow-up.
6. Every step is recorded in a complete audit trail for governance and continuous improvement.

Core concepts

How it fits together

Knowledge management defines what the assistant knows. Guardrails define what it must not do. Confidence scoring defines how certain it must be. Access control defines who gets access to what. And governance gives you the tools to monitor, measure, and improve over time.

Each concept builds on the others — groups bundle knowledge packs with guardrails, and the audit trail captures how all of these interact during every conversation.

The RIESE Model

Five pillars that define what your AI assistant does — Resolve, Inform, Engage, Support, and Escalate

The RIESE model defines five distinct ways your AI assistant interacts with users. Each pillar represents a different intent — from answering a straightforward question to escalating a complex issue to a human agent. Understanding these pillars helps you organize your knowledge base and configure your assistant to handle each scenario appropriately.

The five pillars

RIESE and Knowledge Packs

Each pillar maps naturally to how you organize content in knowledge packs:

Next steps

• Knowledge Management — learn how to curate and organize content for each RIESE pillar
• Confidence & Escalation — configure when the assistant escalates to humans
• Knowledge Packs — create and manage content collections

Knowledge Management

How content enters AskRAI, gets organized into knowledge packs, and powers AI responses through Text Search and AI Search

The knowledge base is the single source of truth for your AI assistant. Every response is grounded in content your team put there, rather than in the model's own training data, and nothing outside it is retrieved. To see exactly which items a question retrieved, replay it in the Sandbox — the retrieval set is not kept in the audit record.

Two content types

AskRAI supports two types of knowledge content, each suited to different use cases:

Both types live in the same knowledge base. The difference is in how they enter the system — and in how they are read: admin AI Search merges both, while at query time the assistant searches each separately and keeps whichever leg scored higher, summarizing that one alone.

Content ingestion pipeline

When you upload a document to a knowledge pack, AskRAI processes it through an automated pipeline.

1. Upload — drag and drop files into a document-type knowledge pack. A document-type pack accepts PDF only; a Q&A-type pack also accepts CSV and XLSX tables of ready-made pairs. There is no OCR, so a PDF has to carry a real text layer — a scan or a photograph fails.
2. Extract — the system reads the file's text layer, keeping track of which page each passage came from.
3. Chunk — long documents are split into overlapping windows that each capture a coherent unit of information.
4. Embed and store — each chunk is turned into a vector and written to the knowledge base, which makes it searchable the moment it lands.

For Q&A pairs, you write the question and answer directly — no parsing pipeline needed.

Knowledge Packs

Knowledge packs are containers that group related content together. Each pack has a content type (Q&A or Document) chosen at creation, and content within a pack shares the same lifecycle and access controls.

Packs serve two purposes:

• Organization — group content by topic, department, or use case (e.g., "HR Policies", "Building Permits", "IT Self-Service")
• Access control — groups reference knowledge packs to control which users can access which content

Content lifecycle

Every knowledge item follows a status workflow that controls whether it appears in search results:

Only Approved content is retrievable, so an item in any other status answers nothing.

Two search modes

AskRAI offers two ways to search the knowledge base:

• Text Search — substring matching against questions, answers, titles, section headings, document content and tags. Fast and predictable, and the only mode that reliably finds an exact identifier or phrase.
• AI Search (semantic) — meaning-based search that finds relevant content even when the wording differs from the query. Uses vector embeddings to match by concept rather than keywords.

Next steps

• Knowledge Base — create and manage Q&A pairs and document chunks
• Knowledge Packs — organize content into collections and upload files
• Guardrails — understand how safety rules protect knowledge-based responses
• Access Control — learn how groups control access to knowledge packs

Guardrails

A guardrail is three decisions — what it checks, which payload it reads, and what it does about a violation

A guardrail is one check AskRAI runs over one payload of a turn. Every guardrail is three decisions made independently of each other: a kind that decides how the check is performed, a stage that decides which payload it reads, and an action that decides what happens when it fires.

Those three are what the rest of this page is about. One more setting does change behaviour: priority is the order guardrails are evaluated in, and it decides which blocker is reported as the reason. The category is a label for the people reading the console.

Kind — how the check is made

Six kinds ship today. Five of them are evaluated in code: they cost no model call, add no provider latency, and give the same answer every time.

The last column matters because Redact is an action a kind has to be able to perform. A judge returns a verdict rather than the position of what it objected to, and a link-domain check has nothing to substitute, so neither can redact. The console refuses that combination when you save it rather than at runtime.

Stage — which payload it reads

A rule phrased as a constraint on the response is enforceable: set its stage to Output. Set that same rule on Input and it fires on nothing and reports compliance on every turn.

Redact means something different at each stage. On Input it rewrites the question, and what continues into retrieval — and what the audit record stores as the question — is the rewritten text. On Output it rewrites what is delivered. On Tool call it refuses the call entirely: a redacted argument is a different call to the far side, and sending one the model never wrote would be worse than not sending it at all.

Action — what happens when it fires

There is no majority rule and no scoring: one blocking guardrail is enough. A Warn does not stop the turn — its effect is the record it leaves, which is why it is worth reviewing the audit trail for warnings rather than waiting for something to break.

How an evaluation runs

The deterministic kinds are evaluated first, one after another in priority order. The LLM judges run afterwards, concurrently — at most three at a time. If something has already blocked, only the judges that would Block are skipped; a Warn judge still runs. (A judge cannot be set to Redact — see the table above.)

Three consequences follow from that order:

• A regex settles what a judge would be billed to decide. This one is free: the deterministic kinds always run before any judge, whatever priority you give them, so a cheap check that blocks keeps the turn off a model. Priority does not control it.
• Priority orders the deterministic checks among themselves, lowest number first, and when several would block, the first is the one reported as the reason. Note that the console's bands run the other way — it labels 8–10 High and 1–4 Low — so a guardrail in the Low band is evaluated before one in the High band, and is the one an administrator is told about. Set the number you want evaluated first to the lowest value, not the highest.
• A redaction earlier in the order changes what every later guardrail reads. A PII guardrail that redacts at priority 1 means the judge at priority 9 sees the redacted text, not the original.

Once a deterministic check has blocked the turn, blocking judges are skipped — their verdict cannot change the outcome and the tenant should not be billed for it. Judges set to Warn still run, because they were attached to observe exactly these turns and a skipped one would report "never fired" on the population it exists to watch.

When a guardrail cannot be evaluated

AskRAI stops the turn instead of allowing it. A control that quietly stops checking while still reporting compliance is the failure this whole plane exists to prevent, so it is the one outcome the evaluator will not produce. A turn fails closed when:

• an LLM judge names a model that no longer resolves, or the tenant has no model configured for the guardrail stage;
• a pattern runs long enough to be abandoned;
• a Block or Redact guardrail is handed a payload longer than 100,000 characters — neither "I refused all of this" nor "I handled every span in this" can be said honestly about a prefix. A Warn guardrail scans the first 100,000 characters and says so in its record;
• a guardrail was written by a newer build than the one evaluating it. Only the subjects that attached it fail; everyone else in the tenant is unaffected.

Where a guardrail comes from

A guardrail governs a subject, never the tenant at large.

• For a person asking a question, the subject is their groups — the union of every guardrail listed by every group they resolve into.
• For a team agent, the subject is the agent itself.

Either can also carry policies, and a policy brings its own guardrails. All of those sources compose into one effective set, under a single rule: every merge is a narrowing. Guardrails union together, ceilings take the lower value, and a host must be admitted by every allowlist in force. Attaching a second policy can tighten what a subject may do and can never loosen it — so there is no "bypass" group, and no way to exempt one population by adding something to it.

Only guardrails whose status is Active take part. A draft governs nothing, however carefully it is written.

What is recorded

Every evaluation — pass or fail — is written to the audit trail and counted in the analytics. There is no setting that reduces this. A redaction records how many spans it replaced and never the values, which would put the content back into the record the redaction exists to keep it out of.

Administrators can mark a guardrail trigger as a false positive in the conversation logs. Those marks drive the per-guardrail false-positive rate on the guardrail analytics, which is how you find a rule that fires too eagerly without weakening it blind.

Next steps

• Settings → Guardrails — author a guardrail's kind, stage, action and status
• Guardrails — review how each one is performing
• Policies — attach a set of guardrails, and ceilings, to a group or an agent
• Governance & Audit — how evaluations feed the audit trail
• Sandbox — test guardrail behaviour before it reaches production

Confidence & Escalation

How AskRAI scores response confidence and uses escalation rules to decide when conversations need human attention

Every AI response receives a confidence score — a percentage reflecting how well the knowledge base supports the answer. The score sorts the answer into a band, and your escalation rules decide what follows from that band. Scoring happens as the answer is produced; escalation happens after it has been delivered, which is what decides the difference between the two.

Confidence bands

Confidence scores fall into three configurable bands. You set the thresholds in Settings — the defaults are shown here.

Escalation Rules

Escalation rules define what happens when a response falls into a particular confidence band. Each rule specifies a condition (confidence band) and an action to take.

Four escalation actions

Rule ordering

When multiple escalation rules match (for example, if two rules both apply to the Medium band), the rules are evaluated in priority order. The first matching rule's action is executed.

Confidence in practice

The confidence score is not a simple keyword-match percentage. After the assistant drafts an answer, a lightweight assessment step scores two things:

• Answer quality and groundedness — how well the drafted answer actually addresses the user's question using the retrieved knowledge base content, rather than guessing or padding.
• Retrieval relevance — how close the top knowledge base results the answer was built from are to the question, measured as the similarity score retrieval returns for each of them.

Only the first of those bands the answer. Retrieval relevance is kept as a separate signal and does not move the confidence band. A high score means the assistant judged its own answer well-grounded in what it retrieved. A low score means it is uncertain — perhaps the question is outside the knowledge base's scope, or the best-matching content only partially fits.

Testing Escalation Rules

Use the Sandbox to test how confidence scoring and escalation rules behave with real queries. The sandbox's audit preview tab shows:

• The confidence score the query received
• Which escalation rule matched
• What action would be taken in production

This lets you tune thresholds and rules before they affect live conversations.

Next steps

• Settings — configure confidence thresholds and escalation rules
• Tickets — view and manage escalated conversations
• Sandbox — test confidence and escalation behavior
• Governance & Audit — learn how confidence data feeds into analytics

Access Control

Users, roles, groups, and multi-tenant isolation control who can access which knowledge and guardrails

AskRAI's access control model determines who can access which knowledge and what safety rules apply to their interactions. The model is built on four layers — users, roles, groups, and tenants — that combine to provide fine-grained, scalable access management without per-user configuration.

Two kinds of access

AskRAI separates administering the tenant from using its content, and the word "role" means something different on each side. Keep them apart:

A Platform Admin can open Settings and reconfigure the tenant. A Member cannot, but still uses the console and their projects normally. This is a separate question from what content anyone can see — that is decided by groups and resource-access roles, described below, and applies to admins and members alike.

The access hierarchy

Users

Users are real people in your organization. They are synced automatically from your identity provider — you do not create user accounts manually in AskRAI. Every synced user is a Member until an admin grants them a tenant role on the Access page — with one exception: the very first user to sign in to a brand-new tenant is automatically promoted to Platform Admin, so there is always someone who can open Settings and grant roles to everyone else. Administrators can:

• View user details (name, department, job title, contact info)
• Monitor activity (last login, active sessions, online status)
• Block or unblock user accounts

Users gain access to knowledge and guardrails through their group memberships, never directly.

Roles

The roles in this section are resource-access roles — the ones that decide which knowledge and guardrails an interaction gets. They are unrelated to the tenant roles (Platform Admin / Member) that unlock Settings.

Roles represent usage patterns rather than individual people. Each role defines mapping rules — conditions that automatically classify incoming interactions based on properties like:

Those six are the whole list. Location and Time rule types were offered until recently and never worked — they were compared against a field the record did not carry, so a role carrying one matched nothing. They have been removed rather than fixed; making them work would be a new feature, not a repair.

A role saved with one of them before the removal still carries it. AskRAI holds that rule aside and refuses to match the role at all, rather than matching on the rule's surviving siblings — which is the decision the role already produced. Rewrite such a role using the six types above to bring it back into use.

When a user sends a message, the system evaluates mapping rules to determine which roles apply. Every rule on a role must match for that role to apply, and a user receives the combined access of every role that does — there is no ordering between roles. Roles are assigned to groups, so matching a role grants the user that group's knowledge packs and guardrails.

Groups

Groups are the central connection point in the access model. A group bundles together:

• Members — users and roles that belong to the group
• Knowledge packs — content collections the group can access
• Guardrails — safety rules that apply to the group's interactions

A user's effective access is the union of all knowledge packs and guardrails from every group they belong to. If a user belongs to Group A (with packs 1 and 2) and Group B (with packs 2 and 3), they can search packs 1, 2, and 3.

Group types

Multi-tenant isolation

AskRAI is a multi-tenant platform. Each tenant operates in complete isolation — data, configuration, users, knowledge, and guardrails are fully separated between tenants.

From a product perspective, this means:

• Each organization gets its own independent AskRAI environment
• Users in one tenant can never see or search content from another tenant
• Configuration changes in one tenant have no effect on others
• Administrative actions (creating groups, editing guardrails, managing users) are scoped entirely to the current tenant

Multi-channel support

AskRAI supports multiple communication channels through the same access control model. Roles with channel-based mapping rules allow you to:

• Serve different content to different channels (e.g. a narrow public-facing set for API callers, internal policies on Teams)
• Apply stricter guardrails to public-facing channels
• Track usage and confidence metrics per channel in the Dashboard

Next steps

• Access — grant the Platform Admin and Member tenant roles that decide who can reach Settings
• Users, Roles & Groups — manage users, create roles, and configure groups
• Knowledge Packs — create content collections to assign to groups
• Guardrails — author the safety rules a group or policy carries
• Governance & Audit — learn how access control ties into audit and compliance

Governance & Audit

Audit trails, analytics dashboards, sandbox testing, and the continuous improvement loop for responsible AI

AskRAI is designed for organizations that need accountability, transparency, and continuous improvement in their AI operations. Every interaction is recorded, every guardrail evaluation is logged, and every confidence score is tracked — giving your team the data needed to monitor, measure, and improve your AI assistant over time.

The continuous improvement loop

Governance in AskRAI is not a one-time setup — it is an ongoing cycle of interaction, review, tuning, and testing.

1. Interact — users ask questions through connected channels. Every interaction flows through the query pipeline.
2. Audit — the platform records the full context of every interaction: the query, the response, the confidence score, guardrail evaluation results, and processing metadata.
3. Review — administrators use the dashboard and conversation logs to identify patterns: frequent low-confidence answers, guardrail false positives, knowledge gaps, and usage trends.
4. Tune — based on review findings, administrators adjust configuration: add knowledge content, refine guardrail prompts, adjust confidence thresholds, or modify escalation rules.
5. Test — changes are validated in the sandbox environment before going live, ensuring they work as expected.
6. Deploy — verified changes are applied to production, and the cycle continues.

What the audit trail captures

Every conversation generates an audit record containing:

This data powers the dashboard analytics, conversation review, and compliance reporting.

Dashboard analytics

The Dashboard provides at-a-glance metrics for monitoring platform health:

• Request volume — total conversations and trends over time
• Response quality — success rate based on confidence thresholds
• Response speed — average processing time and distribution
• User engagement — active user counts and channel breakdown
• Confidence trends — distribution of responses across high, medium, and low bands
• Channel analytics — per-channel usage and performance

All metrics are filterable by date range and channel.

Conversation review

The Conversation Logs let administrators drill into individual interactions to:

• Review the full conversation thread with user and assistant messages
• Inspect the audit details for any message — confidence score, guardrail results, and model and routing information
• Mark guardrail triggers as false positives to track and reduce over-triggering
• Identify knowledge gaps where the assistant could not provide a good answer

Search debug data is not part of the audit record; to see which knowledge items a question retrieves, replay it in the Sandbox.

Escalation tracking

When conversations are escalated, they create tickets in the Tickets queue. Each ticket captures:

• The original question, and the AI response unless the rule clears Include AI context
• The confidence score and which escalation rule triggered
• Severity classification for triage
• Status tracking through the resolution workflow (Open → In Progress → Resolved → Closed)

Ticket patterns reveal systemic issues — if many tickets come from the same topic, it signals a knowledge gap that can be addressed by adding content.

Sandbox testing

The Sandbox is a safe environment for validating configuration changes before they affect live users. You can:

• Test with custom configurations — select specific guardrails and knowledge packs to evaluate
• Test as a group — simulate a group's exact configuration to verify the end-user experience
• Inspect the full pipeline — view guardrail evaluation results, search debug data, audit previews, and model information for every test message
• Validate escalation rules — see which rules would trigger and what actions would be taken, without creating real tickets

Compliance posture

AskRAI's governance model supports compliance requirements by providing:

• Traceability — every response is recorded with the question that produced it, the confidence score it received, and the result of every check that ran. Which knowledge items were retrieved is not part of that record — replay the question in the Sandbox to see the retrieval set
• Accountability — audit records capture who asked, what was answered, and what safety checks were applied
• Transparency — a guardrail that fires records why. A check that passed records no reason, and the deterministic kinds never produce prose at all — a fired pattern check records how many spans matched
• Control — the content lifecycle (Draft → Approved → Needs Attention → Discarded) decides what is retrievable, and gives you a place to park an item you no longer trust

Next steps

• Dashboard — monitor platform metrics and trends
• Conversation Logs — review individual interactions
• Tickets — manage escalated conversations
• Sandbox — test configuration changes safely
• Settings — configure thresholds and escalation rules

Admin Console

Overview of the AskRAI admin console — run projects, monitor activity, manage content, and configure settings

The AskRAI admin console is the central hub for managing your AI assistant. From here you can run projects where people and AI agents collaborate, monitor platform activity, curate knowledge, configure guardrails, and manage users and settings.

Projects

Organize work into projects within an organization — create, browse, edit, and delete projects, and track each project's team and budget from a single workspace

Projects are the workspaces where your people and AI agents collaborate. Each project lives inside an organization and brings together a team, a budget, and an overview of how everyone connects. Open the Projects plane from the plane switcher in the top header.

The Projects plane pairs a searchable project list with a per-project workspace — here the Customer Insights Platform project, scoped to the active Acme Corporation organization.

How projects fit in

A project always belongs to one organization, and the active organization scopes which projects you see. Switching organizations in the header switcher reloads the Projects plane for the newly selected organization.

• Organization → projects — you need at least one organization before you can create a project. If your tenant has no organizations yet, the Projects plane prompts you to Create an organization in Settings first.
• One workspace per project — selecting a project opens its workspace, where the Overview, Team, Interactions, Activity, and Budget tabs live.
• Team and budget — every project tracks the people and agents working on it (the Team tab) and an optional spending budget.
• A live team — the Interactions tab runs the project's agents and people as an autonomous workspace, with messages, a task board, and Start/Stop controls.

Browsing projects

The Projects sidebar lists every project in the active organization. Use it to search, switch between projects, and start a new one.

• Search — type in the Search projects… box to filter the list by name or description.
• Two groups — the six most recently updated projects appear at the top, unlabelled; the rest are grouped under All projects.
• Status at a glance — every entry shows the project name and a status badge (Active, Paused, or Archived). The top group also shows when each project was last updated.
• Select a project — click any project to open its workspace on the right.

When the active organization has no projects yet, the workspace shows an empty state with a New project button to create your first one.

Creating a project

Click the + button next to the Projects heading in the sidebar (or the New project button in the empty state) to open the create dialog.

New projects are created in the current organization; you set the status and budget afterward by editing the project.

New projects start as Active, with no budget and an Idle check-in of 60 minutes. To set a status, allocate a budget or change the idle check-in, edit the project after creating it.

Project fields

The project workspace

Selecting a project opens its workspace. The header shows the project name and status, its description, the Start / Pause / Stop run controls, and Edit and Delete actions. A stats strip summarizes the People & agents count, Interactions, and Budget usage. Below, five tabs organize the work.

Overview tab

The Overview tab summarizes how the project is staffed and connected.

• Project relationships — a live interaction graph of how people, agents, and models connect.
• People, Active agents, and Open tasks — headline counts for the project's team and its task board.
• Recent activity — a feed of what the team's agents and people are doing, drawn from the Interactions workspace.
• Resources — marked Coming soon; this surface will populate in a future release.

The interaction graph

The Project relationships card on the Overview tab draws the project as a graph: who is on the team, who reports to whom, and which model each agent runs on. While the team is running, it also animates the work as it happens.

The graph places the team on the left and their models on the right. Beacon reports to Atlas, and both run on the same model — so it collapses into a single shared node. The green outline is a beat in progress.

What the graph shows

Watching the team work

Start the team and the graph goes live — the status line reads Live — pulses show real agent heartbeats, messages, and task handoffs. Three kinds of pulse then light up the graph as the events actually happen:

Reading the graph

• Focus a member. Click a person or agent to focus it. Everything it connects to stays sharp, everything else dims, and a hint reads Showing relationships for …. Click the canvas to clear it.
• Filter. Untick People, Agents, or Models to hide that kind of node, or use All to toggle everything at once.
• Pan and zoom. Drag to pan, scroll or pinch to zoom. Nodes cannot be dragged.

If the project has no team yet, the card reads No team yet — add people and agents on the Team tab to populate it.

Budget tab

The Budget tab tracks spending against the project's budget total.

The Budget tab does not update live the way the Overview and Interactions tabs do: it shows what had been spent when the page loaded, so reload it to see a running team's latest figure.

When a budget is allocated, the Budget overview shows the amount used against the total (for example $62.43 of $100.00 allocated), a Utilization bar with the percentage consumed, and a closing line that reads Remaining: $37.57, At budget limit, or Over budget by $20.00. If no budget is set, the tab reports that spend is uncapped, tells you how much has accrued so far, and prompts you to edit the project to allocate one.

Spend is reported to the cent, and small amounts are never rounded away: a project that has spent a fraction of its budget total reads <1% used rather than 0%, one that is nearly exhausted reads >99% used rather than 100%, and an amount under half a cent reads <$0.01 rather than $0.00. The project header carries the same figure as a Budget stat.

Idle check-in

Beneath the budget, the Budget tab reports the project's Idle check-in — the longest an agent with nothing to do waits between beats.

The readout states the ceiling in plain units, and says outright that a wake overrides it.

This is the other half of what a team costs. The budget total bounds the total a project may spend; Idle check-in bounds the rate a team that has finished its work keeps spending, because a beat is a model call whether or not there is anything to do. An agent whose beats change nothing doubles its own interval toward this ceiling; anything sent to it — a message, a task, a person acting — wakes it immediately regardless. See Agent Heartbeat for the full rule.

Change it by editing the project.

Editing a project

Click Edit in the project header to open the edit dialog. Editing is where you set the project's status and budget.

The edit dialog adds Status, Budget total (USD) and Idle check-in (minutes) on top of the name and description. The Status selector shows the stored value — active, paused or archived — while the open list shows the names below.

The edit dialog exposes these fields:

Click Save changes to apply. If you enter zero or a negative number, AskRAI reports Budget total must be a positive number — correct the value and save again. To remove a cap entirely, clear the field rather than setting it to zero.

Idle check-in is validated the same way: a value outside 1 to 525,600, or one that is not a whole number, is refused with Idle check-in must be a whole number of minutes between 1 and 525600.

Project status

Deleting a project

Click Delete in the project header to open a confirmation dialog. Deleting a project is permanent, so AskRAI asks you to confirm.

Troubleshooting

Related pages

• Team — add people and AI agents to a project and map how they report to one another
• Interactions — run the project's team as a live, autonomous workspace with messages, tasks, and run controls
• Organizations — the organizations that contain your projects and scope the Projects plane
• Users, Roles & Groups — manage the directory of people you can assign to projects

Team

Build a project team of people and AI agents, organize them into a reporting hierarchy, and configure each agent's model and behavior from the project's Team tab

The Team tab is where you staff a project with the people and AI agents that do its work, and arrange them into a reporting hierarchy. People come from your directory; agents are configured here with a model and behavior. Open it from the Team tab inside any project workspace.

The org chart shows the whole team at a glance — people (solid cards) and AI agents (dashed cards, labeled with their model) connected by reporting lines.

People and agents

A project team mixes two kinds of members:

• People — members drawn from your organization's directory. Each carries their directory identity (name, email, job title, department).
• Agents — AI agents you create on the project, each backed by a model and an optional behavior prompt.

Both kinds share one reporting hierarchy: any member can report to any other member, whether person or agent. The Team header shows the total member count for the project.

Org chart and list views

Switch between two views of the team using the toggle in the Team header:

• Org chart — a visual hierarchy. People appear as solid cards with their initials; agents appear as dashed cards labeled with their model. Paused agents are dimmed, and a dimmed agent is genuinely idle: it is not beating and not spending. Click any card to open that member's details. When viewing the chart, use Fit to screen, Expand all, and Collapse all to navigate large teams.
• List — a sortable table of every member.

List view shows people and agents together. The Title / Model column holds a person's job title or an agent's model, which is why one column carries both.

In the list view you can:

• Search members by name, title, email, or model.
• Filter by type — All types, People, or Agents.
• Filter by status — All statuses, Active, Paused, or Archived (agents only; people have no status).
• Sort by Name, Type, or Title / Model.

Adding a person

Click Add person to assign someone from your directory to the project.

Search your directory, select a person, and choose who they report to before adding them to the project.

Adding an agent

Click Add agent to create an AI agent on the project.

Give the agent a name, pick the model that powers it, and optionally describe how it should behave.

Agent fields

Configuring an agent

Click an agent — a card in the org chart or a row in the list — to open its detail panel.

The reporting line is what the org chart draws, and it is how a manager agent knows whose work to oversee: each of its beats includes what its direct reports have been doing.

The agent's avatar sits in the panel header — click it to open Edit avatar and choose an Initials or Icon avatar (image upload is coming soon). Below the header, the settings are split across two tabs, each saved with Save & Close:

• The Overview tab is the agent's identity: Name, Description, Model, Status (Active, Paused, or Archived — see Status), and Reports to. An agent's reporting line is editable here at any time; AskRAI prevents a reporting loop by excluding the agent and everything beneath it from the options. This tab also holds Delete agent.
• The Behavior tab shapes how the agent works: Goals (one per line), the System prompt prepended to every run, and an Advanced section for model parameters (temperature, max tokens, top P) and response settings (format, language, citation style).

Goals are what the agent is told to pursue on every beat, so editing them changes the agent's behaviour from its next beat onward — there is nothing to restart.

The Heartbeat tab is where you set how often the agent acts and how much it may do without you — see Agent Heartbeat & Autonomy.

Agent tabs

Watching what an agent does

The Activity tab answers what has this agent been doing? It lists the actions the agent has taken on this project — beats, messages it sent, tasks it created, and status changes it made — newest first.

The Activity tab is the agent's own slice of the project activity feed — the fastest way to see whether an agent is pulling its weight.

The feed is a snapshot rather than a live stream — click Refresh to pull the latest actions. It shows the 50 most recent.

Reading an agent's logs

The Logs tab is the operational view: one entry per heartbeat, with the cost and outcome of each.

Each log entry is one beat — its outcome, what triggered it, how many actions it took, and what it cost. The failed beat records why.

Every entry carries a level badge, a timestamp, and a one-line summary in the form succeeded — beat #42 · scheduled · 3 actions · 4,830 tok · $0.02 · gpt-4.1. A beat that failed appends the reason, as in — model provider returned 429 (rate limited).

Select any entry to open its transcript — the full record of what the agent read, reasoned, and decided on that beat. Some beats have no transcript captured, in which case the dialog says so.

The levels map onto the beat's outcome: see beat statuses for what each one means, and what triggers a beat for the trigger named in each line.

Capping what one agent spends

The agent's Budget tab is where you bound what a single agent costs, inside the budget the whole project runs against.

A cap is a ceiling drawn against the project pool — not an allocation carved out of it.

Project budget pool. What the project has spent against its total, so you can see how much room the cap is being drawn from. If the project itself is out of budget, the card says so — no agent on the team runs until the project budget is raised and its status is set back to Active, whatever its own cap says.

Agent spend cap. Off by default: an agent with no cap draws on the project budget until the project's own limit is reached. Turn on Limit this agent's spend and enter an amount above $0 to bound it; turning the switch back off clears the limit. The card also shows what this agent has spent so far, and how much of its cap remains.

Two properties are worth knowing before you rely on a cap:

• It is checked after each run completes, so a run already in flight can carry spend slightly past the cap.
• It bounds what this agent spends on its own runs. Work it hands to another agent is charged to that agent.

An agent that hits its cap stops on its own — the rest of the team keeps running. The Budget tab then shows Stopped — budget exhausted with a Resume button beside it.

Raise or clear the cap first, then press Resume. Pressing it while the cap is still below what the agent has spent is refused with Agent budget is exhausted, which tells you the cap is the step still outstanding. Resume returns this agent to its schedule and leaves the rest of the team where it is — unlike Start in the project header, which restarts every agent.

Cadence & tokens. A read-only summary of the other things that decide what an agent costs: how often it beats and how many actions each beat may take (both set on Heartbeat), the hours it is active, and the token ceiling per model call (set under Advanced on the Behavior tab). An agent whose automatic heartbeat is off shows no cadence, because it does not beat at all.

Checking a person's workload

Click a person in the org chart or list to open their detail panel. Alongside their read-only Directory profile (display name, email, job title, department, and project role, all managed in your corporate directory) and their Project assignment, an Inbox on this project card shows what the team has sent their way:

• Assigned tasks — every task currently assigned to them, with its status and when it last changed.
• Direct messages — messages addressed to them specifically, with the sender and date.

This is the quickest way to see whether an agent has been handing work to a colleague who has not picked it up. Agents have no inbox card — use their Activity and Logs tabs instead.

The reporting hierarchy

The org chart is built from each member's reporting line, which can span both people and agents:

• People and agents alike are assigned a manager when added, and you can change it later from the member's detail panel.
• No loops — a member can't be set to report to one of its own reports.

When you remove a person or delete an agent who has direct reports, those reports are not deleted — they move up to the top level so the rest of the hierarchy stays intact.

Removing people and agents

• Remove a person — open the person's detail panel and click Remove from project, then confirm. The person is removed from the team but remains in your directory.
• Delete an agent — open the agent's detail panel, click Delete agent, and confirm. The agent is permanently removed from the project.

In both cases, any direct reports move to the top level of the org chart.

Troubleshooting

Related pages

• Agent Heartbeat & Autonomy — set each agent's cadence, autonomy, and per-beat limits, and review the beats it has run
• Projects — the project workspace that contains the Team tab
• Interactions — run the team as a live workspace with messages, tasks, and proposals
• Model Providers — manage the models that power your agents
• Users, Roles & Groups — the directory of people you can add to a team
• Organizations — the organizations that contain your projects

Agent Heartbeat & Autonomy

Set how often an AI agent acts and how much it can do without you — configure an agent's cadence, autonomy level, and per-beat limits, and review the beats it has run

An agent works in heartbeats: a beat is a single, bounded turn in which the agent reads its inbox, plans, takes a few actions — sending messages, creating or updating tasks — and then stops until its next beat. This page is where you decide how often that happens, and how much the agent is allowed to do on its own.

Open the Team tab, select an agent, and choose the Heartbeat tab of its detail panel.

The Heartbeat tab pairs the agent's cadence and autonomy settings with its run state. That state is read when you open the project, not streamed — reopen the agent to refresh it. The history of past beats lives on the Logs tab.

Autonomy levels

An agent's autonomy decides what happens to the actions it produces on each beat.

Heartbeat settings

The Heartbeat protocol card configures the agent's cadence. Edit the fields and click Save.

Cadence while an agent has nothing to do

A beat is a model call whether or not there is anything to do, so an agent that has finished its work does not keep asking at its full rate. A beat that changes nothing doubles this agent's interval, and it keeps doubling — one idle beat after another — up to the project's Idle check-in ceiling, which defaults to 60 minutes and is set when you edit the project.

A beat counts as changing something when it applied an action or raised a proposal for you to decide. A beat that produced nothing — including one that failed — counts as idle.

Three things reset the backoff and put the agent straight back on its own interval:

• Anything sent to the agent. A message, a reply, or a task assigned to it. Any of these wakes it immediately and clears the streak — including a nudge that arrives while a beat is already running.
• A beat that changes something. Doing real work is itself the reset.
• Starting or resuming the team. Every agent comes back on its own interval, however long the team was paused.

Run heartbeat is outside all of this: a beat you run by hand neither moves the schedule nor resets the idle streak, whatever it does. Use it to test an agent without disturbing its cadence.

It overrides the run state — it works on a stopped project — but not the agent's own configuration. An agent whose Status is Paused or Archived, or one over its spend cap, is refused rather than beaten, so a manual beat cannot spend money an agent has been stood down from spending.

Watching an agent's beats

The Heartbeat tab shows this agent's run state: whether it is Running, how many Beats it has completed, when its Last beat ran, and its Current action. These are read when the project is opened rather than streamed, so reopen the agent to see the current figures. Click Run heartbeat to queue a single beat immediately — useful for testing an agent without waiting for its timer.

The history is on the agent's Logs tab, not here. Each entry there is one beat, with what triggered it, its outcome, and the actions, tokens and cost it consumed; select an entry to read its transcript.

Beat statuses

What triggers a beat

A beat is triggered by one of several events, shown next to each entry on the Logs tab:

Sending the agent a message, replying to it, or assigning it a task does not produce a trigger of its own — it brings the agent's next Scheduled beat forward to now.

Troubleshooting

Other symptoms and fixes for a team that is not behaving are collected in one place — see Interactions → Troubleshooting.

Related pages

• Team — staff the project with people and agents, and configure each agent's model and behavior
• Interactions — run the team, exchange messages, work the task board, and approve proposals
• Projects — the project's budget, which hard-stops the team when it is exhausted
• Model Providers — the models an agent can run on, and their token prices

Interactions

Run a project's team of AI agents and people as a live, autonomous workspace — start and stop the team, exchange messages, manage a shared task board, watch activity, and approve agent proposals

The Interactions tab turns a project into a live workspace where its AI agents and people collaborate: they message one another, create and hand off tasks, and post activity, all driven by a multi-agent heartbeat loop that you start and stop. Open it from the Interactions tab inside a project, alongside Overview, Team, Activity, and Budget.

With the team running, the Interactions tab pairs a Pending approvals card with a live message thread between the project's agents — here Atlas and Beacon coordinating a new-hire onboarding run.

How interactions work

Each agent on a project runs a heartbeat: a single, bounded turn in which it reads its inbox, plans, takes a few actions (sending messages, creating or updating tasks, posting activity), and stops until its next beat. Agents coordinate only through the project's shared state — the message thread and the task board — so the team behaves like a set of collaborators working from a common board rather than one monolithic process.

• People and agents share one workspace. The same team you staff on the Team tab is the cast here. Agents act on their own cadence; people message, create tasks, and approve agent proposals.
• You control the loop. Nothing runs until you Start the team, and a hard budget stop or an explicit Stop ends it. Start requires the project's status to be Active — a paused project is refused however much budget it has.
• Autonomy is per agent. Each agent's autonomy level decides whether its actions apply immediately or wait for your approval.

Running the team

The project header carries the run controls: Start, Pause / Resume, Stop and Reset, plus a live run-state badge and a count of how many agents are currently running.

While the team runs, the header shows a green Running badge, the number of agents running, and a badge for any pending approvals.

Run states

Clearing a run with Reset

Reset is beside the run controls, and it is the broadest of the three destructive controls on this page — the other two are Delete on a single task card and Clear all on the task board. It clears everything the run produced: every task, message, activity entry, proposal and beat transcript on the project is permanently deleted. This cannot be undone.

The confirmation names both halves — what is destroyed, and what survives.

The project itself is kept: its agents, goals, budget and connections are untouched, so it can be run again from the start. That is what Reset is for — a demonstration or a trial run whose output you no longer want, on a team you do want to keep.

Budget and hard stops

A running team spends money on every beat, so AskRAI will stop it for you.

Messages

The Messages view (the default toggle in the Interactions tab) is the team's conversation. A composer sits at the top, with the reverse-chronological thread below. Every message shows its author, an Agent badge for agents, the recipient when it is a direct message, and how long ago it was sent.

Tasks

Switch the Interactions tab to the Tasks view to see the shared task board — the unit of work agents hand off to one another and to people.

Each task shows its title, description, who created it, when it was last updated, and its status — sorted so in-progress work surfaces first. The header counts the board as 5 total · 4 open.

Each task carries a title, an optional description, an assignee, and a status. Under the title, a meta line records Created by — so you can always tell whether a person or an agent raised the work — followed by a due date if one is set and the time of the last update. Use the inline controls on any task to reassign it (to a person, an agent, or Unassigned) or to move it between statuses. Click New task to add one yourself.

Task statuses

The board sorts In progress first, then To do, Blocked, and Done, and orders each group by most recently updated.

Only a running agent works a task

Assigning a task never makes it happen by itself. An agent only picks work up on its next heartbeat, so if the assignee is not a running agent, AskRAI tells you so as soon as you assign:

Task created, but Atlas won't work it until its heartbeat is running. Start the project to act on it.

The same notice appears (worded Reassigned, but…) when you move an existing task to such an assignee. The assignment always goes through — the message is advisory, not a block.

Opening a task

Click a task's title to open its detail drawer.

The drawer pairs the task's details with its own activity trail — here the agent's closing entry records exactly what the finished work produced.

The drawer has two parts:

• The task's details — its status, description, Assignee, Created by, Due date, when it was last Updated, and its Acceptance criteria when one was set.
• Activity & logs — everything that has happened to this task, newest first. Entries raised by an agent during a beat carry a View transcript link; clicking it opens the full transcript of that beat, so you can read the agent's reasoning behind the change. Entries made by a person have no transcript. Use Load More to page back through a long trail.

Deleting tasks

Each task card has a delete button. Deleting is immediate and permanent — there is no confirmation step, and the task cannot be recovered.

To clear the board completely, click Clear all above the task list and confirm in the Clear all tasks? dialog. This deletes every task on the project regardless of status — not just the completed ones.

Deleting a task removes it from the board; nothing else on the board changes.

Activity

The Activity tab is a reverse-chronological feed of everything happening on the project — agent beats, messages, task creation and hand-offs, and submitted proposals. Each entry names the actor, labels agents with an Agent badge, and timestamps the action. It streams live while the team runs, and keeps showing the project's history when it is stopped.

The message thread, task board, and activity feed each hold the 10 most recent entries, with a Load More button at the bottom to pull in the next ten older ones. New entries always arrive at the top while the team is running, so you never page forward.

The Activity feed is the audit trail of the running team — useful for seeing what the agents did and when. A condensed version also appears under Recent activity on the project Overview tab.

Reviewing agent proposals

Agents set to manual autonomy do not act directly — every action they want to take becomes a proposal that waits for your decision, apart from an activity post, which is a log entry and always applies. Pending proposals appear in the Pending approvals card at the top of the Interactions tab and as a count badge in the project header.

Each proposal summarizes the action an agent wants to take — here Beacon proposing to create a task and send a message — with Approve and Reject controls.

• Approve applies the proposed action — the task is created, the message is sent, or the task update lands — and removes the proposal from the list.
• Reject discards the proposed action without applying it.

The card lists each proposal's summary, the agent that raised it, and how long ago. It is hidden entirely when nothing is pending.

Whether an agent proposes or simply acts is decided by its autonomy: Manual approval agents propose, while Suggest and Full autonomy agents apply their actions directly. Autonomy is set per agent — see Agent Heartbeat & Autonomy.

Troubleshooting

Related pages

• Agent Heartbeat & Autonomy — set each agent's cadence, autonomy, and per-beat limits, and review the beats it has run
• Projects — the project workspace and its budget that the team runs against
• Team — staff the project with the people and agents that collaborate here, and configure each agent
• Model Providers — manage the models, including bring-your-own models, that power your agents
• Organizations — the organizations that contain your projects

Monitoring

Track requests, review conversations, and manage support tickets

The monitoring section provides visibility into platform activity and user interactions.

Dashboard

Monitor platform activity with usage charts, programmable access analytics, and top credential usage

The Dashboard gives you an at-a-glance view of your AI assistant's activity and performance. Four metric cards summarize request volume, response speed, answer quality, and user engagement. Below them, interactive charts break down usage by time, channel, confidence level, and response time — and when programmable traffic is present, additional charts surface interface mix, block rate, caller type, and the credentials driving the most requests.

The Dashboard combines interactive KPIs with usage charts and programmable access analytics.

Metrics cards

Four cards at the top of the page show your tenant's key performance indicators for the selected filter range.

Metric cards update automatically whenever you change the date range or interface filter.

When a service credential produced at least one request in the selected range, the fourth metric card changes its label from Active Users to Active Users / Credentials and displays a compound value such as 1,284 / 7. The first number is the count of distinct user IDs across every audit record in the range — credential-issued requests carry a user identity too, so this is not a count of interactive humans only. The second number is the count of distinct service credentials that produced at least one request.

Core charts

Requests over time

A line chart showing daily request volume across the selected range. Use it to identify usage patterns, spot traffic spikes, and track adoption trends.

• X-axis — date in MM/DD format
• Y-axis — number of requests
• Days with no requests render as zero rather than gaps
• A toggle in the top-right of every chart switches the view between the visual chart and an accessible data table

Requests by channel

A bar chart breaking down request volume by messaging channel. Each channel appears in the legend above the chart. The palette holds four colours and repeats beyond that, so with many channels in range two bars can share one — read the legend rather than the colour. Use this to understand which channels your users prefer — for example, whether Teams traffic is eclipsing Web Chat.

Confidence level trends

A donut chart showing the distribution of AI confidence scores across three bands:

The center of the donut displays the total number of scored responses.

Response time distribution

A donut chart categorizing response times. It normally shows three buckets:

Each bucket shows the count and percentage of total responses. Those three strings are the exact labels the UI renders — the legend, the tooltip and the accessible table all print them verbatim — and the table's columns are Range, Count and Percentage. There is no fourth bucket: the ranges are open-ended at the top and response times cannot be negative, so every measured response falls into one of the three.

Programmable Access analytics

When at least one request in the selected range came from a service credential — an API, CLI, or MCP integration — an additional row of three charts appears below the core charts. This row is hidden whenever no programmable traffic is present, so tenants that only use Teams or Web Chat continue to see the classic dashboard without visual clutter.

The Programmable Access row appears whenever programmable traffic is detected in the selected range.

Each of these three cards has the same chart/table toggle as the core charts. Use the table view when you need exact values — the charts are sized for a quick read, and their x-axis tick labels can be cut off at the bottom of the card.

Requests by interface

A bar chart splitting traffic into two buckets:

• programmable — requests that entered through a service credential (API, CLI, MCP)
• traditional — requests that entered through a human-facing channel (Teams, Web Chat, WhatsApp, Mobile App, SMS, Phone, Email)

Use this chart to see how much of your volume is driven by automations versus interactive users.

Block rate by interface

A bar chart showing the percentage of requests that were blocked by a guardrail. Despite its title, the breakdown is by channel, not by interface: the chart draws one bar per channel present in the range, with no programmable/traditional bucketing — so traditional channels such as Teams and Web Chat appear alongside API, CLI and MCP.

In the table view, the rows are channels — the "Interface" column header notwithstanding.

The table view also reports the Total requests and the count of Blocked requests per channel, so you can contextualize the percentage against traffic volume. A sustained high block rate on a programmable channel typically indicates that an integration is sending malformed input or hitting a content-safety rule — review the blocked requests in Conversation Logs to confirm.

Requests by Caller Type

A bar chart grouping traffic by the caller type declared on the service credential that issued the request. Caller type is a property of the credential, so only programmable requests (API, CLI, MCP) carry one:

This chart lets you see at a glance whether your agentic traffic is growing relative to human-operated and application traffic.

Top credentials

Below the Programmable Access row, a Top Credentials table lists the service credentials with the highest request volume in the selected range. The table is hidden when no credential usage exists.

The Top Credentials table is sorted by request volume and color-codes block rate for quick triage.

The Block Rate column uses a color dot next to the percentage:

• Green — block rate at or below 5% (healthy)
• Yellow — block rate between 5% and 20% (investigate soon)
• Red — block rate above 20% (urgent — the credential is producing frequent guardrail violations)

Click any row to jump to Programmable Access. The click opens that page, not the specific credential — locate the credential in the table there to open its details drawer or rotate its secret.

Filtering

Use the filter row in the sticky header to refine every metric and chart simultaneously.

The Interface filter is the quickest way to isolate programmable traffic — pick Programmable Only to hide interactive users from every chart and metric, or pick a specific channel like MCP to focus on a single integration type.

All metric cards and charts update whenever you change any filter. Filter selections are preserved in the URL as query parameters, so you can bookmark or share a filtered view.

Refreshing the Dashboard

Click the refresh icon next to the page title to re-fetch all metrics and charts without changing any filters. This is useful after a burst of traffic — for example, right after a product launch — when you want to see the latest numbers without losing your current filter state.

Field reference

Related pages

• Conversation Logs — drill into individual conversations and filter by caller type
• Programmable Access — manage service credentials surfaced in the Top Credentials table
• Tickets — review escalated conversations driven by low-confidence responses
• Confidence Thresholds — adjust the bands that drive Success Rate and the Confidence Level chart

Conversation Logs

Browse, filter, and review user conversations with audit details, caller type filtering, and false positive marking

Conversation Logs give you a complete view of every interaction between users — and service integrations — and the AI assistant. Browse conversations, filter by channel, caller type, or confidence, review audit details for individual messages, and flag guardrail triggers as false positives. The page is reachable from the Log item in the main navigation.

One card is a whole conversation, not a single question. Every summary figure on it — message count, confidence, response time, the preview text — describes only the turns that match your current filters, not the whole thread.

Browsing conversations

The page header carries a List / Table view toggle. The page opens in List view, which renders each conversation as a card; Table view renders the same conversations as rows and is the only view with sort controls.

List view

Each conversation card displays:

A Refresh button beside the filters re-runs the current query — the page loads once and does not poll, so this is how you pick up conversations that arrived since.

Click any card to open the details drawer on the right side of the page. Every card carries a coloured left border already — that is its confidence band — so the selected one is marked differently, with a tinted background and a ring around it.

Table view

Switch the header toggle to Table to see the same conversations as a sortable table:

Table view adds the Last Answer column and the sortable column headers; the active sort column shows a single arrow.

Last Answer appears only in Table view — the conversation cards do not show it. Clicking a row opens the same details drawer as clicking a card.

Loading more conversations

The list loads 10 conversations at a time. A Load More button below the list (or table) fetches the next 10. Paging is cursor-based, so changing a filter or a sort restarts the list from the top. The message thread inside the details drawer pages the same way, through a Load More Messages button at the bottom of the thread.

Filtering and search

Search

Use the search bar at the top to find conversations by user name, question text, answer text, or session ID. Search is debounced — results refresh roughly half a second after you stop typing.

Advanced filters

Click Filters to open the advanced filters popover. It holds five controls — Channel, Caller Type, Response Time (seconds), Confidence Level (%), and Show false positives only — which you can combine without leaving the page. Search is a separate control in the page header, and the page has no date-range filter.

Every filter set here is written into the page URL, so a filtered view can be shared with a colleague as a link.

Active filters appear as chips below the search bar. Click the X on any chip to remove that filter.

Apply commits the draft selections in the popover and closes it.

Clear All immediately resets the five filters owned by the popover (Channel, Caller Type, Response Time, Confidence Level, False Positives Only) — their chips disappear and the query re-runs at once, so you do not need to press Apply afterwards — and the popover stays open so you can pick a new filter. It does not clear the search box or the current sort.

Channel reference

The Channel filter includes every interface that can produce a conversation, not just the traditional human-facing channels. This makes it possible to investigate programmable traffic directly from the log view:

Caller Type reference

Caller Type describes who produced the request. It is declared by the credential the request was made with, so only programmable traffic carries one:

Sorting

Sorting is performed in the Table view, reached through the List/Table toggle in the page header — the page opens in List view, and the card list has no sort control. In Table view, click a sortable column header — User, Channel, Response Time, Confidence, or Last Activity — to sort by it. Clicking the active column flips its direction; clicking a new column starts it ascending. The active column is announced to assistive technology through aria-sort. The default sort is Last Activity descending (newest first).

The sort itself is not tied to the view: it is sent to the API in both views and persists in the URL (sortField and sortDirection), so a sort chosen in Table view still orders the cards after you switch back to List. Only the control is missing from List view.

Conversation details

Click a conversation card to open the details drawer with two tabs: Overview and Audit.

This drawer is also where a guardrail trigger is marked a false positive, which is what feeds the per-guardrail false-positive rate on the guardrail analytics.

Overview tab

The Overview tab displays:

• Thread info — thread ID (copyable), message count, first and last message timestamps
• User info — display name, user ID (copyable), role, and channel — or credential name and caller type for programmable traffic
• Conversation stats — average confidence, average response time, total processing time, and AI sources used

Audit tab

Select a message in the conversation thread (listed below the tabs) to view its audit record. The Audit tab opens with a stat grid — Audit ID (copyable), Timestamp, Response Time, AI Source, and Execution Mode (shown only when recorded) — followed by these sections:

Model — how the message was routed and which models answered it:

AI Confidence — the confidence score as a percentage with a color-coded progress bar. If the response also has a vector confidence score (the similarity of the best match retrieval found), it is displayed separately. A Degraded chip appears beneath the bars when the answer's low score records a retrieval failure rather than a knowledge gap — the knowledge base could not be searched at all. Treat that as a platform fault to investigate, not as missing content to author.

Nearest Knowledge Base Match — when the confidence score is below the medium threshold, this section shows the closest matching question and answer from the knowledge base.

Governance — compliance status (Compliant, Not compliant, or Not checked), checked timestamp, reasoning, and per-guardrail results. Each guardrail result shows its compliance status, guardrail ID, and reasoning.

Escalation — the confidence band (High, Medium, or Low), whether the conversation was escalated, the escalation reason, and — when a ticket was created — the ticket ID as a copyable value (a truncated monospace string with a Copy button). The ticket ID itself is not a link; a separate View ticket button below it closes the drawer and opens the Tickets page with the ticket ID pre-filled in the search box, which resolves to that single ticket.

False positive marking

When a guardrail evaluation produces an incorrect result — for example, flagging safe content as non-compliant — you can mark it as a false positive. Flagging records a review annotation on that single guardrail result (who flagged it, when, and an optional reason) and feeds the per-guardrail false-positive-rate metric on the Guardrails page.

To remove a false positive flag, click the orange flag icon on a flagged guardrail and confirm the removal.

Field reference

Conversation card fields

Audit record fields

Related pages

• Dashboard — view aggregated confidence, caller type, and interface metrics
• Tickets — manage tickets created by escalation rules
• Guardrails — configure the guardrails evaluated in audit records
• Programmable Access — manage the service credentials that appear as API, CLI, and MCP conversations
• Confidence Thresholds — adjust the bands that drive the confidence filter

Tickets

View and manage escalated support tickets with status tracking, severity classification, and conversation details

The Tickets page displays support tickets created when conversations are escalated by your configured escalation rules. You can search, filter, and update tickets to track resolution progress, review the original conversation, and assign tickets to team members. Tickets are listed newest-updated-first by default, and Table view lets you re-sort them by creation or update time. A Refresh button beside the filters re-runs the current query — the page loads once and does not poll, so this is how you pick up tickets raised since you opened it.

The Tickets page displays all escalated conversations as actionable support tickets.

Key concepts

• Ticket — a record created when an AI conversation triggers an escalation rule. Each ticket captures the original question, the AI response, and metadata about the interaction.
• Status — tracks where a ticket is in the resolution workflow: Open, In Progress, Resolved, or Closed.
• Severity — indicates urgency: Critical, High, Medium, or Low.
• Confidence score — the AI's self-assessed confidence in its response, expressed as a percentage. Lower confidence often triggers escalation. Not every ticket carries one — see Conversation Tab.

Viewing Tickets

Tickets appear as cards in list view by default. Each card shows the ticket ID (truncated to its first 8 characters, with a copy button), an Escalation badge, the title and description, the assignee, the creator, status, severity, and the creation date. A confidence badge is shown only when the ticket carries an AI confidence score. Toggle between List and Table view using the buttons in the top-right corner.

The list shows 10 tickets per page. When there is more than one page, a pager appears below the list; it is hidden when all results fit on a single page. The current page is kept in the URL alongside the active filters — the page parameter is zero-based and omitted on the first page (page 2 is ?page=1) — so a filtered, paged view can be shared as a link. Changing a filter returns you to the first page, but opening a ?page=N link takes you straight to that page.

Tickets load newest-updated-first (updated_at descending) by default. To change the order, switch to Table view and click the Created or Updated column header — the two sortable columns. Clicking a column sorts by it; clicking the active column again flips the direction between descending and ascending, shown by the arrow on the header. Those are the only two sortable fields, and the card (List) view has no sort control, so the order can only be changed from Table view.

Searching and filtering

Use the search bar and filter dropdowns above the ticket list to narrow results:

The truncated 8-character ID shown on a ticket card will not match a ticket by ID — it is still substring-matched against the name, question, and answer text, so it may incidentally hit an unrelated ticket. Use the copy button on the card or in the details drawer to get the full ID, then paste it into the search box.

Active filters appear as dismissible chips below the filter bar. Click the X on any chip to remove that filter.

Viewing ticket details

Click any ticket card — or any row in Table view — to open the details drawer on the right side of the page.

The details drawer provides full ticket information across three tabs.

The drawer has three tabs:

Details tab

A Created by Escalation Rule banner is pinned at the top of the tab for tickets tagged escalation — which is every ticket the platform creates, since escalation rules are the only ticket-creation path.

Below the banner, the tab displays the ticket metadata:

Conversation tab

Shows the original interaction that triggered the escalation:

This is the exchange that triggered the ticket, captured as it was answered — editing the knowledge base afterwards does not change what is shown here.

• Original Question — what the user asked
• Answer — the response that was provided
• AI Response (Raw) — the model's own response text as recorded on the source conversation. Shown only when it differs from the answer that was delivered, which makes it uncommon
• AI Confidence — a labeled score with a progress bar showing how confident the AI was in its answer. Shown only when the ticket carries a confidence score

Audit tab

Shows governance and tracking information: the audit ID (with a copy button), the last update timestamp, and the ticket's status. When the ticket carries a chat/session ID, a View conversation in Log button appears and navigates to the full conversation in the Conversation Logs page.

Editing Tickets

You can update ticket status, severity, and assignment in two ways:

Inline editing

Click directly on a ticket card's Status chip, Severity chip, or Assigned To value to edit it in place. Select the new value from the dropdown, or type a new assignee name and press Enter — or clear the field entirely and commit to remove the current assignee (press Escape to cancel; clicking away also commits what you typed). Changes save automatically. Inline editing is available on the cards in List view only — in Table view, use the edit (pencil) action on a row to open the edit dialog.

Clicking the Assigned To value on a card turns it into a text field you can type a new assignee into.

Edit dialog

Click the Edit button on any ticket card to open the edit dialog. The dialog lets you update Status and Severity from dropdown menus, and set Assigned To in a free-text field (placeholder "Unassigned"); clearing that field and saving removes the assignee. Save Changes stays disabled until at least one of the three fields differs from its saved value; Close prompts you to confirm before discarding unsaved changes.

Field reference

Related pages

• Conversation Logs — view the full conversation that generated a ticket
• Dashboard — monitor overall platform activity and response metrics
• Settings — configure escalation rules that create tickets
• Guardrails — the safety and compliance checks that run on every turn

Content Management

Manage knowledge bases, knowledge packs, and guardrail policies for your AI assistant

Content management controls what your AI assistant knows and how it responds.

Knowledge Base

Create, search, and curate the Q&A pairs and document chunks that power your AI assistant's responses

The Knowledge Base is the central content store that powers your AI assistant's responses. It holds two types of items — Q&A pairs, which you author in the console or which the platform imports or generates from a file uploaded to a Q&A-type content pack, and document chunks that the platform extracts from uploaded files — and gives curators a unified workspace to search, filter, edit, rate, and govern every entry across the entire tenant.

The Knowledge Base page in Text Search mode, with the seeded Employee Handbook entries visible.

Key concepts

• Q&A pairs — questions and answers. Each pair has a question (what the user might ask) and an answer (what the assistant should respond with). Pairs reach the Knowledge Base by three routes: authored manually with Add Q&A, imported row-by-row from a CSV or XLSX file uploaded to a Q&A-type content pack, or generated by the platform from a PDF uploaded to a Q&A-type pack. Imported and generated pairs are created as Draft.
• Document chunks — when you upload a PDF to a document-type content pack, the system splits it into chunks. Each chunk contains a portion of the document together with the page numbers it came from. The same file uploaded to a Q&A-type pack produces Q&A pairs instead.
• Content packs — containers that group Q&A pairs or document chunks together. Most items belong to a pack, but a Q&A can be saved without one.
• Status workflow — items move through statuses that control visibility: Draft (hidden from users), Approved (live), Needs Attention (flagged for review), and Discarded (soft-deleted). Only Approved items are retrievable, so a new Q&A — which starts as Draft — is invisible to the assistant until you approve it.
• Vector index — Q&A pairs and document chunks are embedded as they are written, so there is nothing to rebuild and no reindex to schedule: a curator's change is searchable as soon as it saves. Every search is scoped to your tenant.

Search modes

The toolbar exposes two search modes via a toggle on the left of the search field. Switch between them by clicking the magnifying-glass icon (Text Search) or the sparkles icon (AI Search).

The search-mode toggle sits to the left of the search field, with Filters and Add Q&A to its right.

Text Search

Text Search is the default mode. As you type in the search field, results filter in real time across the question, answer, title, section heading, chunk content and tag fields — so a free-text tag that the Filters popover cannot offer is still findable by typing it here. Combine it with the Filters popover to narrow by tag, item type, status, sensitivity, and rating, and to change the sort order.

Active filters other than Item Type appear as chips beneath the toolbar, where you can clear them individually. Item Type has no chip — reset it from inside the Filters popover, either by setting Type back to All or by clicking Clear All, which lives inside the popover and resets every filter at once.

AI Search

AI Search runs a vector similarity search across the entire tenant knowledge base: your query is embedded and the nearest items are returned. Use it when you cannot remember the exact wording of an item, want to find paraphrased duplicates, or need to discover items by intent rather than by keyword.

It matches on meaning only — there is no keyword matching underneath it. An exact string such as a policy code, a part number or a person's name is what Text Search is for; a short identifier carries almost no meaning to embed, so AI Search is the wrong tool to look one up with.

AI Search returns ranked semantic matches with confidence labels, highlighting the full query phrase wherever it appears in a title or preview.

Each result card shows:

• A type badge (QnA or Document) with an icon
• A confidence chip combining the qualitative label (Low / Medium / High) and the normalized score percentage
• The current status of the underlying item
• For document chunks, the chunk position within the source file (Chunk 3/12)
• The title and content preview, with the full query phrase highlighted wherever it appears verbatim

The result list is sorted by score. The score is a cosine similarity between your query and the item, expressed as a percentage: it is a measure of closeness in meaning, not a percentage of correctness, and unrelated text still scores around 50%. The colored bar on the left of every card mirrors the confidence level so you can scan a busy result set quickly.

Running an AI Search

Advanced filters

Click Filters in the toolbar to open the filter panel. The panel opens in both search modes, but not every filter is applied in both — see the warning below.

Four of these — Tags, Sensitivity, Rating and Sort — narrow Text Search only. In AI Search their chips still render, but just Status and Item Type reach the query.

The tag vocabulary is a fixed built-in list — it is not derived from the tags in your tenant, so a tag you invented on an item will not appear as a filter option.

Creating a Q&A item

Viewing and editing items

Click any card — a Q&A pair or a document chunk — to open the details modal. The modal exposes two tabs at the top:

• Details — the editable view with status and sensitivity badges, quick-action buttons (Approve, Flag for Attention, Discard), inline-editable question and answer fields, rating, tags, and metadata.
• History — a chronological list of every prior version. The Compare button (enabled once two or more versions exist) opens a side-by-side diff view where you can pick a base version and a comparison target to inspect exactly what changed and who changed it.

For document chunks, the modal includes chunk navigation buttons (Previous / Next) to step through all chunks from the same source file, with a "Chunk N of Total" indicator.

Changing an item's status

There are two ways to move an item through the workflow:

• From the card — open the three-dot menu on any card (Q&A pairs and document chunks both have one) and choose Approve, Discard, or Flag as Needs Attention. The change saves immediately.
• From the details modal — open the item and use the quick-action buttons (Approve, Flag for Attention, Discard). These buttons only stage the new status in the open modal; you must click Save to persist it. If you close the modal without saving, the staged status change is discarded.

Approving is what publishes an item: every query filters out anything that is not Approved, so the assistant cannot retrieve a draft until the item is approved and the change is saved.

Deleting an item

Open the item's details modal and click Delete at the bottom-left. The item is deleted the moment you click — there is no confirmation prompt and no undo. This is a hard delete, distinct from the Discarded status, which only hides the item from users and can be reversed.

Reviewing ingestion runs

Edits you make here take effect when you save them — there is nothing to rebuild afterwards. To see what the platform ingested from an uploaded file, and how much knowledge came out of it, click Processing History in the toolbar; the page is described under Knowledge Packs.

Uploading files

Files are uploaded through Knowledge Packs. What the platform produces from a file depends on the type of the pack you upload it to.

Uploading to a document-type pack

When you upload a file to a document-type knowledge pack, the system automatically:

1. Extracts the PDF's text layer — the only document format a document-type pack accepts.
2. Splits the text into overlapping chunks, each carrying the page numbers it came from.
3. Creates individual document chunk items in the Knowledge Base, each created Approved.

Each chunk can then be reviewed, edited, approved, or discarded independently.

Uploading to a Q&A-type pack

A file uploaded to a Q&A-type pack becomes Q&A pairs, not document chunks. There are two paths:

• CSV or XLSX — imported directly, row by row, into question/answer pairs. Document parsing is skipped entirely. Both formats are accepted by Q&A-type packs only.
• PDF — parsed and chunked like a document, after which the platform generates Q&A pairs from the chunks.

Both paths create the pairs with Draft status, so nothing they produce is retrievable until you review and approve it.

Field reference

Q&A item fields

Document chunk fields

Page numbers are captured during parsing and kept on the record, but the console does not display them today. Chunks carry no section heading: text extraction yields the page's text, not the document's structure.

AI Search result fields

Related pages

• Knowledge Packs — create content pack containers and upload documents
• Guardrails — set safety policies that evaluate user input
• Conversation Logs — see how the assistant uses knowledge items in responses
• Confidence Thresholds — tune the minimum scores the runtime accepts before answering from a knowledge item

Knowledge Packs

Organize knowledge base content into collections with file uploads, content type selection, processing history, and bulk status management

Knowledge packs let you organize your knowledge base content into logical collections. Each pack groups related question-and-answer pairs or documents, making it easier to manage, upload, and maintain the content that powers your AI assistant's responses.

The Knowledge Packs page displays all packs as cards with content type badges, source counts, and action buttons.

Key concepts

Before working with knowledge packs, understand these core concepts:

• Content type — each pack is either QnA (question-answer pairs for direct lookup) or Document (whole documents for agent search without creating QnAs). You choose the type when creating a pack and it cannot be changed later. The console spells the content type QnA and an individual pair Q&A — this page follows the same convention.
• Sources — files you upload to a pack. Each source is first transferred from your browser (Uploading), then goes through the processing pipeline (Queued → Parsing → Chunking → Generating QnA → Indexing) before its content becomes searchable. Per-source progress and failures are shown inside the pack's details modal.
• Status — packs display an overall status derived from their sources: Syncing while any source is Queued, Parsing, Chunking, Generating QnA or Indexing; Error if a source failed and no source is in flight (in-flight work outranks a failure, so a source that is being retried reads Syncing, not Error); Active otherwise. A source that is still Uploading is deliberately not counted as in flight — the bytes are still with the browser — so on its own it does not make the pack Syncing: such a pack reads Active (or Error, if a sibling source failed). Stopped (cancelled) sources are likewise excluded from the in-flight set.
• Bulk actions — you can approve or draft all QnAs or document chunks in a pack at once, controlling whether content appears in search results.
• Searchable on write — there is nothing to sync. Processing embeds each chunk or Q&A as it stores it, so content is searchable the moment it lands. What decides whether the assistant may use an item is its status: only Approved items are retrievable, and approving one takes effect immediately. Past processing runs are listed on the read-only Processing History page.

Browsing Knowledge Packs

The Knowledge Packs page displays all packs as cards in a responsive grid. Each card shows the pack name, description, content type badge (QnA or Document), source and file counts, a Last processed timestamp for that pack (see the field reference), and quick-action buttons. The quick actions are Delete and, for QnA and Document packs only, the ⋮ bulk-actions menu.

Searching and filtering

Use the Search knowledge packs field at the top of the page to find packs by name. For more precise filtering, click Filters to open the advanced filters popover, where you can filter by status — Active, Syncing, or Error (the three states a pack can be in; see the status reference).

Active filter selections appear as chips below the search bar. Click the x on any chip to remove that filter, or use Clear All Filters in the popover to reset all filters at once.

Creating a Knowledge Pack

Managing pack details

Click any knowledge pack card to open its details modal. From here you can edit the pack's name and description, upload files, and manage sources.

The modal opens on the Files tab. The drop zone names the formats this pack accepts, and every source already uploaded to it is listed beneath, with its processing state.

Editing name and description

To rename a pack, hover over (or tab to) the pack name in the modal header and click the pencil button that appears — clicking the name itself does nothing. The description is directly clickable. Make your changes and press Enter to confirm or Escape to cancel, then click Save Changes to persist your edits to the server.

Uploading files

When processing will not start

The transfer and the processing are two separate steps, and the second one can be refused after the bytes have already landed. When that happens the toast names the reason, the file stays safely in storage, and the source waits in Uploading with Retry processing offered — nothing has to be uploaded again.

Source row controls

Every row in the Uploaded Files list carries a Download button that retrieves the original file exactly as it was uploaded — useful for checking what was actually ingested, or for recovering a file whose local copy is gone. It is disabled only while the source is still Uploading, when there is nothing stored yet to fetch. Retry processing and Stop processing appear only for the statuses described above.

In the same list, click the trash icon next to a source to remove it, along with everything derived from it and the uploaded file itself. The trash icon is disabled only while the source is in the processing pipeline (Queued, Parsing, Chunking, Generating QnA or Indexing) — stop it first, then delete it. Completed, failed, stopped and abandoned Uploading sources can all be deleted directly.

Bulk status management

Bulk actions let you approve or draft all items in a knowledge pack at once. This is useful when you have uploaded new content and want to make it all searchable, or when you need to temporarily hide all content from users.

Deleting a Knowledge Pack

To delete a pack, click the trash icon on the pack card. A confirmation dialog warns that the action is permanent and will remove the pack, all Q&A entries, and all uploaded files. Click Delete Knowledge Pack to confirm.

Processing history

There is no index to rebuild and nothing to trigger: each chunk and each Q&A pair is embedded and written as it is processed, so it is searchable as soon as it lands, and changing an item's status to Approved makes it retrievable immediately. What the platform records instead is the ingestion itself — one run per file processed.

Click Processing History at the top of the Knowledge Packs page to open that record. It is read-only, and it shows:

• Active run banner — while a file is being processed, a banner shows the run's status, when it started, who started it, and the file it is working on
• History table — a paginated table of past runs: what was ingested, how long it took, and how much knowledge came out of it

The page polls automatically every 3 seconds while a run is in progress, so you can watch it complete without reloading. Processing continues whether or not the page is open.

A run is recorded for each file that goes through the pipeline, whether it was uploaded to a Document pack, uploaded to a QnA pack for pair generation, or imported as a CSV or XLSX table of pairs. Bulk status changes and deletions are applied directly and are not runs, so they never appear here.

Field reference

Knowledge Pack card

Add Knowledge Base dialog

Source processing statuses

Processing history table

Related pages

• Knowledge Base — browse and manage individual Q&A items and document chunks
• Guardrails — set safety policies for AI responses
• Settings — configure confidence thresholds that affect how knowledge pack content is used
• Sandbox — test your AI assistant's responses using knowledge pack content

Guardrails

Review how each guardrail is performing — fire rates, compliance trend and false positives — and set its name, category and priority

Guardrails are the safety and compliance checks AskRAI runs over an interaction. This page is where you review how they are performing — how often each one fired, against what, and how many of those were false positives — and where you set a guardrail's name, description, category and priority.

Each card carries the guardrail's category, its priority and a preview of its prompt — the fields this page owns.

What this page can and cannot change

How the guardrails on this page behave at runtime

• Six kinds, five of them free. PII, regex deny, word deny, secret scan and link domain are evaluated in code and cost no model call. Only an LLM judge reaches a provider — and only a judge has a prompt.
• Three stages. A guardrail reads the user's question (Input), the generated answer (Output), or the arguments an agent is about to send to a tool (Tool call). A rule about how answers should be worded belongs on Output.
• Three actions. Block stops the turn, Warn lets it continue and leaves a record, Redact substitutes what it found and continues. One blocking guardrail is enough — there is no majority rule.
• Priority is evaluation order. The deterministic kinds run one after another in priority order, and the first one to block is the one reported as the reason. Judges run afterwards, at most three at a time. Once something has blocked, only the judges that would also Block are skipped — a Warn judge still runs, so it keeps firing on exactly the turns it was attached to watch. (A judge cannot be set to Redact: it returns a verdict, not the position of what it objected to.)
• Audit recording. Every evaluation, pass or fail, is recorded and counted in the analytics. No setting reduces this.
• Only Active guardrails run. A draft is listed and attachable and evaluates nothing. The status badge is in the editor here; the status itself is set in Settings.

Viewing guardrails

Three view modes, switchable from the toolbar:

• List view (default) — a card per guardrail with its name, category badge, priority chip, a preview of its prompt, and a fire badge.
• Table view — columns for name, description, category, priority, reporting level, fire count, fire rate and a sparkline trend.
• Analytics view — a compliance trend line, a category distribution breakdown, and a top-fired guardrails chart.

Table view sorts on Name, Category and Priority only — Description, Fires, Fire Rate and Trend carry no sort control. Note the pager: the list is ten guardrails to a page.

Guardrails are listed highest-priority first by default. Sorting, filtering and paging are all done by the server, so the sort applies to the whole set rather than to the page you are looking at.

Filter the list from the toolbar:

• Search — case-insensitive substring match across the name, description and prompt fields.
• Category — Safety, Compliance, Quality, Privacy, or Custom.
• Priority — a band: High (8–10), Medium (5–7), or Low (1–4).
• Reporting level — None, Alerts or All. This is stored metadata that no longer has an editor control and does not affect what is audited (see below); the filter is here because the value is still recorded and still shown in Table view.

Active filters appear as removable chips below the toolbar. Clear All lives inside the Filters popover, not next to the chips.

Creating a guardrail

Add Guardrail navigates to Settings → Guardrails, where authoring happens. That is the only surface carrying kind, stage, action and status — the four fields that decide whether a guardrail does anything at all — so a guardrail cannot be created from this page.

Once the guardrail exists, come back here to set its category and priority, and to watch it perform.

Editing a guardrail

Click any guardrail card or table row to open the editor. Name and description are edited in place from the header; category, priority and — for an LLM judge — the prompt are below.

The strip at the top is what this guardrail does at runtime — status, kind, stage and action — none of which this editor can change. It is shown because the fields beneath it read as the whole of a guardrail without it.

The Prompt field appears only for an LLM judge, and Save Changes is held until it has one: a judge with an empty prompt asks the model nothing. Every other kind is refused by the server for carrying a prompt at all, so no prompt is offered for them.

Priority never holds Save. A guardrail already carrying a value outside 1–10 has that value offered as its own option in the dropdown, so renaming a guardrail does not silently rewrite its priority.

Saving applies from the next turn onward, including turns in conversations already under way, and is in force everywhere within five minutes at the outside.

Platform guardrails

Guardrails that ship with AskRAI are read-only on both pages — an edit or a delete is refused. Neither List nor Table view marks them: the Platform badge appears once you open one, alongside the notice that it cannot be changed.

Every control is disabled and Delete is absent — the editor says so rather than letting you reach a permission error. To change one, duplicate it under Settings → Guardrails to get an editable copy.

Deleting a guardrail

Open the guardrail and click Delete at the bottom-left of the editor, then confirm. Deletion is permanent. Platform guardrails have no Delete control.

Field reference

Analytics

Switch to the Analytics view to see guardrail performance over a configurable date range. That view has its own date picker (with 30d/60d/90d presets) and a Channel filter, separate from the Analytics period picker used by List and Table view. The Analytics picker allows up to 90 days; the List/Table picker is limited to a 30-day span. In both, a wider selection is silently clamped — and the fire-rate API backing the List/Table fire badges rejects ranges over 30 days.

• Compliance trend — the share of blocking guardrail evaluations that passed. Warn and Redact results are left out: they found something and let the turn through, so how often those fire is what the fire count and the Top Triggered chart show instead. It counts evaluations rather than conversations, so a message screened by three guardrails contributes three results.
• Category distribution — fires broken down by category.
• Top Triggered Guardrails — guardrails ranked by fire count, not by priority. Each bar's label also carries that guardrail's fire rate and, where one exists, its false-positive rate — the share of its fires later marked as false positives in the conversation logs. There is no separate false-positive chart. Clicking a bar opens that guardrail's editor, but only when the guardrail is on the page of the list currently loaded — the list is ten to a page, so for anything further down, page to it in List or Table view first.

In List view each card shows a fire badge with the total for the selected range, and beneath it the fire rate as a percentage of conversations plus a sparkline. The badge colour reflects the rate: green below 15%, orange from 15% up to 50%, red at 50% or more. Table view shows the same numbers as plain columns.

Troubleshooting

Related pages

• Guardrails (concept) — kind, stage and action, and how an evaluation runs
• Settings → Guardrails — author a guardrail and set what it does
• Policies — attach a set of guardrails, and ceilings, to a group or an agent
• Users, Roles & Groups — attach guardrails to a group
• Conversation Logs — review evaluation results and mark false positives

Administration

Manage users, roles, groups, directory sync, organizations, sandbox testing, and platform configuration settings

Administration covers user and organization management, identity provisioning, testing, and platform configuration.

Users, Roles & Groups

Manage user accounts, define roles with mapping rules including caller and credential types, and organize access through groups

User Management is your central hub for access management. Users represent real people in your organization, roles define usage patterns with automatic mapping rules, and groups tie users and roles together with access to specific knowledge packs and guardrails. Find it under Settings → Users.

The Users tab displays all user accounts with their status, department, and group memberships.

What this page does not control

Everything on this page — groups, roles, mapping rules — decides which knowledge and guardrails apply to an interaction. It does not decide who may administer AskRAI. That is a separate, tenant-level question answered on the Access page, which grants the Platform Admin and Member roles. Adding someone to an "Admin Access" group here does not let them into Settings.

Users tab

The Users tab shows all user accounts provisioned through Directory Sync — whether your identity provider pushes them over SCIM 2.0 or AskRAI pulls them from Google Workspace. Each user card displays the person's name, job title, department, contact information, last login time, and group memberships.

User status

Users have one of two statuses:

Administrators can block or unblock users by updating their status.

User details drawer

Click any user card to open the details drawer with three tabs:

• Details — personal information including name, job title, department, employee ID, email, phone, and address
• Activity — account status, current online state, last login timestamp, and active session details (device, browser, location)
• Groups — lists all groups the user belongs to, with member counts and knowledge pack access

Filtering Users

Use the search bar and filter dropdowns to narrow the list:

Groups tab

Groups are permission containers that connect users and roles to knowledge packs and guardrails. A user's access to knowledge base content is determined by their group memberships.

Each group card shows its members, assigned roles, accessible knowledge packs, and applied guardrails.

Creating a group

Group types

Group fields

Roles tab

Roles represent usage patterns rather than individual people. Each role defines mapping rules that automatically classify incoming interactions — for example, routing requests from a specific channel to the appropriate access level.

Each role card displays its mapping rules, active user count, and enabled/disabled status.

Role properties

Mapping rules

Mapping rules define when a role applies based on interaction properties:

All of a role's rules must match for the role to apply, and a user receives the combined access of every role that matches — there is no ordering between roles and no first-wins.

Creating a role

Switch to the Roles tab and click Add Role. Enter a name, select a category, and optionally add a description. After creation, open the role to configure mapping rules.

Directory tab

The Directory tab is a read-only view of the identities provisioned through Directory Sync, shown separately from the native Groups and Roles you manage in AskRAI. Use it to confirm that a sync took effect and that the right people and groups arrived.

The Directory tab lists every synced user and group, each tagged Synced from directory.

The tab is split into two sections:

• Directory Users — every synced person, with their name, email, and a Synced from directory badge. A user suspended or removed in the source directory is tagged Deprovisioned.
• Directory Groups — every synced group, with its member count and source badge.

Use the Search directory box to filter both lists by name or email. If no source is enabled yet, the tab prompts you to configure Directory Sync; if a source is enabled but the first sync hasn't completed, it shows an "awaiting first sync" state.

How access control works

The access control model follows this hierarchy:

1. Users are real people synced from your identity provider
2. Roles classify interactions automatically via mapping rules
3. Groups bundle users and roles together
4. Groups grant access to knowledge packs and guardrails

A user's effective access is the union of all knowledge packs and guardrails from every group they belong to.

Administrative access runs on a separate track: a user's tenant role — Platform Admin or Member — is granted on the Access page and decides whether they can open Settings at all. The two are independent, so a Member can still hold rich content access through their groups, and a Platform Admin has no extra knowledge access just for being an admin.

Related pages

• Access — grant the Platform Admin and Member tenant roles to the users and groups managed here
• Directory Sync — provision the users and groups shown on the Directory tab from SCIM 2.0 or Google Workspace
• Knowledge Packs — content collections that groups can access
• Guardrails — safety policies assigned to groups
• Programmable Access — issue credentials whose Caller Type and Credential Type are matched by the role rules described above
• Sandbox — test how a credential resolves to a role and group before pointing live traffic at it
• Settings — configure system-wide settings and escalation rules

Directory Sync

Provision users and groups into AskRAI from your identity provider via SCIM 2.0, or pull them from Google Workspace with the Admin SDK Directory API

Directory Sync keeps AskRAI's users and groups in step with your identity provider so access management always reflects your real org. Open it under Settings → Directory Sync. AskRAI supports two directory sources — pick the one your identity provider uses.

Directory Sync presents both sources on one page: generic SCIM 2.0 at the top and Google Workspace below. A tenant uses one source at a time.

Choosing a directory source

Generic SCIM 2.0

With SCIM provisioning, your identity provider pushes user and group changes to AskRAI in near real time. Enable it, generate a bearer token, and paste the endpoint URL and token into your IdP's provisioning settings.

To replace a token, click Rotate token — this immediately invalidates the previous one, so update your IdP right away. Revoke removes the token entirely and stops inbound provisioning.

SCIM connection reference

Google Workspace Directory Sync

Google Workspace can't push SCIM 2.0 to third-party apps, so AskRAI reads your directory through the Google Admin SDK Directory API using a service account with domain-wide delegation. There is no service-account key to download, upload, store, rotate, or leak — AskRAI authenticates with keyless Workload Identity Federation (WIF). AskRAI acts as its own OpenID Connect identity provider, mints a short-lived token, and federates it into your GCP project, which you configure to trust AskRAI. You connect AskRAI with six non-secret identifiers — your workspace domain, an admin email, the service-account email, your GCP project number, and the workload identity pool and provider IDs. No credential ever leaves Google.

The issuer, subject and JWKS URI are values you paste into GCP, not values you enter here — they are what your project trusts, which is why no key ever changes hands.

Set up

principal://iam.googleapis.com/projects/<project-number>/locations/global/workloadIdentityPools/<pool-id>/subject/<federation-subject>

Connection reference

AskRAI also displays, read-only, the federation issuer, federation subject, and JWKS URI you authorize in GCP (steps 3 and 4). These are AskRAI-side values — copy them into your pool and IAM binding; you never enter them.

Sync controls

How sync works

AskRAI reconciles your Google directory into the same directory store as SCIM, so synced users and groups appear in Users, Roles & Groups and can back native groups — the rest of the platform doesn't care which source provisioned them.

• Scheduled pull — users, groups, and group members are polled on an interval. Use Sync now to run an on-demand sync.
• Near-real-time user changes — when Real-time user sync is on, AskRAI registers a Google users.watch channel so user changes (including offboarding) propagate within seconds. Groups and members are always reconciled on the scheduled pull (Google offers no group watch channel).
• Deprovisioning — a user suspended or removed in Google Workspace is marked inactive in AskRAI on the next sync, revoking their access. Identities map by Google's stable user id, so a synced user lines up with the same person who signs in via Google.

Where synced users appear

Provisioned users and groups land in the directory store and show up on the Users, Roles & Groups page. The Directory tab there lists every synced identity with a Synced from directory badge, so you can confirm a sync took effect.

Troubleshooting

Related pages

• Users, Roles & Groups — where synced users, the Directory tab, and native groups live
• Programmable Access — issue credentials for machine callers alongside human identities
• Settings — the rest of the platform configuration

Organizations

Group your projects under tenant-scoped organizations, switch the active organization to scope the Projects workspace, and manage each organization's name, slug, and plan tier

Organizations are tenant-scoped containers that group your projects. The active organization scopes the Projects workspace, so switching organizations switches which projects you see and work in. Manage them under Settings → Organizations, in the Platform section of the Settings sidebar.

Each row shows an organization's name, slug, and plan, with the active organization marked. The header switcher (top) reflects the same active organization.

How organizations fit in

A tenant can hold many organizations, and each organization groups its own projects. Exactly one organization is active at a time. It scopes the Projects plane and also the Skills library and tool-server grants, so switching organization changes what Settings shows you as well as which projects you see. The choice is remembered across sessions.

• Tenant → organizations → projects — organizations sit between your tenant and your projects. Creating projects requires an organization to put them in.
• Active organization — the organization currently in context. It carries an Active badge in the list and its name appears in the header org switcher.
• Identity — every organization has a name, a slug that is unique within the tenant, and a plan tier shown as a badge.

Switching the active organization

The active organization is shown in the org switcher in the top header — the button labelled with the organization's name and plan. Open it to move between organizations without leaving the page.

The org switcher lists every organization in the tenant and highlights the active one; the New organization item jumps to the Organizations page with the create dialog open.

You can switch the active organization two ways:

• From the header switcher — open it and choose an organization.
• From the Organizations page — click any organization's name to make it active.

Switching the active organization reloads the Projects workspace, and the Skills library with it — both are org-scoped, so Settings changes under you as well as the Projects plane. The switcher appears only once the tenant has at least one organization.

Creating an organization

Click New organization — either the button on the Organizations page or the shortcut at the bottom of the header org switcher — to open the create dialog.

The slug is derived from the name as you type; you can override it before creating.

Organization fields

Plan tiers

The plan tags an organization with a tier and is shown as a badge in the list and the switcher. Nothing else in AskRAI reads it — it changes no limit and unlocks no feature. Tiers are listed in upgrade order.

Editing and deleting

Each organization row has an actions menu (the ⋮ button) with Edit and Delete.

Delete asks for confirmation and is refused while the organization still owns projects — move or delete those first.

Edit an organization

Choose Edit to open the edit dialog, where you can update the Name, Slug, and Plan. The slug is required when editing and must stay unique within the tenant. Click Save changes to apply.

Delete an organization

Choose Delete to open a confirmation dialog titled Delete organization?. It names the organization and warns that the deletion is permanent and cannot be undone; click Delete to confirm.

Troubleshooting

Related pages

• Users, Roles & Groups — manage who can access the platform alongside how work is organized
• Directory Sync — provision users and groups from your identity provider
• Settings — the rest of the platform configuration

Sandbox

Test AI responses with configurable guardrails and credentials, and inspect search, audit, identity, and guardrail results

The Sandbox lets you test AI responses in a safe environment before deploying configuration changes. Send messages, select which guardrails and knowledge packs to apply, and inspect every detail of the response pipeline — including guardrail evaluations, search results, audit records, and model information.

Nothing you send here reaches an end user, and nothing is written to your audit trail — the Audit tab shows a preview of the record this query would have produced. It is still a real model call, so it costs tokens.

Page layout

The Sandbox is divided into three sections:

• Configuration panel (left) — select guardrails, knowledge packs, or a group to test with
• Chat panel (right) — send messages and view AI responses
• Results inspector (bottom, collapsible) — inspect guardrail results, search debug data, audit preview, and model information for any response

Configuration

The configuration panel lets you choose between Custom and Group mode.

Configuration modes

Choose one of two modes at the top of the configuration panel:

When you switch modes, selections from the previous mode are cleared automatically.

In Custom mode, two chip lists appear — each a scrollable bordered box, not a dropdown:

• Guardrails — click a chip to toggle that guardrail on or off for the next message
• Knowledge Packs — click a chip to toggle that knowledge pack into the search scope

Selected chips are highlighted. If nothing is configured yet, the list shows None available.

In Group mode, a single dropdown lets you select a group. After selection, the panel displays the knowledge packs resolved from that group's configuration, and the guardrails attached to the group directly. Guardrails the group inherits from an attached policy are not listed here, though they do run — so the preview can show fewer controls than the turn actually applies.

Context overrides

Expand the Context Overrides section to simulate different user contexts and, for programmable access work, impersonate a specific credential:

The Channel selector offers only those six channels (plus None). SMS, Phone, Email and Mobile App exist in the platform-wide channel list but cannot be simulated from the Sandbox.

What Test as Credential inherits

• Caller type — taken from the selected credential.
• Channel — taken from the credential, unless you set the Channel override to something other than Web yourself.
• Group — taken from the credential only when you have not selected a group and have not selected any guardrails in Custom mode.

The picker lists only credentials whose status is active, capped at the first 100. A revoked credential does not appear — it must be re-issued on the Programmable Access page before it can be tested here. Credential status is independent of expiry, so a credential that is past its expires_at date but has not been revoked still appears in the list.

Starting a new conversation

Click New Conversation to abort any in-flight request and clear the messages, the thread ID and the open Results Inspector.

Sending messages

Type a message in the text field at the bottom of the chat panel and press Enter or click the send button. The Sandbox runs your message through the full pipeline under the configuration you selected.

Each response appears as a chat bubble. User messages align to the right, assistant responses to the left, each with a timestamp.

Draft model routing

The Sandbox always evaluates your draft routing configuration — there is no toggle. When your tenant has a draft model-routing configuration, the Sandbox evaluates the draft; if there is no draft, it falls back to the published configuration. There is nothing beneath that — a stage your configuration never assigns a model to has no platform model to fall back on. This is what lets you exercise unpublished routing rules before publishing them, but it also means the Routing Decision shown on the Model Info tab reflects the draft, not necessarily what live traffic is using.

A rule on Group ID is exercised here, against whichever group is actually governing the turn — the one you selected, or the credential's own group when you simulate a credential and override neither the group nor the guardrails. Choose explicit guardrails alongside a credential and no group governs the turn, so a Group ID rule matches nothing. A rule on Role Category is never exercised: the Sandbox names a group outright instead of matching the caller against your roles, so it has no matched role to report a category for, whatever that rule does in production.

Blocked responses

If a guardrail tripwire triggers, a yellow warning banner appears above the assistant message. The banner lists which guardrails caused the block.

Inspecting results

Click the eye icon on any assistant message to open the Results Inspector panel below the chat. The inspector has five tabs, in this order — Guardrails (the default active tab), Audit Preview, Identity & Access, Search Debug, and Model Info.

Guardrails tab

Shows the evaluation results for each guardrail applied to the response:

• Summary bar — displays how many guardrails passed (for example, "2/3 guardrails passed")
• Per-guardrail cards — each card shows the guardrail name, category, pass/fail status, and expandable reasoning

If a tripwire triggered, an error alert appears at the top listing the blocking guardrails.

Audit preview tab

Displays the audit record that would be created in production. The panel renders these sections, in order:

Below the sections, the tab shows the Processing Time in milliseconds and ends with an expandable Raw Audit JSON viewer (with copy-to-clipboard) containing the full preview payload.

The audit preview also evaluates the response against your tenant's escalation rules in real time.

Identity & Access tab

The Identity & Access tab shows how AskRAI resolved the caller for this request — essential when debugging Programmable Access credentials, role mapping rules, or agent pipelines. The tab is organised into up to four sections:

The Agent Augmentation section requires both conditions: the request must have been executed in Agent Augmentation mode and run against a credential, since the section renders inside this tab. Its fields mirror the structured context that the runtime returns to downstream agents, letting you verify the exact payload your agent will receive before shipping.

Search debug tab

Shows how retrieval processed your query:

• Original query — what you typed
• Refined query — the query after AI refinement
• Results table — top knowledge base matches ranked by relevance score, with title, content preview, and source

Model info tab

The tab opens with the routing decision that selected the model, then shows the model used at each pipeline stage and the total processing time.

Routing decision

The Routing Decision section is always shown — including when no routing rule applied, in which case the source reads Not configured. Because the Sandbox evaluates your draft routing configuration, this section tells you what a draft rule would do before you publish it.

When more than one distinct rule contributed, an Extra Details (N rules contributed) expander appears. Expanded, it lists the non-summarization contributing rules by stage (for example Tool Call Rule or Guardrail Rule) and then shows the Refinement and Guardrail model chips.

Models and timing

Related pages

• Guardrails — create and manage the guardrails you test in the sandbox
• Knowledge Base — manage Q&A content that powers search results
• Knowledge Packs — organize knowledge items into collections
• Users, Roles & Groups — configure groups used in Group mode and Caller Type or Credential Type role rules
• Programmable Access — issue the credentials impersonated by the Test as Credential selector
• Settings — set confidence thresholds and escalation rules that affect audit previews

Model Routing

Configure which AI models handle different pipeline stages and define rules to route requests based on context

Model Routing controls which AI models process each stage of the response pipeline. Configure default models, enable automatic complexity-based routing, and create rules that match requests by channel, authentication status, device type, or user group. Changes follow a draft-and-publish workflow with full version history and rollback support.

To open it, switch to the Settings plane from the top navigation bar and select Routing in the Settings sidebar.

Everything below the publish bar is a draft. Nothing on this page affects a live query until you publish it.

Key concepts

Before configuring routing, understand the three layers that determine which model handles a request. They are evaluated in order of precedence:

There is no fourth layer beneath these. Every model AskRAI runs is one your tenant registered under Model Providers, so a stage that no layer assigns has nothing to fall back on — and what happens then depends on the stage. Summarization has nowhere to go without one, so the request fails; guardrail evaluation fails the same way, but only for a tenant whose guardrails include an LLM judge — the deterministic kinds need no model. Query refinement is the only stage that degrades quietly — it answers on the unrefined question. Confidence assessment fails closed: every answer it scores comes back 0.0, so the answer bands Low for the user, 0.0 is what the audit record keeps, and any Escalation Rule on the Low band fires on that conversation. It scores every generated answer — a curated answer served straight from the knowledge base is banded from its own score and is unaffected. The retrieval relevance score is recorded alongside it but is not what bands the answer. Set a tenant default for every stage you rely on.

Pipeline stages

Each request passes through up to four pipeline stages. You can assign a different model to each:

Draft and publish workflow

Routing configuration uses a draft-and-publish workflow. Changes you make on the page are local edits until you save them as a draft. The draft must then be published to take effect in production.

The publish bar at the top of the page shows:

Default models

The Default Models section sets tenant-level fallback models for each pipeline stage. If no routing rule or auto-route tier assigns a model for a stage, the default is used.

Select a model from the Summarization Model dropdown to set the primary default. Expand the Advanced section to override the Query Refinement, Guardrail and Confidence Assessment models separately.

Leave a dropdown on No override and that stage has no tenant default at all. Nothing falls through to the Summarization Model — it is the summarization stage's own default and no other stage's. The single exception is Confidence Assessment, which reuses the Query Refinement Model when its own is left on No override. Routing rules and auto-route tiers fall through to these defaults; nothing falls through beneath them, which is the state the note above describes.

Auto-route

Auto-route automatically classifies each incoming query by complexity and routes it to the most appropriate model. This optimizes cost and performance by sending simple queries to faster models and reserving powerful models for complex requests.

The classifier is itself a model call on every query, so it is worth pointing at a cheap, fast model — its only job is to decide which tier the question belongs to.

Enabling auto-route

Toggle Enable auto-route classification to activate this feature. When enabled, two additional configuration areas appear:

• Classifier Model — the model that evaluates query complexity. Every query pays its latency, so a small, quick model is the right choice.
• Tier Mapping — assign a summarization model (and optionally query refinement and guardrail models) for each complexity tier.

Tier configuration

For each tier, select a primary model from the dropdown. Expand Advanced under each tier to set stage-specific overrides for query refinement and guardrail models.

Routing rules

Routing rules let you assign models based on request context. Rules are evaluated by priority — lower numbers run first. Multiple matching rules can contribute different stages (one rule might set the summarization model while another sets the guardrail model).

Creating a rule

Condition fields

Each condition matches a request attribute against a value:

Each field below (except Group ID) is a fixed dropdown — these are the complete option lists, not examples:

Group ID matches against every group the caller resolved into — those assigned to them directly, and those reached through a role whose mapping rules matched the request. Automatic groups are not among them: nothing supplies the user attributes those rules read, so an automatic group never contributes to routing. Role Category matches the categories of the roles the request matched, whether or not those roles lead to a group. A caller who resolves into no groups at all is a real answer, not a blank: a not_in rule is satisfied by it.

Operators

Testing rules

The Test Rules section at the bottom of the page lets you simulate a request and see which rules match and which models are resolved for each stage.

Select values for the context fields (channel, auth status, device type, group ID, role category, and complexity tier if auto-route is enabled), then click Evaluate. The result shows:

• Source — where the final model selection came from (Tenant Default, Routing Rule, Auto-Route, Merged Rules, or Not configured when nothing assigned the stage a model)
• Resolved Models — which model handles each pipeline stage (summarization, refinement, guardrail) and the source of each assignment
• Matched Rules — which rules matched the test inputs, sorted by priority

Version history

Click History in the publish bar to open the version history drawer. Each published configuration is stored as a numbered version.

For each version, you can:

• Compare an archived version with the current published version to see what changed
• Rollback to restore an archived version as the active configuration