Evaluate

Submit a call or chat for evaluation.

Returns an eval ID that can be used to track evaluation status.

POST /v1/evaluate

Evaluate

cURL

curl --request POST \
  --url https://api.getbluejay.ai/v1/evaluate \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: <x-api-key>' \
  --data '
{
  "agent_id": "<string>",
  "start_time_utc": "2023-11-07T05:31:56Z",
  "participants": [
    {
      "role": "AGENT",
      "spoke_first": true,
      "name": "<string>",
      "phone_number": "<string>"
    }
  ],
  "recording_url": "<string>",
  "recording_base64": "<string>",
  "call_direction": "INBOUND",
  "interface": "PHONE",
  "transcript_url": "<string>",
  "transcript": [
    {
      "start_offset_ms": 123,
      "end_offset_ms": 123,
      "speaker": "AGENT",
      "utterance": "<string>",
      "filler_utterance": true
    }
  ],
  "tool_calls": [
    {
      "name": "<string>",
      "start_offset_ms": 1,
      "description": "<string>",
      "parameters": {},
      "output": "<unknown>"
    }
  ],
  "events": [
    {
      "title": "<string>",
      "start_offset_ms": 123,
      "description": "<string>",
      "end_offset_ms": 123,
      "tags": [
        "<string>"
      ],
      "metadata": {}
    }
  ],
  "tags": [
    "<string>"
  ],
  "trace_id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "metadata": {},
  "webhook_tag": "<string>",
  "custom_metrics": [
    "3c90c3cc-0d44-4b50-8888-8dd25736052a"
  ],
  "custom_metric_tags": [
    "<string>"
  ],
  "label": "<string>",
  "enable_phrase_latencies": true,
  "conversation_ended_by": "AGENT",
  "multichannel": false
}
'

200

{
  "status": "<string>",
  "call_id": "<string>",
  "message": "<string>"
}

Integration Prompt for AI Agents

# Bluejay — Testing & Monitoring Platform for Conversational AI Agents

You are a senior backend engineer integrating the Bluejay API. Think step-by-step: first understand the endpoint, then plan the integration, then implement with minimal changes.

## Evaluate — POST /v1/evaluate

> **What this endpoint does:** Submit a call or chat for evaluation. Returns an eval ID that can be used to track evaluation status.

**Endpoint:** POST `https://api.getbluejay.ai/v1/evaluate`
**Auth:** `X-API-Key` header
**Content-Type:** application/json

### Required Parameters
| Name | Type | Description |
|------|------|-------------|
| X-API-Key | string | API key required to authenticate requests. |

### Request Body
```json
{
  "agent_id": "string",
  "recording_url": "string",
  "recording_base64": "string",
  "start_time_utc": "2024-01-01T00:00:00Z",
  "participants": [
    {
      "role": "AGENT",
      "spoke_first": true,
      "name": "string",
      "phone_number": "string"
    }
  ],
  "call_direction": "INBOUND",
  "interface": "PHONE",
  "transcript_url": "string",
  "transcript": [
    {
      "start_offset_ms": 123,
      "end_offset_ms": 123,
      "speaker": "AGENT",
      "utterance": "string",
      "filler_utterance": true
    }
  ],
  "tool_calls": [
    {
      "name": "example_name",
      "start_offset_ms": 123,
      "description": "string",
      "parameters": {
        "key": "value"
      }
    }
  ],
  "events": [
    {
      "title": "string",
      "start_offset_ms": 123,
      "description": "string",
      "end_offset_ms": 123,
      "tags": [
        "string"
      ],
      "metadata": {
        "key": "value"
      }
    }
  ],
  "tags": [
    "string"
  ],
  "trace_id": "string",
  "created_at": "string",
  "metadata": {
    "key": "value"
  },
  "webhook_tag": "string",
  "custom_metrics": [
    "string"
  ],
  "custom_metric_tags": [
    "string"
  ],
  "label": "string",
  "enable_phrase_latencies": true,
  "conversation_ended_by": "AGENT",
  "multichannel": true
}
```

### Example
**POST with body:**
```python
import requests

def evaluate(payload: dict, api_key: str) -> dict:
    url = "https://api.getbluejay.ai/v1/evaluate"
    headers = {"X-API-Key": api_key}
    response = requests.post(url, headers=headers, json=payload)
    response.raise_for_status()
    return response.json()
```

### Constraints
- Minimal changes — only add/change files needed for this integration.
- Match existing codebase patterns (naming, file structure, error handling).
- Include error handling for 422: Validation Error.

### Integration Checklist
Before writing code, verify:
1. Which module/service owns this API domain in the codebase?
2. What HTTP client and error-handling patterns does the project use?
3. Are there existing types/interfaces to extend?

Then implement the integration, export it, and confirm it compiles/passes lint.

Any key-value pairs passed in the metadata field can be referenced as dynamic variables in the custom metrics that run on this call.

Headers

X-API-Key · string · required

API key required to authenticate requests.

Body

application/json

Call Request Using Bluejay Agent ID

Call Request Using External Agent ID

agent_id · string · required

ID of the agent associated with the call

start_time_utc · string<date-time> · required

When the call started, in UTC ([python] log: datetime.now(timezone.utc) at call start)

participants · Participant · object[] · required

List of call participants

Required array length: 2 elements

Show child attributes

recording_url · string<uri> | null

URL to the recording file (supported formats: .mp3, .mp4, .wav, .m4a)

Required string length: 1 - 2083

recording_base64 · string | null

Base64 encoded recording file in data URL format (supported formats: .mp3, .mp4, .wav, .m4a)

call_direction · enum<string> | null · default: INBOUND

Direction of the call from the Agent's perspective (INBOUND or OUTBOUND)

Available options: INBOUND, OUTBOUND

interface · enum<string> | null · default: PHONE

Interface used for the call

Available options: PHONE, WEB

transcript_url · string<uri> | null

URL to the transcript file (supported formats: .json)

Required string length: 1 - 2083

transcript · TranscriptEntryRequest · object[] | null

Transcript of the call in text format

Show child attributes

tool_calls · ToolCall · object[] | null

List of tool calls made during the call

Show child attributes

events · Event · object[] | null

List of events that occurred during the call

Show child attributes

tags · string[] | null

List of tags associated with the call

trace_id · string | null

Trace ID of the call

created_at · string<date-time> | null

When this request was created (defaults to now in UTC)

metadata · Metadata · object

Additional metadata associated with the evaluation

webhook_tag · string | null

Tag of the webhook to send the evaluation to

custom_metrics · string<uuid>[] | null

List of additional custom metrics to be used in the call

custom_metric_tags · string[] | null

Specify one or more tags to include all custom metrics associated with those tags. This allows you to run groups of related custom metrics without listing each individually.

label · string | null

Label of the agent to use for the evaluation (determines which prompt and knowledge base to use and defaults to the production label if not provided)

enable_phrase_latencies · boolean | null

Whether to enable phrase-level latency tracking for this call

conversation_ended_by · enum<string> | null

Which participant role ended the conversation: 'USER' or 'AGENT'

Available options: AGENT, USER

multichannel · boolean | null · default: false

Whether the participants of the call are on different audio channels

Response

200

application/json

Successful Response

status · string · required

call_id · string · required

message · string · required