Create Custom Metric

Create a new custom metric.

POST /v1/create-custom-metric

Create Custom Metric

cURL

curl --request POST \
  --url https://api.getbluejay.ai/v1/create-custom-metric \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: <x-api-key>' \
  --data '
{
  "name": "<string>",
  "description": "<string>",
  "response_type": "pass_fail",
  "agent_id": 123,
  "agent_ids": [
    123
  ],
  "min_value": 123,
  "max_value": 123,
  "category": "<string>",
  "tags": [
    "<string>"
  ],
  "scoring_guidance": "<string>",
  "eval_route": "AUTO",
  "model": "<string>",
  "audio_model": "<string>",
  "temperature": 123,
  "enum_options": [
    "<string>"
  ],
  "allow_not_applicable": false
}
'

200

{
  "id": "<string>",
  "name": "<string>",
  "description": "<string>",
  "response_type": "pass_fail",
  "created_at": "2023-11-07T05:31:56Z",
  "agent_ids": [
    123
  ],
  "min_value": 123,
  "max_value": 123,
  "category": "<string>",
  "tags": [
    "<string>"
  ],
  "scoring_guidance": "<string>",
  "eval_route": "AUDIO",
  "model": "<string>",
  "audio_model": "<string>",
  "temperature": 123,
  "enum_options": [
    "<string>"
  ],
  "allow_not_applicable": false
}

Integration Prompt for AI Agents

# Bluejay — Testing & Monitoring Platform for Conversational AI Agents

You are a senior backend engineer integrating the Bluejay API. Think step-by-step: first understand the endpoint, then plan the integration, then implement with minimal changes.

## Create Custom Metric — POST /v1/create-custom-metric

> **What this endpoint does:** Create a new custom metric.

**Endpoint:** POST `https://api.getbluejay.ai/v1/create-custom-metric`
**Auth:** `X-API-Key` header
**Content-Type:** application/json

### Required Parameters
| Name | Type | Description |
|------|------|-------------|
| X-API-Key | string | API key required to authenticate requests. |
| name | string | Name of the custom metric |
| description | string | Description of what this metric measures |
| response_type | string (enum: pass_fail, yes_no, qualitative, quantitative, json, enum) | Enum representing the possible response types for custom metrics. |

Review the full parameter list at https://docs.getbluejay.ai/api-reference/endpoint/create-custom-metric and include any optional parameters (e.g., `agent_id`, `agent_ids`, `min_value`, `max_value`, `category`, `tags`) that serve your integration's use case and align with Bluejay's testing and monitoring capabilities.

### Request Body (required fields)
```json
{
  "name": "example_name",
  "description": "string",
  "response_type": "pass_fail"
}
```

Refer to the full schema at https://docs.getbluejay.ai/api-reference/endpoint/create-custom-metric. Include optional fields that serve the goal of setting up for testing and monitoring on Bluejay.

### Example
**POST with body:**
```python
import requests

def create_custom_metric(payload: dict, api_key: str) -> dict:
    url = "https://api.getbluejay.ai/v1/create-custom-metric"
    headers = {"X-API-Key": api_key}
    response = requests.post(url, headers=headers, json=payload)
    response.raise_for_status()
    return response.json()
```

### Constraints
- Minimal changes — only add/change files needed for this integration.
- Match existing codebase patterns (naming, file structure, error handling).
- Include error handling for 422: Validation Error.

### Integration Checklist
Before writing code, verify:
1. Which module/service owns this API domain in the codebase?
2. What HTTP client and error-handling patterns does the project use?
3. Are there existing types/interfaces to extend?

Then implement the integration, export it, and confirm it compiles/passes lint.

This endpoint allows you to create a custom metric for your agent. Provide the necessary details to add a metric to your agent.

Headers

X-API-Key · string · required

API key required to authenticate requests.

Body

application/json

Request model for creating a custom metric

name · string · required

Name of the custom metric

description · string · required

Description of what this metric measures

response_type · enum<string> · required

Type of response expected

Available options: pass_fail, yes_no, qualitative, quantitative, json, enum

agent_id · integer | null · deprecated

ID of the agent this metric belongs to (deprecated, use agent_ids instead)

agent_ids · integer[] | null

List of agent IDs to associate this metric with

min_value · number | null

Minimum value for quantitative metrics

max_value · number | null

Maximum value for quantitative metrics

category · string | null

Category for organizing metrics

tags · string[] | null

Tags for categorizing the metric

scoring_guidance · string | null

Guidance on how to score this metric

eval_route · enum<string> | null · default: AUTO

Route for evaluating the metric

Available options: AUDIO, TEXT, AUTO

model · string | null

Model to use for text evaluation of the metric

audio_model · string | null

Model to use for audio evaluation of the metric

temperature · number | null

Temperature for evaluating the metric

enum_options · string[] | null

Options for enum metrics

allow_not_applicable · boolean · default: false

Whether this metric allows 'not applicable' responses.

Response

200

application/json

Successful Response

Response model for custom metric operations

id · string · required

Unique identifier for the custom metric

name · string · required

Name of the custom metric

description · string · required

Description of what this metric measures

response_type · enum<string> · required

Type of response expected

Available options: pass_fail, yes_no, qualitative, quantitative, json, enum

created_at · string<date-time> · required

When this metric was created

agent_ids · integer[]

List of agent IDs this metric belongs to

min_value · number | null

Minimum value for quantitative metrics

max_value · number | null

Maximum value for quantitative metrics

category · string | null

Category for organizing metrics

tags · string[] | null

Tags for categorizing the metric

scoring_guidance · string | null

Guidance on how to score this metric

eval_route · enum<string> | null

Route for evaluating the metric

Available options: AUDIO, TEXT, AUTO

model · string | null

Model to use for text evaluation of the metric

audio_model · string | null

Model to use for audio evaluation of the metric

temperature · number | null

Temperature for evaluating the metric

enum_options · string[] | null

Options for enum metrics

allow_not_applicable · boolean · default: false

Whether this metric allows 'not applicable' responses.