Guides

Asynchronous Batch API

The Batch API allows you to execute millions of requests asynchronously over a 24-hour turnaround window at a 50% discount. It is ideal for bulk categorization, dataset generation, model evaluations, and backfilling.

Overview

Instead of managing rate limits, worker threads, and connection pools, you upload a single JSON Lines (.jsonl) file containing your batch of requests. Cortiqa processes the batch against off-peak LPU capacity and saves the completed outputs to a downloadable file.

1. Prepare a JSONL File

Each line must be a valid JSON object specifying a unique custom_id and request body:

{"custom_id": "req-001", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-oss-120b", "messages": [{"role": "user", "content": "Classify this tweet: Loved the flight!"}]}}
{"custom_id": "req-002", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-oss-120b", "messages": [{"role": "user", "content": "Classify this tweet: Luggage was lost."}]}}

2. Submit the Batch Job

curl https://api.cortiqa.co/api/v1/batches \
  -H "Authorization: Bearer sk-cortiqa-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-xyz789",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

3. Check Job Status

Poll job status via GET /api/v1/batches/{batch_id}:

{
  "id": "batch_abc123",
  "object": "batch",
  "status": "in_progress",
  "request_counts": {
    "total": 10000,
    "completed": 8420,
    "failed": 0
  }
}

4. Download Results

Once status reaches completed, retrieve the output file ID to stream the corresponding results JSONL.