Coolhand LLM Request Logging API (v2)

Creates a new LLM request log entry for monitoring and analysis purposes. See the Understanding an LLM Request Log guide for a full description of the fields Coolhand captures, which fields each provider exposes, and common gotchas. id is the log's hashid (a string), not the internal integer primary key — every log identifier returned by this API is a hashid, matching the GET endpoints below; consumers that previously parsed id as an integer need to treat it as an opaque string instead.

Returns this client's LLM request logs as a bare JSON array (unlike GET /api/v2/llm_request_log_feedbacks's { feedback:, pagination: } envelope - coolhand-cli's fetchLastSync relies on this response being Array.isArray, so the body shape is intentionally not wrapped). Pagination totals are exposed via X-Page/X-Per-Page response headers, plus X-Total-Count/X-Total-Pages when include_total=true is passed (omitted by default to skip the COUNT(*) query - analyze-claude-sessions's frequent per=1 dedup-cutoff poll never needs a total). Supports Ransack search via q[...] params (e.g. q[source_api_in][]=claude_code), sorting via q[s] (e.g. created_at desc for newest first; when no q[s] is supplied, the endpoint applies a stable default sort of id desc for newest first so pagination is deterministic - pass q[s] explicitly to override), and pagination via page/per (per_page also accepted; default 25, max 100). Requires the private API key - the public key is write-only and cannot read. id is always the log's hashid (not the internal integer PK) - the sort itself still orders by the real primary key internally, but a cutoff/cursor derived from results should key off created_at, not id, since hashids don't sort lexically in creation order. analyze-claude-sessions uses this to derive its dedup cutoff by requesting the newest claude_code/claude_cowork record (per=1), instead of a bespoke endpoint. Also supports named filters (applied on top of any q[...] filtering, not in place of it): template_id/workload_id (hashid), system_prompt_contains/user_prompt_contains (case-insensitive substring), model, source_api, source_api_result, project_path (exact match against metadata.project_path), unmatched_only, days_back (unset by default - unlike search_logs's MCP tool, index has always returned unrestricted results and does not implicitly apply a 30-day window), and include_prompts (adds system_prompt/user_prompt, truncated to 500 chars, to each result).

Fetches full input/output content for a single log by its ID (hashid). Accepts any of this client's private credentials: the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the log it was told to look at. Supports section/max_chars for large logs and search_query for snippet search. Only directly-collected logs are fetchable here (matching index's .client_logs restriction) - internally generated records (evals, synthetic logs) 404 even when the hashid is known.

Creates a new feedback entry for LLM request monitoring and improvement purposes.

Field Guide - All fields are optional, but here's how to get the best results:

Matching Fields

• llm_request_log_id 🎯 Exact Match - Hashid from the Coolhand API response when the original LLM request was logged. Provides exact matching.
• llm_provider_unique_id 🎯 Exact Match - The x-request-id from the LLM API response (e.g., "req_xxxxxxx")
• original_output 🔍 Fuzzy Match - The original LLM response text. Provides fuzzy matching but isn't 100% reliable.
• client_unique_id 🔗 Your Internal Matcher - Connect to an identifier from your system for internal matching
• workload_hashid 📦 Workload Association - Optional. Associate feedback with a specific workload to increase fuzzy match success rate

Quality Data

• revised_output ⭐ Best Signal - End user revision of the LLM response. The highest value data for improving quality scores.
• explanation 💬 Medium Signal - End user explanation of why the response was good or bad. Valuable qualitative data.
• like 👍 Low Signal - Boolean like/dislike. Lower quality signal but easy for users to provide.
• creator_unique_id 👤 User Tracking - Unique ID to match feedback to the end user who created it
• collector 🏷️ Metadata - Optional name/stamp identifying the collection method (e.g., "coolhand-node-0.1.0")

Returns this client's feedback entries. Supports Ransack search via q[...] params (e.g. q[sentiment_eq]=2 - sentiment is stored as the integer codes 0=dislike/1=neutral/2=like, which is what responses render as the strings 'dislike'/'neutral'/'like' - or q[created_at_gteq]=...), sorting via q[s] (e.g. created_at desc; when no q[s] is supplied, the endpoint applies a stable default sort of id desc so pagination is deterministic), and pagination via page/per (default 25, max 100). Filtering and sorting on original_output/revised_output are not supported. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read. List items omit original_output/revised_output (which can each hold up to 1GB) - fetch a specific item via GET /api/v2/llm_request_log_feedbacks/{id} to get the full text and any feedback_partials. Also sets X-Total-Count/X-Page/X-Per-Page/X-Total-Pages response headers - the same pagination signal every paginated v2 endpoint exposes (see GET /api/v2/llm_request_logs); the body's pagination object is kept for backward compatibility, not the pattern to rely on for new integrations.

Updates an existing feedback entry. Only specific fields can be updated - identity fields are immutable.

Partial Feedback Updates: When partial_id is provided in the request body, the update is applied to that specific FeedbackPartial instead of the parent feedback. This is useful when adding explanations to partial feedback (highlighted text sections). The response will include updated_partial_id to confirm which partial was updated.

Updatable Fields (Parent Feedback):

• like - Boolean like/dislike rating
• explanation - End user explanation
• revised_output - End user revision of the LLM response
• original_output - The original LLM response text
• llm_provider_unique_id - The x-request-id from the LLM API response
• collector - Collection method identifier
• client_unique_id - Your internal request identifier
• workload_hashid - Associate with a workload (or clear by setting to null/empty)

Updatable Fields (FeedbackPartial - when partial_id is provided):

• explanation - End user explanation for this specific partial
• sentiment - Sentiment value (like/dislike/neutral)
• like - Boolean (automatically converted to sentiment)

Immutable Fields (cannot be updated):

• creator_unique_id
• llm_request_log_id

Note: After a successful update, if the feedback is not already matched to a log and has original_output, the fuzzy matching job will be re-triggered to attempt matching.

Fetches a single feedback entry by hashid, including original_output, revised_output, and any feedback_partials - the full record that GET /api/v2/llm_request_log_feedbacks omits from list results. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read.

Returns a list of all inference models tracked by Coolhand, including per-token pricing metadata.

Authentication: Authentication is optional. Unauthenticated callers are rate-limited (see below). Authenticated callers bypass the rate limit entirely. Pass a public API key via the X-API-Key header or as a Bearer token in the Authorization header. Sign up at coolhand.ai to get a free API key. A private key is not accepted here — this endpoint only recognizes public keys (and rejects a private key the same way it would an invalid one).

How pricing data is gathered: Pricing data is collected by the Godfrey agent, an automated system that periodically fetches and parses pricing pages published by model providers. All records undergo human review before being considered authoritative. The sources field on each record links to the original provider pricing page used as the data source.

Cost units: Every _per_token field is a raw USD cost-per-token value. Multiply by 1,000,000 for the "per 1M tokens" pricing commonly published by providers. per_cache_storage_cost_per_hour_per_token is the one exception: it's priced per hour of storage, not per request — an ongoing hourly charge for holding tokens in a cache (as Gemini's explicit context caching bills), not a one-time per-token cost.

Rate limiting: Unauthenticated requests are rate limited to 1 request per minute per IP address. Exceeding the limit returns a 429 Too Many Requests response. Authenticated requests are never rate limited.

Returns every field #index's summary response has, plus the fields #index omits: the internal not_human_reviewed review-status flag, the slug used in this model's public page URL, and the batch and reasoning-output per-token pricing columns (index has never exposed either). Same authentication and rate limiting as #index.

Attaches an existing LlmRequestLogFeedback to an optimization as supporting evidence. Requires a private credential (the client's own private API key, a team member's UserClient private key, or an AdminClientCredential) or a Doorkeeper Bearer token - the public api_key is rejected on this endpoint.

Removes a feedback-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.

Attaches an existing LlmRequestLog to an optimization as supporting evidence. Requires a private credential (see the feedback-links create operation's description) - the public api_key is rejected on this endpoint.

Removes a log-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.

Attaches an existing LlmRequestEval to an optimization as supporting evidence. Requires a private credential (see the feedback-links create operation's description) - the public api_key is rejected on this endpoint.

Removes an eval-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.

Records a change suggestion (a proposed fix/PR) against the authenticated client.

suggestion_type - one of setup, optimization_fix.

status - one of draft, open, merged, closed, awaiting_client_review, incomplete, awaiting_repo_selection.

The suggestion is always scoped to the authenticated client - any client_id in the request body is ignored.

Requires a private credential (the client's own private API key, a team member's UserClient private key, or an AdminClientCredential) or a Doorkeeper Bearer token - the public api_key is rejected on this endpoint.

Uploads a document as a ClientFile, e.g. so a tool like coolhand-cli can attach project files to a client. id is the file's hashid. metadata is an optional free-form object for client-supplied context - see the project_path convention in the Understanding an LLM Request Log guide's Metadata section, which applies here too. Uploads always land as draft: status is not settable through this endpoint, since nothing validates an uploaded file's content - promote a file to published from the admin UI once it's been reviewed. Requires the private API key - the public key cannot upload files. Files are currently proxied through the API and capped at 20MB; larger uploads are not yet supported.

Displays the browser-based token authorization page for the Coolhand CLI.

Flow overview:

1. The CLI starts a local HTTP server and opens the browser to GET /cli/auth?redirect_uri=http://localhost:<port>/cb&state=<random>&scope=<scope>.
2. If the user is unauthenticated, they are redirected to /users/sign_in, which stores the CLI auth URL as the post-login return URL.
3. After authenticating, the user sees this page, selects (or creates) a Coolhand account, checks the confirmation box(es), and clicks Connect.
4. The server redirects to redirect_uri with token(s) in the query string (see POST /cli/auth).

scope parameter:

• public (default) — returns only the public API key (token). Use for log ingest and REST API calls.
• private — returns both the public API key (token) and the private MCP key (private_token). The UI displays an additional red warning banner and a second confirmation checkbox; the CLI Connect button is disabled until both checkboxes are checked. Only request private scope when the CLI needs to call the /mcp endpoint.

Security: redirect_uri must be an http:// URL with host localhost, 127.0.0.1, or ::1. HTTPS and non-localhost hosts are rejected with 400 Bad Request.

Processes the CLI auth form submission and redirects the browser to the CLI's local callback URL with API key(s) in the query string.

Callback query parameters:

At least one of token / private_token is always present — submitting an empty scope[] is rejected with 422 Unprocessable Entity.

Using the private key: Pass private_token as the X-API-Key header when calling POST /mcp. The public token is used everywhere else (SDK configuration, ingest endpoint). Never embed the private key in client-side code or commit it to source control.

New client creation: Pass new_client=1 and new_client_name=<name> instead of client_id to create a new Coolhand account on the fly. The callback will carry the new account's freshly generated keys.