> ## Documentation Index > Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details. # Start a run > Start a run. The statement is accepted, its questions are derived from the eval functions it projects, and the judging happens on the queue: the answer is the queued run, and its progress is read back from the run endpoint. A statement the query policy refuses, one that projects no TraceId, one that projects no eval function, and a row limit past what the plan allows are all refused before anything is judged. Instead of a statement you may send a target and your questions, and the statement is written for you and handed back on the run; sending both is refused. ## OpenAPI ````yaml POST /api/v1/instant-evals openapi: 3.1.0 info: title: LangWatch API version: 1.0.0 description: LangWatch openapi spec servers: - url: https://app.langwatch.ai security: - project_api_key: [] paths: /api/v1/instant-evals: post: tags: - Instant Evals description: >- Start a run. The statement is accepted, its questions are derived from the eval functions it projects, and the judging happens on the queue: the answer is the queued run, and its progress is read back from the run endpoint. A statement the query policy refuses, one that projects no TraceId, one that projects no eval function, and a row limit past what the plan allows are all refused before anything is judged. Instead of a statement you may send a target and your questions, and the statement is written for you and handed back on the run; sending both is refused. operationId: createInstantEvalRun requestBody: required: true content: application/json: schema: type: object properties: sql: type: string minLength: 1 maxLength: 50000 description: >- The LangWatchQL statement to judge. It must project TraceId and at least one eval function column. Send this or target, never both. parameters: type: object additionalProperties: anyOf: - type: string - type: number - type: boolean - type: 'null' description: Values for the parameters the statement declares. target: type: string enum: - traces - threads - llm_spans description: >- What one judged row is, in place of a statement: a trace, a conversation, or one model call. The statement is written for you from this and the questions, and handed back on the run so you can edit it and resubmit. filter: type: string maxLength: 4000 description: >- With target: a trace filter, in the language the trace explorer's search bar speaks, narrowing which rows are judged. start: type: string format: date-time description: >- With target: the oldest instant to judge, as an ISO 8601 timestamp. Defaults to seven days ago. end: type: string format: date-time description: 'With target: the newest instant to judge. Defaults to now.' questions: type: array items: type: object properties: id: type: string minLength: 1 maxLength: 64 description: >- What to call this question. It becomes the statement's output column and the name every judgement is filed under. Defaults to q1, q2 and so on. kind: type: string enum: - boolean - score - category default: boolean description: >- What kind of answer you want: a yes or no, a rating on a scale, or one of a list of options. instructions: type: string minLength: 1 maxLength: 2000 description: >- The question, in your own words, as you would write it for a human reader. criteria: type: array items: type: string minLength: 1 maxLength: 500 minItems: 2 maxItems: 2 description: >- For a yes or no question: what counts as yes, then what counts as no. Cannot be combined with a threshold. threshold: type: number minimum: 0 maximum: 1 description: >- For a yes or no question: the probability at or above which the answer counts as yes. Without one the column carries the probability itself and a run draws the line at an even chance. range: type: object properties: min: type: integer description: The lowest level of the scale. max: type: integer description: The highest level of the scale. required: - min - max description: 'For a rating: the two ends of the scale.' options: type: array items: type: object properties: name: type: string minLength: 1 maxLength: 100 description: >- What the column holds when this option is the answer. description: type: string minLength: 1 maxLength: 500 description: What this option means, in your own words. required: - name - description minItems: 2 maxItems: 255 description: 'For a choice: the options to pick between.' required: - instructions description: >- With target: what to ask of each row. One classification asks them all, which is why a three-question run costs about what a one-question run does. name: type: string minLength: 1 maxLength: 200 description: What to call the run. Yours to choose. limit: type: integer exclusiveMinimum: 0 maximum: 100000 description: >- Rows the run may judge. Ten thousand by default on every plan, up to one hundred thousand on a plan that lifts the cap. responses: '202': description: Success content: application/json: schema: type: object properties: id: type: string description: The run id. name: type: - string - 'null' description: What the run was called, if anything. sql: type: string description: The statement, exactly as submitted. parameters: type: object additionalProperties: anyOf: - type: string - type: number - type: boolean - type: 'null' description: The values the statement's parameters were filled with. questions: type: array items: type: object properties: id: type: string description: >- The statement's own output column, which is the name this question is addressed by everywhere else. function: type: string description: The eval function that asked it. kind: type: string enum: - boolean - score - category description: What kind of answer the question takes. reads: type: string description: >- Which part of the verdict the statement's column carries. threshold: type: - number - 'null' description: >- Where a boolean question's probability becomes a pass. Null for a question that is not a boolean. required: - id - function - kind - reads - threshold description: >- One entry per eval function the statement projects, derived from it when the run was accepted. limit: type: integer description: Rows this run may judge. status: type: string enum: - queued - planning - running - finished - failed - cancelled description: Where the run is in its life. total: type: - integer - 'null' description: >- Rows the run found, bounded by its limit. Null until it has looked. progress: type: integer description: Rows judged so far. matched: type: - integer - 'null' description: >- Judgements that matched, across this run's boolean questions. Null when the run asked none: a score or a category question has no match to count. matchedByQuestion: type: object additionalProperties: type: number description: >- Per question: matches for a boolean question, judged rows for a score or a category one. failed: type: integer description: Rows the judge could not answer. skipped: type: integer description: Rows the judge declined to answer. tokens: type: integer description: Input tokens the judge billed for. priceUsd: type: number description: What the judging costs you, in United States dollars. error: type: - string - 'null' description: The code of the failure that ended the run, when one did. createdAt: type: string description: When the run was accepted. updatedAt: type: string description: When the run was last written to. startedAt: type: - string - 'null' description: When the run began reading rows. finishedAt: type: - string - 'null' description: When the run ended. required: - id - name - sql - parameters - questions - limit - status - total - progress - matched - matchedByQuestion - failed - skipped - tokens - priceUsd - error - createdAt - updatedAt - startedAt - finishedAt security: - project_api_key: [] components: securitySchemes: project_api_key: type: apiKey in: header name: X-Auth-Token description: >- Project API key for sending traces and accessing project-scoped resources. Format: sk-lw-... (no underscore). Obtain one by creating a project via the Admin API or the LangWatch UI. ````