> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gumloop.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run evaluation on sessions

> Grades up to 200 existing sessions with this evaluation. Grading is asynchronous: each accepted session gets a result with `status: queued`; poll it with `GET /evaluations/{evaluation_id}/results/{result_id}` until it is `completed` or `failed`.

Sessions are skipped, not rejected, when they are not completed sessions of an agent the evaluation covers (`ineligible`) or already have a queued or running result for this evaluation (`in_flight`, with the existing `result_id`). The caller is charged one credit per queued session. Set `dry_run: true` to see the cost and skips without queuing anything.




## OpenAPI

````yaml post /evaluations/{evaluation_id}/run
openapi: 3.0.0
info:
  title: Public API
  version: 1.0.0
servers:
  - url: https://api.gumloop.com/api/v1
security: []
paths:
  /evaluations/{evaluation_id}/run:
    post:
      tags:
        - Organization Evaluations
      summary: Run evaluation on sessions
      description: >
        Grades up to 200 existing sessions with this evaluation. Grading is
        asynchronous: each accepted session gets a result with `status: queued`;
        poll it with `GET /evaluations/{evaluation_id}/results/{result_id}`
        until it is `completed` or `failed`.


        Sessions are skipped, not rejected, when they are not completed sessions
        of an agent the evaluation covers (`ineligible`) or already have a
        queued or running result for this evaluation (`in_flight`, with the
        existing `result_id`). The caller is charged one credit per queued
        session. Set `dry_run: true` to see the cost and skips without queuing
        anything.
      operationId: runOrganizationEvaluation
      parameters:
        - in: path
          name: evaluation_id
          required: true
          schema:
            type: string
          description: ID of the organization evaluation.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - session_ids
              properties:
                session_ids:
                  type: array
                  minItems: 1
                  maxItems: 200
                  uniqueItems: true
                  items:
                    type: string
                  description: Sessions to grade. Duplicates are rejected.
                dry_run:
                  type: boolean
                  default: false
                  description: Report cost and skipped sessions without queuing.
      responses:
        '200':
          description: >-
            Dry run — nothing was queued. `results` lists the sessions that
            would run with `status: planned` and no `id`.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationRunResponse'
        '202':
          description: Sessions queued.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationRunResponse'
        '400':
          description: >-
            Invalid request, evaluation disabled or empty
            (`organization_evaluation_not_runnable`), or no session belongs to a
            covered agent (`organization_evaluation_no_eligible_chats`).
        '401':
          description: Unauthorized — missing or invalid credentials.
        '402':
          description: >-
            Not enough credits for the sessions that would run
            (`insufficient_credits`).
        '403':
          description: >-
            Forbidden — not an organization admin, or the organization is not on
            the Enterprise plan.
        '404':
          description: Evaluation not found.
        '409':
          description: >-
            Every eligible session is already queued or running for this
            evaluation (`organization_evaluation_run_in_flight`).
      security:
        - bearerAuth: []
      x-codeSamples:
        - lang: bash
          label: cURL
          source: >
            curl -X POST
            'https://api.gumloop.com/api/v1/evaluations/EVALUATION_ID/run' \
              -H 'Authorization: Bearer YOUR_ACCESS_TOKEN' \
              -H 'Content-Type: application/json' \
              -d '{"session_ids": ["SESSION_ID_1", "SESSION_ID_2"]}'
        - lang: python
          label: Python
          source: >
            from gumloop import Gumloop


            client = Gumloop(access_token="YOUR_ACCESS_TOKEN")


            queued = client.evaluations.run("EVALUATION_ID", ["SESSION_ID_1",
            "SESSION_ID_2"])

            for result in queued.results:
                print(result.session_id, result.id)
            for skipped in queued.skipped:
                print(skipped.session_id, skipped.reason)
components:
  schemas:
    EvaluationRunResponse:
      type: object
      properties:
        dry_run:
          type: boolean
        credit_cost:
          type: integer
          description: >-
            Credits charged (or, on a dry run, that would be charged) — one per
            queued session.
        results:
          type: array
          description: Sessions accepted, in request order.
          items:
            type: object
            properties:
              id:
                type: string
                nullable: true
                description: Result ID to poll. Null on a dry run.
              session_id:
                type: string
              status:
                type: string
                enum:
                  - queued
                  - planned
        skipped:
          type: array
          items:
            type: object
            properties:
              session_id:
                type: string
              reason:
                type: string
                enum:
                  - ineligible
                  - in_flight
                description: >-
                  `ineligible`: not a completed session of a covered agent.
                  `in_flight`: already queued or running for this evaluation.
              result_id:
                type: string
                nullable: true
                description: The in-flight result, when `reason` is `in_flight`.
      example:
        dry_run: false
        credit_cost: 2
        results:
          - id: 9f2d6a41-3b7c-4e0a-8f11-2c5d7e9b0a3f
            session_id: int_xyz789
            status: queued
          - id: 0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9
            session_id: int_abc123
            status: queued
        skipped:
          - session_id: int_old001
            reason: ineligible
            result_id: null
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        A personal API key or an [OAuth 2.0](/api-reference/oauth) access token.
        Personal API keys also require the `x-auth-key` header with your user
        ID.

````