> ## Documentation Index > Fetch the complete documentation index at: https://docs.gumloop.com/llms.txt > Use this file to discover all available pages before exploring further. # Create evaluation > Creates an organization evaluation. A new evaluation has no targets, so it cannot start enabled: set targets with `PUT /evaluations/{evaluation_id}/targets`, then enable it with `PATCH /evaluations/{evaluation_id}`. Rubric values are validated strictly: an unknown `frequency`, criterion `priority`, `type`, data point `data_type`, or session type, a criterion without `name` and `prompt`, or a duplicate tag name returns `400 invalid_request` with the offending paths in `error.details.fields`. ## OpenAPI ````yaml post /evaluations openapi: 3.0.0 info: title: Public API version: 1.0.0 servers: - url: https://api.gumloop.com/api/v1 security: [] paths: /evaluations: post: tags: - Organization Evaluations summary: Create evaluation description: > Creates an organization evaluation. A new evaluation has no targets, so it cannot start enabled: set targets with `PUT /evaluations/{evaluation_id}/targets`, then enable it with `PATCH /evaluations/{evaluation_id}`. Rubric values are validated strictly: an unknown `frequency`, criterion `priority`, `type`, data point `data_type`, or session type, a criterion without `name` and `prompt`, or a duplicate tag name returns `400 invalid_request` with the offending paths in `error.details.fields`. operationId: createOrganizationEvaluation requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/EvaluationCreateRequest' responses: '201': description: The created evaluation. content: application/json: schema: $ref: '#/components/schemas/EvaluationResponse' '400': description: >- Invalid request, invalid rubric value (`invalid_request` with `details.fields`), evaluation limit reached, or `enabled` set without targets (`organization_evaluation_no_targets`). '401': description: Unauthorized — missing or invalid credentials. '403': description: >- Forbidden — not an organization admin, or the organization is not on the Enterprise plan. '409': description: >- An active evaluation with this name already exists (`organization_evaluation_name_taken`). security: - bearerAuth: [] x-codeSamples: - lang: bash label: cURL source: | curl -X POST 'https://api.gumloop.com/api/v1/evaluations' \ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN' \ -H 'Content-Type: application/json' \ -d '{ "organization_id": "ORG_ID", "name": "Support tone", "config": { "criteria": [ {"name": "Greets the customer", "prompt": "Did the agent greet the customer by name?", "priority": "needs_review"} ] } }' - lang: python label: Python source: | from gumloop import Gumloop client = Gumloop(access_token="YOUR_ACCESS_TOKEN") created = client.evaluations.create( organization_id="ORG_ID", name="Support tone", config={ "criteria": [ { "name": "Greets the customer", "prompt": "Did the agent greet the customer by name?", "priority": "needs_review", } ] }, ) print(created.evaluation.id) components: schemas: EvaluationCreateRequest: type: object required: - organization_id - name additionalProperties: false properties: scope: type: string enum: - organization default: organization organization_id: type: string name: type: string minLength: 1 maxLength: 256 description: Unique among the organization's active evaluations. description: type: string maxLength: 4000 nullable: true enabled: type: boolean description: >- Must be omitted or false on create; a new evaluation has no targets yet. config: $ref: '#/components/schemas/EvaluationRubric' EvaluationResponse: type: object properties: evaluation: $ref: '#/components/schemas/Evaluation' EvaluationRubric: type: object description: >- What and how the evaluation grades. Entries in `criteria`, `tags`, and `data_points` accept additional fields as the product evolves. properties: model_name: type: string description: Grading model. `auto` picks the recommended model. frequency: type: string enum: - debounced - per_turn - manual description: >- When to grade new sessions automatically. `manual` only grades via `POST /evaluations/{evaluation_id}/run`. language: type: string description: Language for summaries and rationales, or `auto`. include_auto_tags: type: boolean session_types: type: array items: type: string description: >- Session types to grade. See `session_types` in `GET /evaluation-options`. criteria: type: array maxItems: 30 items: type: object required: - name - prompt properties: id: type: string description: >- Assigned by the server when omitted; keep it to update a criterion in place. name: type: string prompt: type: string description: Plain-language question the grader answers about the session. type: type: string enum: - prohibited_action - prohibited_words - voice_tone - other priority: type: string enum: - needs_review - needs_attention description: The grade a failing session receives. enabled: type: boolean default: true tags: type: array maxItems: 50 items: type: object required: - name properties: name: type: string description: >- Stored upper-snake-cased (`refund request` becomes `REFUND_REQUEST`). description: type: string data_points: type: array maxItems: 40 items: type: object required: - name properties: id: type: string name: type: string data_type: type: string enum: - string - boolean - integer - number description: type: string sentiment: type: object additionalProperties: true notifications: type: object additionalProperties: true Evaluation: type: object properties: id: type: string scope: type: string enum: - organization agent_id: type: string nullable: true description: Reserved for agent-scoped evaluations; null today. organization_id: type: string name: type: string description: type: string nullable: true enabled: type: boolean description: >- Whether new sessions of covered agents are graded and manual runs are allowed. targets: type: array items: $ref: '#/components/schemas/EvaluationTarget' covered_agent_count: type: integer description: Agents the targets currently expand to. config: $ref: '#/components/schemas/EvaluationRubric' run_summary: type: object additionalProperties: true description: >- Rollup of the latest result per graded session — `evaluated_count`, `graded_count`, `pass_count`, `needs_review_count`, `needs_attention_count`, `failed_count`, `success_rate`, `last_run_at`. creator: type: object nullable: true properties: id: type: string first_name: type: string nullable: true last_name: type: string nullable: true email: type: string nullable: true profile_picture: type: string nullable: true created_at: type: string format: date-time updated_at: type: string format: date-time example: id: 1c1b3c8e-7d61-4c1e-9c66-4a8d2f1e0b7a scope: organization agent_id: null organization_id: org_4f8c92ab name: Support tone description: Customer-facing agents stay polite and on topic. enabled: true targets: - type: team id: team_4f8c92ab covered_agent_count: 12 config: model_name: auto frequency: debounced language: auto include_auto_tags: true session_types: - chat - slack criteria: - id: crit_1 name: Greets the customer prompt: Did the agent greet the customer by name? type: voice_tone priority: needs_review enabled: true tags: [] data_points: [] run_summary: evaluated_count: 48 graded_count: 47 pass_count: 41 needs_review_count: 4 needs_attention_count: 2 failed_count: 1 success_rate: 0.872 last_run_at: '2026-09-03T21:40:00+00:00' creator: id: user_1a2b3c first_name: Ada last_name: Lovelace email: ada@acme.com profile_picture: null created_at: '2026-09-01T12:00:00+00:00' updated_at: '2026-09-03T21:40:00+00:00' EvaluationTarget: type: object required: - type properties: type: type: string enum: - organization - team - user - agent description: >- What the target expands to. `user` covers a member's personal agents. id: type: string description: >- Team, user, or agent ID. Omitted for `organization`; responses return the organization ID. securitySchemes: bearerAuth: type: http scheme: bearer description: >- A personal API key or an [OAuth 2.0](/api-reference/oauth) access token. Personal API keys also require the `x-auth-key` header with your user ID. ````