Ask AI

This document explains the Ask AI node, which lets you interact with AI models to process text and generate responses.

​

Node Inputs

​

Required Fields

• Prompt: Main instruction or question for the AI. It should be clear and detailed to guide the AI to give you the best possible answer.

​

Optional Fields

• Context: Additional information to help AI effectively respond to the query in the prompt. Context can be referenced in the prompt.

​

More Options

• Prompt Template: Structure for combining prompt and context

  • Default Template:

    Given the context:
    <context>
    {context}
    </context>
    Respond to the user's query: {prompt}
    

  • Visible under “More Options” in the node
  • Only modify if default structure doesn’t meet your needs
  • Contains placeholders {prompt} and {context}
  • Stays constant across runs unless modified

• Choose AI Model: Select the AI model you want to use

• Temperature: Controls response creativity (0-1). Default temperature is ‘1’

  • 0: More focused, consistent
  • 1: More creative, varied

• Maximum Tokens: Limits response length

• Cache Response: Save responses for reuse (only works when inputs, context, and prompts remain constant)

• Thinking Tokens (for Claude 3.7 Sonnet Thinking only): Set a budget for extended thinking mode

  • Minimum: 1024 tokens
  • Must be less than Maximum Tokens
  • Larger budgets improve reasoning quality for complex tasks

​

Show As Input

The node allows you to configure certain parameters as dynamic inputs. You can enable these in the “Configure Inputs” section:

• prompt: String

  • The main instruction/prompt for the AI
  • Example: “Summarize this article”

• context: String

  • Additional context information
  • Example: “This article is about artificial intelligence”

• model_preference: String

  • Name of the AI model to use
  • Accepted values: “Claude 3.7 Sonnet”, “Claude 3.7 Sonnet - Thinking”, “Claude 3.5 Haiku”, “GPT-4o”, “GPT-4o Mini”, etc.

• prompt_template: String

  • Template structure with and placeholders
  • Example: “Given the context:{context}, Answer: {prompt}”

• Use Function?: Boolean

  • true/false to enable/disable function calling
  • Only works with OpenAI models

• Cache Response: Boolean

  • true/false to enable/disable response caching
  • Helps reduce API calls for identical inputs

• Temperature: Number

  • Value between 0 and 1

• Maximum Tokens: Number

  • Positive integer for maximum response length
  • Example: 2000

• Thinking Tokens: Number (for Claude 3.7 Sonnet Thinking only)

  • Number of tokens allocated for extended thinking
  • Minimum: 1024 tokens
  • Example: 4000

When enabled as inputs, these parameters can be dynamically set by previous nodes in your workflow. If not enabled, the values set in the node configuration will be used.

​

Key Concepts

​

Prompt vs. Prompt Template

1. Prompt:

   • Actual question or instruction for the AI
   • Required input
   • Examples: “Summarize this article”, “Translate to Spanish”

2. Prompt Template:

   • Defines structure for combining prompt and context
   • Optional parameter
   • Uses placeholders {prompt} and {context}
   • Default template works for most cases
   • Modify only for specific formatting needs

​

Claude 3.7 Sonnet vs. Claude 3.7 Sonnet Thinking

Claude 3.7 Sonnet is available in two distinct modes that serve different use cases:

1. Claude 3.7 Sonnet (Standard Mode):

   • Provides direct responses without extended internal reasoning
   • Quick response time suitable for most everyday tasks
   • Excellent for creative content, analysis, and general queries
   • More efficient for straightforward tasks where deep reasoning isn’t required

2. Claude 3.7 Sonnet Thinking (Extended Thinking Mode):

   • Performs additional internal reasoning before delivering the final answer
   • Takes longer to respond but provides higher quality answers for complex problems
   • Especially valuable for tasks requiring:
     • Complex problem-solving with multi-step logic
     • Math calculations and derivations
     • Detailed code writing and debugging
     • Analytical tasks that benefit from thorough consideration
   • While the model performs more extensive thinking internally, only the final response is shown in Gumloop (the thinking process itself is not visible)
   • The model determines how much of the thinking tokens to use based on the task complexity

You can learn more about Claude 3.7 Sonnet here.

​

Thinking Tokens vs. Maximum Tokens

1. Thinking Tokens (Claude 3.7 Sonnet Thinking only):

   • Budget for the model’s internal reasoning process
   • Must be less than Maximum Tokens
   • Minimum: 1024 tokens
   • Recommended: 4,000-16,000 for complex tasks
   • Larger budgets improve reasoning but increase cost and response time

2. Maximum Tokens:

   • Total limit for the entire response (includes thinking + final answer)
   • Must be greater than Thinking Tokens
   • Sets the upper bound for response length

​

Node Output

• Response: AI’s generated answer or output

​

Node Functionality

The Ask AI node connects to various AI models to:

• Generate text responses
• Process/analyze content
• Answer questions
• Summarize information
• Convert data formats
• Perform step-by-step reasoning (with thinking-enabled models)

​

Available AI Models

• Claude 3.7 Sonnet
• Claude 3.7 Sonnet Thinking (extended reasoning capabilities)
• Claude 3.5 Haiku
• OpenAI o1
• OpenAI o3 mini
• GPT-4o
• GPT-4o Mini
• DeepSeek V3
• DeepSeek R1
• Perplexity Sonar Reasoning
• Perplexity Sonar Reasoning Pro
• Gemini 2.0 Flash
• Grok 2
• Azure OpenAI
• And more

Note: Auto-Select uses a third-party model routing service and automatically chooses the appropriate model for cost, performance, and availability. Not ideal if consistent model behavior is needed.

​

AI Model Selection Guide

When choosing an AI model for your task, consider these key factors:

Additional selection factors:

• Task complexity and required accuracy
• Response time requirements
• Cost considerations
• Consistency needs across runs
• Specialized knowledge requirements

For more detailed information on AI models with advanced reasoning capabilities, you can refer to:

• Anthropic Models Overview
• Anthropic Extended Thinking Documentation
• OpenAI Reasoning Guide
• OpenAI o1 Models

​

Common Use Cases

1. Content Creation:

Prompt: "Write a blog post about {topic}"
Context: Key points and style guide

2. Data Analysis:

Prompt: "Analyze these sales figures"
Context: Monthly sales data

3. Customer Support:

Prompt: "Answer this customer question professionally"
Context: Customer query and company policies

4. Step-by-step Problem Solving (with Thinking):

Prompt: "Solve this complex pricing optimization problem"
Context: [business constraints and variables]
Thinking Tokens: 8000

​

Loop Mode Pattern

1. Batch Processing:

   Input: List of articles
   Prompt: "Analyze and find key patterns in this article:"
   Context: {current_article}
   Result: Analysis for each article
   

​

Understanding Cache Response

The Cache Response option can significantly speed up your workflows when using the same inputs repeatedly:

• When enabled, identical inputs will retrieve previous results instead of making new API calls
• For caching to work, all of these must be identical:
  • Prompt text
  • Context text
  • Model selection
  • Temperature setting
  • Maximum tokens setting
  • Thinking tokens (if applicable)
• Even small changes to any input will trigger a new API call
• Perfect for testing workflows or handling repeated queries

​

Important Considerations

1. Credit Costs:

   • Expert models (OpenAI o1, Claude 3.7 Thinking): 30 credits
   • Advanced models (GPT-4o & Claude 3.7): 20 credits
   • Standard models: 2 credits per run
   • With API key: 1 credit (configure in credentials page)

2. Function Calling:

   • ‘Use Function’ option for structured output
   • Only available for OpenAI models
   • More details: OpenAI Function Calling Documentation

3. Model Selection:

   • Consider task complexity when selecting models
   • For reasoning-heavy tasks, consider thinking-enabled or reasoning models

In summary, the Ask AI node is your interface to AI models, helping you automate text processing and generation tasks with customizable control over the output style and format. With models offering extended thinking capabilities, you can tackle complex problems that require step-by-step analysis.