Python SDKSlack

Use image inputs

Overview

The Prompt Playground supports image inputs for working with multi-modal models, like GPT-4o. This feature allows you to iterate on tasks such as image caption generation, visual QA, and more.

You can pass images into your prompts by uploading image URLs as variables. These are used as inputs in your LLM calls—perfect for experimenting with visual content alongside text.

Step-by-Step: Adding Image Inputs

1. Open the Prompt Playground

Navigate to Prompt Playground from the sidebar. Select a multi-modal model (e.g., gpt-4o) from the dropdown.

2. Add an Image Input Variable

Click the image icon (🖼️) in the variables section to define an image input.

  • Give your image variable a descriptive name (e.g., white_lotus_hotel)

  • Click "Add Variable"

Select the "image" icon to add an image input variable.

3. Upload an Image URL

Paste a supported image URL into the input field (JPG, PNG, or base64 formats are accepted).

  • Example: https://example.com/images/hotel.jpg

  • Click "Submit" once the image is loaded.

Upload an image with a supported url format. Accepts web URLs (jpg, png, etc.) or base64 data URLs.

4. View Results Side-by-Side

Run your prompt. The playground will display:

  • The image input

  • The LLM-generated output

Example Output:

“Unwind in style: Where luxury meets tranquility by waterside elegance.” #LuxuryGetaway #PoolsideParadise

View the image input and LLM output side-by-side in the playground UI.

Creating Image Datasets and Using Them in the Playground

Method 1 - CSV Upload:

  1. Prepare images → Upload to accessible URLs

  2. Create CSV → Include image URLs and variables

  3. Upload to Arize → Use "Create via CSV" in Datasets

  4. Map columns → Set image URL column as image variable

  5. Use in Playground → Reference images with {{variable_name}}

Method 2 - From Traces:

  1. Existing traces → Must contain image URLs in input/output/attributes

  2. Create dataset → Use trace-to-dataset functionality

  3. Auto-mapping → Image URLs extracted as variables

  4. Use in Playground → Same as CSV method

Supported Image Formats

  • JPG/JPEG ✅

  • PNG ✅

  • GIF ✅

  • SVG ✅

  • WebP ✅

URL Types Supported

  • Web URLs: https://example.com/image.jpg

  • Cloud Storage: gs://bucket/image.png, s3://bucket/image.jpg

  • Data URLs: data:image/png;base64,iVBORw0K...

Last updated

Was this helpful?