Prompt Playground
Test new prompts variants and run evals using scenarios.
Goals
The prompt playground, management and optimizer are closely related. The playground is a tool for testing new prompts variants and run custom metrics on scenarios by hand. The optimizer uses LLMs to automatically optimize the prompts. Often, after you’re done optimizing, you’ll want to store the best performing prompt in our prompt management.
- Experimentation: The prompt playground is an easy way to experiment with new prompts.
- One-Click Save: Save results as a versioned and managed prompt in one-click.
- Native Evals Support: We support evals natively, so you can select a dataset of scenarios & run evals. This makes sure the prompt changes you make are actually improving overall system performance.
- Function calling: Native support for function calling.
We’re going to merge the playground with Prompt Optimizer to help you optimize your prompts in a more unified way.
How it works
You can store, manage and access your prompts here.
Start a new playground session
Start a new playground session
You can create a playground session from scratch by visiting the playground.
You can also take an existing prompt from the prompt management and start iterating on it in the playground.
Add tools
Add tools
We support OpenAI & Anthropic function calling including syntax highlighting, auto-complete and auto-formatting.
Run evals
Run evals
You can re-use all existing datasets of scenarios and run evals on them in 1-click. In this example below, we’re using the facts compare evaluator to compare the factual content of the LLM responses with expected responses for each row in the dataset of scenarios.
Save as a prompt
Save as a prompt
You can save the work-in-progress prompt as a new prompt.
Prompt Playground
Test new prompts variants and run evals using scenarios.
Goals
The prompt playground, management and optimizer are closely related. The playground is a tool for testing new prompts variants and run custom metrics on scenarios by hand. The optimizer uses LLMs to automatically optimize the prompts. Often, after you’re done optimizing, you’ll want to store the best performing prompt in our prompt management.
- Experimentation: The prompt playground is an easy way to experiment with new prompts.
- One-Click Save: Save results as a versioned and managed prompt in one-click.
- Native Evals Support: We support evals natively, so you can select a dataset of scenarios & run evals. This makes sure the prompt changes you make are actually improving overall system performance.
- Function calling: Native support for function calling.
We’re going to merge the playground with Prompt Optimizer to help you optimize your prompts in a more unified way.
How it works
You can store, manage and access your prompts here.
Start a new playground session
Start a new playground session
You can create a playground session from scratch by visiting the playground.
You can also take an existing prompt from the prompt management and start iterating on it in the playground.
Add tools
Add tools
We support OpenAI & Anthropic function calling including syntax highlighting, auto-complete and auto-formatting.
Run evals
Run evals
You can re-use all existing datasets of scenarios and run evals on them in 1-click. In this example below, we’re using the facts compare evaluator to compare the factual content of the LLM responses with expected responses for each row in the dataset of scenarios.
Save as a prompt
Save as a prompt
You can save the work-in-progress prompt as a new prompt.