Managing Test Prompts & Datasets

Organize test prompts into datasets for reuse.

Test prompts are the user messages that models respond to. Each one is an atomic evaluation unit — models are scored independently on each test prompt.

Creating Test Prompts

Navigate to Library > Test Prompts and click Create. Each prompt has a title, the prompt text, an optional dataset assignment, and tags.

Organizing with Datasets

Datasets are lightweight collections for grouping related test prompts. Create datasets from Library > Datasets, then assign test prompts to them.

When configuring a suite, you can select an entire dataset instead of picking individual prompts. A test prompt can belong to one dataset at most.

Bulk Import

Import up to 500 test prompts at once. Use the import button on the Test Prompts page.

Cache Implications

Editing the text of a test prompt changes its content hash, which invalidates cached responses for that prompt. The title and tags can be changed freely without affecting the cache.