Skip to main content

Tag-driven workflows

Practical recipes that show how to operate the platform from prompts a human would actually type. Every example below is a real LLM conversation against a Claude Desktop wired to proofarc-mcp (or any other MCP-aware client).

You don't memorize ids. You don't open the UI to look up environment numbers. You say what you want, and the resolver narrows until it executes.

Recipe 1 — Run all smoke tests in staging​

The prompt:

Run smoke tests on staging

What Claude does:

1. run_by_tag(tags=["smoke"])
   → {reason: "missing_required",
      ask_user: "Which environment? Available: dev, staging, prod.",
      candidates: [{id:11,name:"dev"}, {id:12,name:"staging"}, ...]}

2. (user already said staging, so Claude skips re-asking)
   run_by_tag(tags=["smoke"], environment="staging")
   → {reason: "ambiguous",
      ask_user: "smoke tests exist in 2 projects: payments, auth. Which?"}

   Claude turns to user: "Smoke tests exist in 2 projects: payments, auth. Which?"

3. (user: "payments")
   run_by_tag(tags=["smoke"], environment="staging", project="payments")
   → {dry_run: false,
      queued: [
        {kind:"api_scenario", id:204, name:"Login flow",  executionId:"…"},
        {kind:"api_scenario", id:205, name:"Cart",        executionId:"…"},
        {kind:"ui_test",      id:79,  name:"Signup happy", jobId:"…"}
      ],
      skipped: []}

4. summarize_execution(executionId="…", kind="api_scenario")  ← per item
   → human-readable pass/fail summary

Three turns. No id was ever quoted by the user.

Recipe 2 — Preview before running​

When a tag could match a lot of tests and you want a sanity check first:

Show me what would run if I trigger smoke on staging in payments

Claude:

run_by_tag(tags=["smoke"], environment="staging", project="payments", dry_run=true)
→ {dry_run: true,
   would_queue: {
     api_scenarios: [{id:204, name:"Login flow"}, {id:205, name:"Cart"}],
     ui_tests:      [{id:79,  name:"Signup happy"}]
   },
   note: "API scenario matches are derived from execution history…"}

Claude shows the list to the user. User confirms or refines. Then call again without dry_run=true.

Recipe 3 — Discover what tags exist​

Tags aren't a list endpoint — they're emergent from past executions (for API scenarios) or from the tags field on UI tests. To see what's around:

What tags do my UI tests use?

Claude:

find_ui_tests(project="payments")
→ [{id:79, name:"Signup", tags:["smoke", "regression"]},
   {id:80, name:"Cart",   tags:["regression"]},
   {id:81, name:"Search", tags:["smoke"]}]

Claude tells the user: "In payments I see two tags on UI tests: smoke and regression."

For API scenarios, the analogous discovery is via the past-execution audit trail rather than a list endpoint. If you find yourself doing this a lot, the real fix is to add a static tags column to ApiScenario — currently a backlog item, not yet shipped.

Recipe 4 — Run only API tests, skip UI​

run_by_tag defaults to both api and ui kinds. To restrict:

Run API smoke tests on staging in payments (no UI)

Claude:

run_by_tag(
  tags=["smoke"],
  environment="staging",
  project="payments",
  kinds=["api"],
)

Recipe 5 — Multi-tag (any-match)​

Tags match by set intersection — a test is matched if it shares any tag with the request:

Run tests tagged smoke or regression in staging

Claude:

run_by_tag(tags=["smoke", "regression"], environment="staging", project="payments")

A test tagged just regression matches; a test tagged just smoke matches; a test tagged both matches once. There is no AND-form today.

Recipe 6 — Find by name, not tag​

Tags are great when you have a discipline of using them. Substring name match works when you don't:

Find scenarios that mention "login"

Claude:

find_scenarios(name_contains_text="login")
→ [{id:204, name:"Login flow",      projectId:1, authType:"BEARER", stepCount:6},
   {id:218, name:"Login retry case", projectId:1, authType:"BEARER", stepCount:4}]

Recipe 7 — Filter scenarios by auth type​

When you're auditing test coverage for a particular auth flow:

Show me every OAuth2 scenario in the payments project

Claude:

find_scenarios(project="payments", test_type="OAUTH2_CLIENT")
→ [{id:217, name:"OAuth2 user fetch", authType:"OAUTH2_CLIENT", stepCount:4}, ...]

Valid test_type values: BEARER, OAUTH2_CLIENT, BASIC, API_KEY, NONE.

Recipe 8 — Use a pipeline instead​

When you have a defined, ordered, multi-step flow (e.g. API → UI → security scan), use a pipeline rather than fanning out by tag:

Run the release-validation pipeline in staging

Claude:

1. list_pipelines(project="payments")
   → [{id:7, name:"release-validation", stepCount:5}]

2. run_pipeline(pipeline="release-validation", environment="staging", project="payments")
   → {pipelineId:7, executionId:"f2e3...", status:"RUNNING"}

Differences vs run_by_tag:

run_by_tagrun_pipeline
Orderingparallel fan-outstrict step order
Membershipimplicit (tag matches)explicit (defined in pipeline)
Failure handlingpartial success, all queuedpipeline stops on first failure
Best forad-hoc developer-driven runsscheduled / CI / formal releases

Recipe 9 — Just author a scenario​

Create a CRUD scenario against http://payments-api in the payments staging environment, with bearer auth tag "admin"

Claude:

create_api_scenario(
  name="Payments CRUD",
  project="payments",
  environment="staging",
  base_url="http://payments-api",
  steps=[
    {stepOrder:1, name:"create", httpMethod:"POST",   endpointPath:"/payments", requestBody:"{...}", expectedStatusCodes:[201],
     extractions:[{variableName:"id", extractionType:"JSONPATH", extractionExpression:"$.id", isRequired:true}]},
    {stepOrder:2, name:"get",    httpMethod:"GET",    endpointPath:"/payments/{{id}}", expectedStatusCodes:[200]},
    {stepOrder:3, name:"delete", httpMethod:"DELETE", endpointPath:"/payments/{{id}}", expectedStatusCodes:[200, 204]},
  ],
  auth_config={
    "authType":"BEARER",
    "tokenSource":"LOGIN_ENDPOINT",
    "loginEndpoint":"/auth/login",
    "credentialTag":"admin",
  },
)
→ {id:307, name:"Payments CRUD", stepCount:3}

Note: project and environment are names, not ids. The resolver finds them.

Anti-pattern: walking the tree by hand​

This is what we're avoiding:

list_projects()                       ← 1 round trip
   ↓ find "payments" id = 1
list_environments(project_id=1)       ← 2nd round trip
   ↓ find "staging" id = 12
list_scenarios(project_id=1)          ← 3rd round trip (tool didn't even exist before)
   ↓ filter by tag manually
execute_scenario(204, environment_id=12) ← per scenario, more round trips

Five+ round trips for one user intent, and the LLM has to do its own filter logic. The new tools collapse this to one call.

See also​