Skip to main content

Swagger / OpenAPI test-readiness advisory

Purpose: score a spec before you generate scenarios from it. If a spec is thin (no auth scheme, no response schemas, no request examples), the generated test is weak or unrunnable — and without the advisory you only find out at execute time. The advisory answers up front: can a proper, runnable API test be built from this spec — and if not, what exactly is missing and how do I fix it?

It is the same band + reasons + fixes + suggested-exemplar pattern as the TestRail / universal import advisory, applied to OpenAPI. And it grades the same parse the generator consumes, so the advisory and create_scenario_from_swagger can never disagree: a MANUAL verdict is a real generation gap, not a lint nit.

What gets checked

Checks are weighted by what actually blocks a runnable test — not OpenAPI-lint completeness.

Per endpoint (each scored 0–100 and banded):

CheckWhy it mattersWeight
2xx response declared + has a schemawithout it there is nothing to assert or extract (e.g. $.id for CRUD chaining)heavy
Write op (POST/PUT/PATCH) has a request-body schemawithout it the generated body is {}heavy
Request body has an examplegenerated bodies reflect real datamedium
application/json content type declaredthe generator knows how to encodelight
Path params ({id}) declared in parametersundeclared params can't be typed or chainedmedium
operationId presentstable test naming (synthesized when missing — not a blocker)light

Spec-global:

  • A concrete (absolute) server / base URL. A missing server is a soft note — the platform can supply {{baseUrl}} from the environment target at run time.
  • Parser validation issues (unresolvable $refs, undeclared path parameters) — surfaced verbatim.
  • Security-scheme sanity: operations referencing undeclared schemes are flagged. A spec with no auth at all is treated as legitimately public (informational note, no penalty).

Bands

BandScoreMeaning
READY≥ 70generate now
NEEDS_WORK40–69generate, but fix the named gaps
MANUAL< 40too thin for meaningful generation — fix the spec first

Same thresholds as the import advisory — one Band definition backs both.

Non-READY specs also get a suggestedTemplateId — the closest gold-standard exemplar (api-crud, api-auth-login, …; browse them at GET /api/import/ideals).

API

POST /api/scanner/api-security/swagger/advisory
{"url": "https://api.example.com/v3/api-docs"} # or {"content": "<raw JSON/YAML>"}

Response:

{
"title": "Orders API",
"score": 55,
"band": "NEEDS_WORK",
"specReasons": ["concrete server / base URL present"],
"specFixes": [],
"endpoints": [
{"method": "POST", "path": "/orders", "score": 35, "band": "MANUAL",
"fixes": ["add a request body schema for this POST so a real payload can be generated"]}
],
"totalEndpoints": 2, "ready": 1, "needsWork": 0, "manual": 1,
"suggestedTemplateId": "api-crud"
}

Roles: ADMIN, ANALYST.

MCP

advise_swagger(url=... | content=... | target=..., environment=..., full=False)

Same spec-source contract as parse_swagger: a direct url, raw content (JSON or YAML), or a registered target (its swaggerJsonUrl / baseUrl+swaggerPath is used).

The default response is lean: overall band + coverage counts + full detail only for the endpoints that need attention; READY endpoints come back as a compact "GET /orders" list. full=True returns the complete per-endpoint advisory (token-heavy on big specs).

Typical flow on an unfamiliar spec:

  1. advise_swagger(url=...) → see the band and the named gaps.
  2. If READY / NEEDS_WORK: create_scenario_from_swagger(...) and fix the listed gaps.
  3. If MANUAL: fix the spec first (the fixes[] are copy-paste actionable), or author by hand from the suggested exemplar.

Known limits

  • The parser does not expose the spec's global security requirement per endpoint, so endpoint-level "auth not applied" is not flagged (it would false-positive on every spec that uses global security). Auth checks are spec-global only.
  • Response examples are not inspected (response schemas are — which is what extraction needs).