Swagger / OpenAPI test-readiness advisory
Purpose: score a spec before you generate scenarios from it. If a spec is thin (no auth scheme, no response schemas, no request examples), the generated test is weak or unrunnable — and without the advisory you only find out at execute time. The advisory answers up front: can a proper, runnable API test be built from this spec — and if not, what exactly is missing and how do I fix it?
It is the same band + reasons + fixes + suggested-exemplar pattern as the
TestRail / universal import advisory, applied to OpenAPI. And it grades the
same parse the generator consumes, so the advisory and create_scenario_from_swagger can never
disagree: a MANUAL verdict is a real generation gap, not a lint nit.
What gets checked
Checks are weighted by what actually blocks a runnable test — not OpenAPI-lint completeness.
Per endpoint (each scored 0–100 and banded):
| Check | Why it matters | Weight |
|---|---|---|
| 2xx response declared + has a schema | without it there is nothing to assert or extract (e.g. $.id for CRUD chaining) | heavy |
| Write op (POST/PUT/PATCH) has a request-body schema | without it the generated body is {} | heavy |
| Request body has an example | generated bodies reflect real data | medium |
application/json content type declared | the generator knows how to encode | light |
Path params ({id}) declared in parameters | undeclared params can't be typed or chained | medium |
operationId present | stable test naming (synthesized when missing — not a blocker) | light |
Spec-global:
- A concrete (absolute) server / base URL. A missing server is a soft note — the platform can
supply
{{baseUrl}}from the environment target at run time. - Parser validation issues (unresolvable
$refs, undeclared path parameters) — surfaced verbatim. - Security-scheme sanity: operations referencing undeclared schemes are flagged. A spec with no auth at all is treated as legitimately public (informational note, no penalty).
Bands
| Band | Score | Meaning |
|---|---|---|
READY | ≥ 70 | generate now |
NEEDS_WORK | 40–69 | generate, but fix the named gaps |
MANUAL | < 40 | too thin for meaningful generation — fix the spec first |
Same thresholds as the import advisory — one Band definition backs both.
Non-READY specs also get a suggestedTemplateId — the closest gold-standard exemplar
(api-crud, api-auth-login, …; browse them at GET /api/import/ideals).
API
POST /api/scanner/api-security/swagger/advisory
{"url": "https://api.example.com/v3/api-docs"} # or {"content": "<raw JSON/YAML>"}
Response:
{
"title": "Orders API",
"score": 55,
"band": "NEEDS_WORK",
"specReasons": ["concrete server / base URL present"],
"specFixes": [],
"endpoints": [
{"method": "POST", "path": "/orders", "score": 35, "band": "MANUAL",
"fixes": ["add a request body schema for this POST so a real payload can be generated"]}
],
"totalEndpoints": 2, "ready": 1, "needsWork": 0, "manual": 1,
"suggestedTemplateId": "api-crud"
}
Roles: ADMIN, ANALYST.
MCP
advise_swagger(url=... | content=... | target=..., environment=..., full=False)
Same spec-source contract as parse_swagger: a direct url, raw content (JSON or YAML), or a
registered target (its swaggerJsonUrl / baseUrl+swaggerPath is used).
The default response is lean: overall band + coverage counts + full detail only for the
endpoints that need attention; READY endpoints come back as a compact "GET /orders" list.
full=True returns the complete per-endpoint advisory (token-heavy on big specs).
Typical flow on an unfamiliar spec:
advise_swagger(url=...)→ see the band and the named gaps.- If
READY/NEEDS_WORK:create_scenario_from_swagger(...)and fix the listed gaps. - If
MANUAL: fix the spec first (thefixes[]are copy-paste actionable), or author by hand from the suggested exemplar.
Known limits
- The parser does not expose the spec's global
securityrequirement per endpoint, so endpoint-level "auth not applied" is not flagged (it would false-positive on every spec that uses global security). Auth checks are spec-global only. - Response examples are not inspected (response schemas are — which is what extraction needs).