QA Getting Started

This page is for the QA volunteer who tests app.proofarc.ai and files bug reports against the release repo. It walks through the first project, the first test in each surface area, and what a good bug report looks like.

If you're a developer onboarding to build with the platform, start at the Quick Start instead.

Access

What	Where
Platform	https://app.proofarc.ai
Swagger / API docs	https://app.proofarc.ai/swagger-ui/index.html
Documentation	https://docs.proofarc.ai
File QA bugs	https://github.com/proofarc/cloud-platform-test-collaboration/issues
Login	Your <admin-user> / <admin-password> — never write real creds in issues or YAML; use those placeholders instead.

The "never write real creds" rule isn't optional. If you accidentally paste a real password into an issue body or a test YAML, ping the engineering team to redact it as soon as you notice.

Step 1 — Create a project

1. Sidebar → Projects → New Project.
2. Name it QA Test Project (or similar).
3. Add two applications:
   • User Service UI — Type WEB_APP, Tech REACT.
   • User Service API — Type REST_SERVICE, Tech SPRING_BOOT.
4. Git Provider: pick Skip Git — not configured (no Git creds needed for QA work).

Step 2 — Link environment + confirm targets

1. Open your project → Link Environment → pick development.

2. The development env should already have:

   Target	Type	URL
   User Service UI	WEB_APP	http://user-service-ui
   User Service API	REST_SERVICE	http://user-service:8089

3. If they're missing, Environments → development → add them.

4. Confirm the environment has a default credential set (a credential row with isDefault=true or defaultCredentialTag filled in). Without it, UI tests that reference {{username}} / {{password}} will refuse to dispatch with a clear error — the platform won't run a test it knows is doomed.

Step 3 — UI testing (primary focus)

3a. Paste a test in the Advanced Editor

1. UI Tests → Advanced Editor.
2. Select your project and the User Service UI target.
3. Paste:

steps:
  - action: NAVIGATE_TO
    value: "{{baseUrl}}"
  - action: WAIT_FOR_ELEMENT
    selector: "[data-testid=login-form]"
    value: "5"
  - action: CLEAR
    selector: "[data-testid=login-username]"
  - action: SEND_KEYS
    selector: "[data-testid=login-username]"
    value: "{{username}}"
  - action: CLEAR
    selector: "[data-testid=login-password]"
  - action: SEND_KEYS
    selector: "[data-testid=login-password]"
    value: "{{password}}"
  - action: CLICK
    selector: "[data-testid=login-submit]"
  - action: WAIT_FOR_ELEMENT
    selector: "[data-testid=create-user-btn]"
    value: "5"
  - action: TAKE_SCREENSHOT
  - action: VALIDATE_TEXT
    selector: "body"
    expectedText: "User Management"

4. Run → pick development → credentials are auto-selected.
5. Check the results page: per-step pass/fail with screenshots.

3b. Use the wizard

1. UI Tests → New Test → wizard opens.
2. Select your project and the User Service UI target.
3. Step 2 — Discover Page: pick a credential, click "Discover Pages." Optionally use Pick from Page to visually click elements — clicking directly on an input or button now selects that input/button, not the wrapper around it.
4. Step 3 — Build Steps: drag discovered elements into the step list, or add steps manually. Steps are drag-reorderable.
5. Step 4 — Run: execute and review.

3c. Test the same flow on Playwright as well as WebDriver

When you run a UI Test or Suite, the run dialog has a Drivers picker:

• WebDriver — Selenium against Chrome.
• Playwright — Microsoft Playwright against Chromium.

Pick both on critical flows. The platform dispatches one job per (driver × browser) combination. Differences between the two surface engine-specific bugs early — that's part of the patent claim.

3d. Wrap tests in a suite

1. UI Test Suites → Create → add multiple tests.
2. Suite runs every test in sequence. Each test's per-step results are still drillable.

What's worth testing in the UI surface

• Login flow (the YAML above).
• Navigation between pages.
• Form submission (create user, edit user).
• Cross-engine (3c) — run identical tests on WebDriver and Playwright; compare.
• Screenshot at each step.
• Variable resolution: {{username}}, {{password}}, {{baseUrl}} should never appear literally in the rendered page — if they do, the test is mis-wired.

See UI Testing → Tips & Gotchas for engine-specific behavior (Playwright vs WebDriver timing, click-bubbling, the JS-multi-line YAML literal-block syntax).

Step 4 — API testing

4a. Create a scenario with the Swagger Wizard

1. API Scenarios → Create Scenario.

2. Project + environment + User Service API target.

3. Swagger URL: http://user-service:8089/v3/api-docs.

4. Parse & Continue — the wizard discovers every endpoint.

5. Configure Flow: select endpoints. For each step you can:

   • Duplicate — copy the step in place (new icon next to delete in the step row). Use this when the same endpoint runs twice in a flow with different expected statuses, e.g. read-after-write returns 200, verify-after-delete returns 404.
   • Add custom step — bottom of the list. Adds a blank step (GET /) you fill in by hand — for endpoints not in the Swagger spec, or path variations.

   The full CRUD verification flow this enables:

   1. GET    /api/users          → 200
   2. POST   /api/users          → 201, extract userId
   3. GET    /api/users/{{userId}} → 200    (read after write)
   4. DELETE /api/users/{{userId}} → 200
   5. GET    /api/users/{{userId}} → 404    (verify deleted — duplicate of step 3)

6. Configure auth — Bearer token via login endpoint (/auth/login, tokenPath $.accessToken) — and execute.

Variables vs path params: {{userId}} is a variable the platform extracts from a previous step's response via JSONPath. The Swagger path parameter {id} is something different — the platform substitutes the extracted variable into the URL at runtime. If you see {userId} (single braces) in the rendered URL, your variable name doesn't match.

4b. Browse the platform's own API

https://app.proofarc.ai/swagger-ui/index.html — every platform endpoint, browsable and runnable directly. Useful for understanding why your test is producing the result it produces.

Step 5 — Performance

1. Performance Scenarios → create or run one against the User Service API target.
2. Phases: warm-up (5 users / 30s) → sustained (20 users / 60s).
3. Endpoints: /api/users GET.
4. Run → review throughput (req/s), latencies (p95, p99), error rate.

Convert an API scenario to load — full CRUD, not just a GET. Open an API scenario (e.g. User CRUD) → Convert to Performance. The whole lifecycle carries over: POST/PUT writes included, the created id chained through get/update/delete, and unique data generated per virtual user so concurrent creates don't collide. Add phases + SLO thresholds and run.

Two things that must never silently pass (file it if they do):

• A run whose environment needs auth but has no resolvable credential must refuse (CREDENTIAL_UNRESOLVED) — not dispatch a run that 403s on every request.
• A run that errors 100% must never report PASS — with no SLO it reports NO_SLO_DECLARED / passed:false; with an SLO threshold it reports FAIL.

The sliders for SLO thresholds and load config now have proper label spacing — if you ever see two labels merged into one (e.g. 10060s), file it, because that regression is one we already fixed once.

Step 6 — Compliance

This is new since your last guide. Compliance runs map your existing scan / API / UI test results to control evidence across SOC 2, PCI-DSS, ISO 27001, NIST SSDF.

1. Compliance → pick a project at the top.
2. Pick a profile → Run.
3. Environment picker: only environments your project has linked appear. If empty, link one in project settings — the picker no longer falls back to listing every environment in the platform.
4. Targets: multi-select via checkboxes. "Select all" / "Clear all" shortcuts available. Execute against just the targets you check.
5. Read the result row carefully — Skipped counters are not failures. They mean "this tool isn't applicable to that target type" (e.g. NMAP against a WEB_APP). A control gap is only counted when a test actually ran and failed.

See Compliance → Running a Profile for the full target-type → compatible-tools table.

Step 7 — Pipelines

1. Pipelines → create one with steps:
   • API Scenario
   • UI Test Suite
   • Performance Test
2. Execute → live progress; per-step drill-down on failure.

Alternative — driving tests via MCP

Everything above is the web UI. You can drive the same tests from an AI client (Claude Desktop, an MCP client) or a script:

1. Get a token: POST https://app.proofarc.ai/api/auth/login with {"username":"<user>","password":"<pass>"} → .token.
2. Connect your MCP client to https://app.proofarc.ai/mcp (Authorization: Bearer <token>).
3. list_projects to sanity-check, then find_scenarios → execute_scenario (API), run_ui_test (UI), or convert_api_to_performance_scenario → run_performance_scenario (performance).

See the Tester Quick Start for the full step-by-step and Python client walkthrough for a scripted example.

YAML cheat sheet

Field	Used for	Example
value	Input data (SEND_KEYS, NAVIGATE_TO)	"{{username}}"
expectedText	Assertion text (VALIDATE_TEXT)	"User Management"
selector	CSS selector for element	"[data-testid=login-btn]"
timeout	Wait time in seconds (as a string)	"5"

Variables resolved at runtime: {{username}}, {{password}}, {{baseUrl}} — from environment credentials and target config.

VALIDATE_TEXT uses contains match, not exact.

Always CLEAR before SEND_KEYS to avoid text concatenation on retries.

For multi-line JavaScript inside EXECUTE_JS, use a YAML literal block (value: |), not a single-quoted string — see the YAML format reference.

Using Claude as your assistant

Open prompts that work well:

• "Write a YAML test for a login page with fields #email and #password."
• "This test failed with <error> — what's wrong?"
• Paste a screenshot → "What CSS selectors should I use?"
• "Generate 5 negative test cases for user registration."

Filing bugs

Open issues at proofarc/cloud-platform-test-collaboration/issues. The shape that produces the fastest turnaround:

Title:    [Area] One-line symptom
Where:    https://app.proofarc.ai/... → exact click sequence
Expected: What should have happened
Actual:   What happened
Repro:    Minimal steps (numbered)
Screenshot: <attach>
Job ID:   <uuid if it's a test/scan execution failure>

Cred placeholders only. Use <admin-user> and <admin-password> in issue bodies, YAML, screenshots that show the page. If a real cred slips in, mention it immediately — engineering will redact and rotate.

One bug per issue. Two distinct bugs in one issue means one gets closed when the other ships, and the open one drifts.

The Job ID matters. For any UI test or scan failure, the URL contains a UUID like 365d9608-58d1-4022-9d9a-9fe47eddd76e — pasting it in the issue body lets us pull the exact row from the DB without a back-and-forth.