Skip to main content

Queue load testing (Kafka)

Load-test a message queue broker at production rates — produce messages at configured rates per phase and measure throughput, publish latency, and errors. Kafka is the first supported queue type; the adapter model means further brokers (RabbitMQ, SQS, …) become additions, not new features.

Setup​

  1. Register the broker as a QUEUE target in your environment — its URL is the bootstrap servers (host:port). SASL credentials go in the environment vault and are referenced by credentialTag — never literal values.
  2. Create a queue test: topic, load phases, optional message template ({{vars}} supported), producer config, and acceptance criteria.

API​

POST /api/queue-tests
{
  "projectId": 1, "name": "Orders topic load", "targetId": 12,
  "queueType": "KAFKA", "topic": "orders",
  "messageTemplate": "{\"orderId\": 42}",
  "phases": [
    {"name": "warm", "durationSec": 30, "messagesPerSecond": 10},
    {"name": "load", "durationSec": 120, "messagesPerSecond": 100}
  ],
  "producerConfig": {"batchSize": 5},
  "criteria": [
    {"metric": "publishP95", "operator": "LTE", "threshold": 500},
    {"metric": "errorRate", "operator": "LTE", "threshold": 1}
  ]
}
  • POST /api/queue-tests/validate — author-time preflight (same checks the runner makes).
  • POST /api/queue-tests/{id}/run — {environmentId?, tags?, confirm?}. Running against a production-flavored environment requires confirm: true (blast-radius guard).
  • GET /api/queue-tests/{id}/runs, GET /api/queue-tests/runs/{runId} — normalized metrics + criteria verdict.

Metrics (normalized queue family): producedTotal, peakRate, publishP50/P95/P99, errorRate, byPhase[]. A criterion referencing a metric the run didn't produce fails loudly with a named reason — never a silent pass. A run that produced zero messages is FAILED (broker unreachable / auth / topic), never an empty green.

MCP​

create_queue_test · list_queue_tests · get_queue_test · run_queue_test (with the same confirm guard) · get_queue_test_result (latest run or by run id).

v1 limits (honest)​

  • Producer side only. The test floods the topic and measures publish throughput/latency/errors. It does not verify the consumer kept up — consumer-group lag correlation and service profiling under load are the named follow-ups.
  • Kafka SASL/PLAIN auth is supported via the vault; TLS/mTLS variants are not yet wired.
  • No UI page yet — API + MCP first.