AI Prompt for Debugging Flaky Tests

Tame flaky tests with this AI prompt for debugging: diagnose timing, randomness and ordering causes, then get a deterministic rewrite plus CI safeguards.

The prompt

Role: You are a test-reliability engineer who specializes in flaky tests — tests that pass and fail without code changes.

Context: Test framework: [TEST_FRAMEWORK], language: [LANGUAGE]. The flaky test: [TEST_CODE]. It fails roughly [FAILURE_RATE, e.g. 1 in 5 runs] with this error: [FAILURE_MESSAGE]. The test involves [ASYNC_CALLS / TIMERS / RANDOMNESS / SHARED_STATE / EXTERNAL_SERVICES — pick all that apply].

Task: 1) List the most likely sources of non-determinism in this test, ranked by probability. 2) For the top cause, explain the exact mechanism (e.g., race between promise resolution and assertion). 3) Rewrite the test to be deterministic, using proper waiting, seeding, isolation, or mocking as appropriate. 4) Suggest one CI-level safeguard (retry policy, quarantine, or detection) with its trade-offs.

Format: Diagnosis list first, then the mechanism explanation, then the fixed test in a code block, then the CI suggestion.

Constraints: Do not fix flakiness by adding arbitrary sleep() calls — use deterministic synchronization. Keep the test's original intent and assertions unchanged.

Variables to fill in

Replace each [variable] in the prompt with your own details:

[TEST_FRAMEWORK] [LANGUAGE] [TEST_CODE] [FAILURE_RATE] [FAILURE_MESSAGE] [ASYNC_CALLS / TIMERS / RANDOMNESS / SHARED_STATE / EXTERNAL_SERVICES]

Example input

TEST_FRAMEWORK=Jest, LANGUAGE=JavaScript, failure rate=1 in 4, error=expect(received).toBe(expected) on a list that sometimes has 9 items instead of 10, involves async API calls

How to use this prompt

Include the failure rate and the exact flaky assertion — patterns like 'sometimes 9 instead of 10' point straight at races. List which non-determinism sources the test touches.

Tips for the best results

Never accept sleep() as a fix; ask for explicit waits or event-based synchronization instead.
If the test depends on time, ask the AI to rewrite it with fake timers.
Run the fixed test 20+ times locally before trusting it — ask the AI for a loop command.