Instrumentation Bug Hunting

Arguments

$ARGUMENTS - The library name, optionally followed by focus context.

Format: <library> [focus on <area>]

Examples:

• /bug-hunt redis — broad bug hunting across all redis functionality
• /bug-hunt redis focus on pub sub interactions — prioritize pub/sub patterns
• /bug-hunt mysql2 focus on prepared statements and transactions — prioritize those areas

Parsing: The first word of $ARGUMENTS is always the library name. Everything after it is the optional focus context. All references to <library> below mean this parsed first word — NOT the raw $ARGUMENTS string.

Library-to-GitHub-Repo Mapping

Use this mapping to clone the package source code for analysis:

E2E Test Variants

Each library has ESM and CJS variants. Use the CJS variant as the primary target for bug hunting:

Phase 0: Environment Setup

0.1 Parse and validate the arguments

Extract the library name (first word) and optional focus context (remaining words) from the arguments.

The library must be one of: fetch, firestore, grpc, http, ioredis, mongodb, mysql, mysql2, nextjs, pg, postgres, prisma, redis, upstash-redis-js.

If the library is invalid, list the valid options and stop.

If focus context is provided, it will guide Phases 1 and 2 to prioritize that area of the library's functionality.

0.2 Docker Setup (Claude Code Web only)

Check if Docker is running. If not, start it:

dockerd --storage-driver=vfs &>/tmp/dockerd.log &
# Wait for Docker to be ready
for i in $(seq 1 30); do
  docker info &>/dev/null 2>&1 && break
  sleep 1
done
docker info &>/dev/null 2>&1 || { echo "Docker failed to start. Check /tmp/dockerd.log"; exit 1; }

If Docker is already running, skip this step.

0.3 Clone the package source code (for analysis only)

If the library has a GitHub repo (see mapping above), clone it for reference:

git clone --depth 1 <repo-url> /tmp/<library-name>-source

This is read-only reference material — you will NOT modify this repo.

0.4 Create a working branch

Skip this step if you are already on a dedicated branch (e.g., in Claude Code Web where each session has its own branch).

git checkout -b bug-hunt/<library>-$(date +%Y-%m-%d)

Phase 1: Develop Understanding

If focus context was provided, prioritize your analysis around that area. For example, if the focus is "pub sub interactions", concentrate on pub/sub-related code paths in the instrumentation, tests, and package source.

1.1 Analyze the Instrumentation Code

Read the instrumentation code at:

src/instrumentation/libraries/<library>/Instrumentation.ts

Identify:

• Which functions from the package are patched/instrumented
• The patching strategy (what gets wrapped, when, and how)
• Any helper files in the same directory
• If focus context provided: Which patches relate to the focus area, and what's missing?

1.2 Analyze Existing E2E Tests

Review the CJS variant's test files:

• src/instrumentation/libraries/<library>/e2e-tests/cjs-<library>/src/index.ts — all test endpoints
• src/instrumentation/libraries/<library>/e2e-tests/cjs-<library>/src/test_requests.mjs — which endpoints are called

Understand what functionality is already tested and identify coverage gaps.

• If focus context provided: What tests already exist for the focus area? What's missing?

1.3 Analyze the Package Source Code

If you cloned the package source, read it to understand:

• The package's entry points and full API surface
• Functions that are currently patched vs functions that exist but aren't patched
• Alternative call patterns, overloads, and edge cases
• If focus context provided: Deep-dive into the focus area's API surface and usage patterns

Phase 2: Identify Potential Gaps

If focus context was provided, prioritize bugs related to that area. You may still note other potential issues, but test the focus area first.

Reason about potential issues. Consider:

• Untested parameters: Parameter combinations not covered by existing tests
• Alternative call patterns: Can patched functions be invoked differently (callbacks vs promises, different overloads)?
• Missing patches: Functions that should be instrumented but aren't
• Edge cases: Null/undefined values, empty results, large payloads, streaming, connection errors
• ORM/wrapper usage: Libraries like Sequelize, Knex, Prisma that wrap the base driver — are those call paths instrumented?
• Real-world usage patterns: How is the package typically used in production apps?

Produce a prioritized list of potential bugs to investigate.

Phase 3: Initialize Bug Tracking Document

Create BUG_TRACKING.md in the CJS e2e test directory:

# Path: src/instrumentation/libraries/<library>/e2e-tests/cjs-<library>/BUG_TRACKING.md

# <library> Instrumentation Bug Tracking

Generated: <current date and time>

## Summary

- Total tests attempted: 0
- Confirmed bugs: 0
- No bugs found: 0
- Skipped tests: 0

---

## Test Results

(Tests will be documented below as they are completed)

Phase 4: Write Tests and Verify Issues

For each potential bug, follow this workflow:

4.1 Initial Setup (Once)

Navigate to the CJS e2e test directory:

cd src/instrumentation/libraries/<library>/e2e-tests/cjs-<library>/

Start Docker containers:

docker compose up -d --build --wait

Install dependencies:

docker compose exec -T app npm install

4.2 Test Each Potential Bug (Repeat for each)

A. Clean Previous Test Data

rm -rf .tusk/traces/* .tusk/logs/*

B. Write New Test Endpoint

Add a new endpoint to src/index.ts that exercises the potential bug. Also add the corresponding request to src/test_requests.mjs.

Example:

app.get("/test/my-new-test", async (req, res) => {
  // Your test code here
  res.json({ success: true });
});

C. Test in DISABLED Mode (No Instrumentation)

Start server without instrumentation:

docker compose exec -d -e TUSK_DRIFT_MODE=DISABLED app sh -c "npm run build && npm run dev"

Wait for server to start:

sleep 5

Hit the endpoint:

docker compose exec app curl -s http://localhost:3000/test/my-new-test

Verify: Response is correct and endpoint works.

Stop the server:

docker compose exec app pkill -f "node" || true
sleep 2

If the endpoint fails in DISABLED mode:

• Update BUG_TRACKING.md with status: "Skipped - Failed in DISABLED mode"
• Fix the test code or move on to next potential bug

D. Test in RECORD Mode (With Instrumentation)

Clean traces and logs:

rm -rf .tusk/traces/* .tusk/logs/*

Start server in RECORD mode:

docker compose exec -d -e TUSK_DRIFT_MODE=RECORD app sh -c "npm run build && npm run dev"

Wait for server to start:

sleep 5

Hit the endpoint:

docker compose exec app curl -s http://localhost:3000/test/my-new-test

Wait for spans to export:

sleep 3

Stop the server:

docker compose exec app pkill -f "node" || true
sleep 2

Check for issues:

1. Endpoint returns error or wrong response vs DISABLED mode:
   • BUG FOUND: Instrumentation breaks functionality
   • Update BUG_TRACKING.md: Status "Confirmed Bug - RECORD mode failure", Failure Point "RECORD"
   • Keep the endpoint, move to next
2. No traces created (ls .tusk/traces/):
   • BUG FOUND: Instrumentation failed to capture traffic
   • Update BUG_TRACKING.md: Status "Confirmed Bug - No traces captured", Failure Point "RECORD"
   • Keep the endpoint, move to next

E. Test in REPLAY Mode

Run the Tusk CLI to replay:

docker compose exec -T -e TUSK_ANALYTICS_DISABLED=1 app tusk drift run --print --output-format "json" --enable-service-logs --disable-sandbox

Check for issues:

1. Test fails ("passed": false in JSON output):
   • BUG FOUND: Replay doesn't match recording
   • Update BUG_TRACKING.md: Status "Confirmed Bug - REPLAY mismatch", Failure Point "REPLAY"
2. No logs created (ls .tusk/logs/):
   • BUG FOUND: Replay failed to produce logs
   • Update BUG_TRACKING.md: Status "Confirmed Bug - No replay logs", Failure Point "REPLAY"

3. Logs contain TCP warnings:

   docker compose exec app cat .tusk/logs/*.log | grep -i "TCP called from inbound request context"

   • BUG FOUND: Unpatched dependency detected
   • Update BUG_TRACKING.md: Status "Confirmed Bug - Unpatched dependency", Failure Point "REPLAY"

F. No Bug Found

If all modes pass with no issues:

• Update BUG_TRACKING.md: Status "No Bug - Test passed all modes"
• Remove the test endpoint from src/index.ts and src/test_requests.mjs
• Move to next potential bug

Phase 5: Bug Tracking Documentation Format

After each test, append to BUG_TRACKING.md:

### Test N: [Brief description]

**Status**: [Confirmed Bug | No Bug | Skipped]

**Endpoint**: `/test/endpoint-name`

**Failure Point**: [DISABLED | RECORD | REPLAY | N/A]

**Description**:
[What this test was trying to uncover]

**Expected Behavior**:
[What should happen]

**Actual Behavior**:
[What actually happened]

**Error Logs**:

[Relevant error messages, stack traces, or warnings]

**Additional Notes**:
[Observations, potential root causes, context]

---

Important: Update BUG_TRACKING.md immediately after each test — do not batch updates.

Phase 6: Cleanup and Commit

After testing all potential bugs:

docker compose down

Clean up cloned package source:

rm -rf /tmp/*-source

Final state of the e2e test files:

• src/index.ts should contain ONLY the original endpoints + new endpoints that expose confirmed bugs
• src/test_requests.mjs should be updated to include requests to bug-exposing endpoints
• BUG_TRACKING.md should have accurate summary counts and all test results

Commit the changes:

git add src/instrumentation/libraries/<library>/e2e-tests/cjs-<library>/
git commit -m "bug-hunt(<library>): add e2e tests exposing instrumentation bugs

Found N confirmed bugs in <library> instrumentation.
See BUG_TRACKING.md for details."

Push the branch (skip if in Claude Code Web where the session handles this):

git push origin bug-hunt/<library>-$(date +%Y-%m-%d)

Success Criteria

1. Created BUG_TRACKING.md before starting any tests
2. Tested all identified potential bugs
3. Updated BUG_TRACKING.md after each individual test
4. Only bug-exposing endpoints remain in test files
5. Removed test endpoints that didn't expose bugs
6. Accurate summary counts in BUG_TRACKING.md
7. Changes committed and pushed to a bug-hunt/ branch