Performance Optimization

Profile, diagnose, and fix performance bottlenecks. Covers CPU profiling, memory analysis, database query optimization, frontend rendering, bundle size reduction, and algorithmic improvements. Always benchmark before and after.

Workflow

Step 1: Establish Baseline

Before optimizing anything, measure the current state:

• Backend latency: Check logs, APM data, or run time curl -s <endpoint> for quick measurement.
• Database queries: Enable query logging or use EXPLAIN ANALYZE on suspect queries.
• Frontend rendering: Check for layout thrashing, excessive re-renders, or large bundle sizes.
• Memory usage: Check process memory with ps aux | grep <process> or language-specific profilers.
• CPU usage: Check with top, htop, or application profilers.

Record baseline metrics:

| Metric              | Current Value | Target Value |
|---------------------|---------------|--------------|
| Response time (p50) | <value>       | <target>     |
| Response time (p99) | <value>       | <target>     |
| Memory usage        | <value>       | <target>     |
| Bundle size         | <value>       | <target>     |
| Query count         | <value>       | <target>     |

Step 2: Identify Bottlenecks

Systematically check each performance layer:

Database (most common bottleneck):

• N+1 queries: Look for queries inside loops. Search for ORM calls (.find(), .get(), .load()) inside iteration blocks.
• Missing indexes: Run EXPLAIN ANALYZE on slow queries. Look for sequential scans on large tables.
• Over-fetching: Check for SELECT * when only a few columns are needed.
• Missing pagination: Look for unbounded queries without LIMIT.
• Connection pool exhaustion: Check pool size vs. concurrent request count.

Backend code:

• Algorithmic complexity: Look for O(n^2) or worse in hot paths (nested loops over collections).
• Synchronous I/O: Blocking calls in async contexts (file reads, HTTP calls without await).
• Unnecessary serialization: Repeated JSON.parse/stringify, excessive object cloning.
• Missing caching: Repeated expensive computations or API calls with same inputs.
• Memory leaks: Growing arrays, unclosed connections, event listener accumulation.

Frontend rendering:

• Excessive re-renders: Components re-rendering on every state change (React: missing useMemo, useCallback, React.memo).
• Layout thrashing: Reading then writing DOM properties in a loop (forces synchronous reflow).
• Large bundles: Importing entire libraries when only one function is needed.
• Unoptimized images: Large images without lazy loading, missing srcset, uncompressed formats.
• Blocking resources: Render-blocking CSS/JS in the critical path.

I/O and Network:

• Sequential API calls: Requests that could be parallelized with Promise.all().
• Missing compression: Response bodies not gzipped/brotli compressed.
• Chatty protocols: Many small requests instead of batched operations.
• Missing connection reuse: Not using HTTP keep-alive or connection pooling.

Step 3: Apply Fixes

For each identified bottleneck, apply the appropriate fix:

Database fixes:

-- Add index for slow query
CREATE INDEX CONCURRENTLY idx_orders_user_created
ON orders (user_id, created_at DESC);

-- Replace N+1 with eager loading (ORM example)
-- Before: users.map(u => u.getPosts())  -- N+1
-- After:  User.findAll({ include: [Post] })  -- 1 query with JOIN

-- Add pagination
SELECT * FROM orders WHERE user_id = $1
ORDER BY created_at DESC LIMIT 20 OFFSET 0;

Caching strategies:

• In-memory cache: For frequently accessed, rarely changing data (config, feature flags).
• Redis/Memcached: For shared cache across instances (session data, API responses).
• HTTP caching: Set Cache-Control, ETag, Last-Modified headers for static and semi-static responses.
• Memoization: Cache pure function results keyed by arguments.

Frontend fixes:

• Code-split routes with dynamic imports: React.lazy(() => import('./Page')).
• Virtualize long lists with react-window or @tanstack/virtual.
• Debounce expensive event handlers (search, resize, scroll).
• Use will-change CSS property sparingly for known animation targets.
• Preload critical resources: <link rel="preload" href="font.woff2" as="font">.

Algorithmic fixes:

• Replace array lookups with Set/Map for O(1) access.
• Use binary search on sorted data instead of linear scan.
• Replace recursive solutions with iterative ones to avoid stack overflow.
• Use streaming/chunked processing for large datasets instead of loading all into memory.

Step 4: Benchmark After Changes

Re-measure with the same methodology as Step 1:

| Metric              | Before  | After   | Improvement |
|---------------------|---------|---------|-------------|
| Response time (p50) | 450ms   | 120ms   | 73% faster  |
| Response time (p99) | 2100ms  | 380ms   | 82% faster  |
| Memory usage        | 512MB   | 340MB   | 34% less    |
| Bundle size         | 1.2MB   | 680KB   | 43% smaller |
| Query count         | 47/req  | 5/req   | 89% fewer   |

Verify no regressions:

• Run the full test suite after optimization changes.
• Check that behavior is identical (same outputs for same inputs).
• Load test under realistic concurrency to verify improvements hold.

Output Format

## Performance Analysis

### Baseline Metrics
<table of current measurements>

### Bottlenecks Identified

#### [P1] <Issue> — <location>
**Impact**: <quantified impact on latency/memory/CPU>
**Root Cause**: <why this is slow>
**Fix**:
```<language>
// before
<slow code>

// after
<optimized code>

Expected Improvement: <estimated gain>

Optimization Plan (priority order)

1. <highest impact, lowest effort fix>
2. <next priority>
3. <next priority>

After Optimization

Remaining Opportunities

• <future optimization ideas that were out of scope>

## Examples

**User**: "This API endpoint takes 3 seconds to respond"
**Action**: Read the endpoint handler, trace database queries, identify N+1 patterns or missing indexes, suggest eager loading and index additions, benchmark before/after.

**User**: "Our React app is sluggish when scrolling the list page"
**Action**: Check for virtualization, look at component re-render patterns, check bundle size, suggest React.memo, virtualization, and code-splitting.

**User**: "Memory keeps growing until the service crashes"
**Action**: Look for event listener leaks, growing caches without eviction, unclosed database connections, circular references preventing garbage collection.

**User**: "Profile this function for performance"
**Action**: Analyze the algorithm complexity, look for unnecessary allocations, check I/O patterns, suggest optimizations with benchmarks.

## Best Practices

- Always measure before optimizing. Intuition about bottlenecks is often wrong.
- Optimize the highest-impact bottleneck first. A 10x improvement on a function called once matters less than a 2x improvement on one called 10,000 times.
- Profile in production-like conditions. Development environments hide real performance issues.
- Avoid premature optimization. If it is fast enough, do not add complexity to make it faster.
- Document why an optimization was made, not just what. Future developers may simplify away your optimization not knowing it was intentional.
- Cache invalidation is harder than caching. Always design the invalidation strategy first.
- Test with realistic data volumes. An O(n^2) algorithm is fine for 100 items but breaks at 100,000.
- Consider the 80/20 rule: 80% of performance issues come from 20% of the code.
- Set performance budgets and alert when they are exceeded rather than only reacting to complaints.