data-formulator / rules

---
description: Prevent information exposure through exception messages in HTTP responses
globs: py-src/**/*.py
alwaysApply: false
---

# Error Response Safety

Never return raw exception text (`str(e)`, `f"...{e}"`) directly in HTTP responses.
Python exceptions may contain stack traces, file paths, database connection strings,
API keys, or internal IP addresses — all of which are security risks (CWE-209).

See `docs/dev-guides/7-unified-error-handling.md` for the full error handling contract.

## Unified Error System

New error paths should use the `AppError` / `error_handler` infrastructure from
`errors.py` and `error_handler.py`. Legacy `message` / `error_message` bodies are
historical formats; do not add compatibility branches for them in new code.
All application-controlled errors return **HTTP 200** with `status: "error"` in the body.
Only auth errors (401/403) and uncontrolled transport errors (404/413/500) use non-200.

### Non-streaming routes

Raise `AppError` for new code or when structured error codes are useful. The
global error handler returns HTTP 200 + JSON error envelope:

```python
from data_formulator.errors import AppError, ErrorCode

# Business error — raise with appropriate code and safe message
raise AppError(ErrorCode.TABLE_NOT_FOUND, "Table not found")

# LLM / external API error — use classify_and_wrap_llm_error
from data_formulator.error_handler import classify_and_wrap_llm_error

except Exception as e:
    raise classify_and_wrap_llm_error(e) from e
```

### Streaming routes

Use `stream_error_event()` to yield a unified NDJSON error line.
Validation before the stream starts returns `200 application/json` (not NDJSON):

```python
from data_formulator.errors import AppError, ErrorCode
from data_formulator.error_handler import (
    classify_and_wrap_llm_error,
    stream_error_event,
    stream_preflight_error,
)

except AppError as e:
    yield stream_error_event(e)
except Exception as e:
    yield stream_error_event(classify_and_wrap_llm_error(e))
```

### Real examples from migrated endpoints

```python
# /data-agent-streaming — standard NDJSON error event
except Exception as e:
    logger.error("Error in data-agent-streaming", exc_info=e)
    from data_formulator.error_handler import stream_error_event, classify_and_wrap_llm_error
    yield stream_error_event(classify_and_wrap_llm_error(e))

# /generate-report-chat — same pattern
except Exception as e:
    logger.exception("generate-report-chat failed")
    from data_formulator.error_handler import stream_error_event, classify_and_wrap_llm_error
    yield stream_error_event(classify_and_wrap_llm_error(e))

# Validation failure before stream starts — use 200 JSON, not NDJSON
if not request.is_json:
    return stream_preflight_error(AppError(ErrorCode.INVALID_REQUEST, "Invalid request"))
```

### Database / workspace errors (tables.py)

Use `classify_and_raise_db_error()` instead of the legacy `sanitize_db_error_message()`:

```python
from data_formulator.routes.tables import classify_and_raise_db_error

except Exception as e:
    classify_and_raise_db_error(e)  # raises AppError with safe message
```

### Connector errors (data_connector.py)

Use `classify_and_raise_connector_error()` instead of the legacy `_sanitize_error()`:

```python
from data_formulator.data_connector import classify_and_raise_connector_error

except Exception as e:
    classify_and_raise_connector_error(e)  # raises AppError with safe message
```

### Legacy functions (deprecated)

These functions still exist as wrappers but should NOT be used in new code:
- `sanitize_db_error_message()` → use `classify_and_raise_db_error()`
- `_sanitize_error()` → use `classify_and_raise_connector_error()`
- `safe_error_response()` / `sanitize_error_message()` → use `raise AppError(...)`
- `classify_llm_error()` is used internally by `classify_and_wrap_llm_error()`

## Rules

1. **Application errors** — HTTP 200 + `status: "error"`; prefer `error: {...}` for new code and never expose exception details.
2. **Auth errors** — `AUTH_REQUIRED`/`AUTH_EXPIRED` → 401, `ACCESS_DENIED` → 403 (transport layer interception).
3. **Transport errors** (uncontrolled) — only `404` (no route), `413` (body limit), `500` (unhandled crash) may be non-200.
4. **Streaming errors** — always use `stream_error_event()` for NDJSON error lines.
5. **Logging** — always log the full exception server-side (`logger.warning` / `logger.error`
   with `exc_info=True` when needed). The client never needs the stack trace.
6. **Debug detail** — `AppError.detail` is only included in responses when `app.debug is True`.

## Bare except Policy

- **Never** use naked `except:` — always specify at least `except Exception:`
- Use specific exception types when possible (`except (ValueError, TypeError):`)
- Always log caught exceptions: `logger.warning(...)` for degraded operations, `logger.debug(...)` for expected failures
- Process cleanup code (`terminate`, `join`, `close`) is exempt from logging requirements

## Common Mistakes

```python
# ❌ BAD — raw exception leaks internal details
return jsonify({"message": str(e)}), 500
return jsonify({"error": f"Failed: {e}"}), 502
yield json.dumps({"type": "error", "error": str(e)}) + "\n"

# ❌ BAD — non-200 HTTP status code for application errors
return jsonify({"status": "error", "error_message": "..."}), 400

# ❌ BAD — naked except catches SystemExit/KeyboardInterrupt
except:
    pass

# ❌ BAD — silent swallow without logging
except Exception:
    pass

# ✅ GOOD — non-streaming (global handler returns HTTP 200)
raise AppError(ErrorCode.TABLE_NOT_FOUND, "Table not found")

# ✅ GOOD — streaming
yield stream_error_event(AppError(ErrorCode.LLM_TIMEOUT, "Timeout", retry=True))

# ✅ GOOD — graceful degradation with logging
except Exception as e:
    logger.warning("Could not read schema for %s", table_name, exc_info=e)

# ✅ GOOD — specific exception types
except (ValueError, TypeError):
    pass  # safe fallback
```