support-ticket-to-eval
Skill0 starsChanged 5 months agoMITView on GitHub
Use when turning support tickets, user complaints, helpdesk transcripts, or failed assistant responses into reusable eval cases with expected behavior, assertions, and failure modes.
In this repository19 other files it gives its agents.
- agent-handoff-briefSkill
- api-doc-to-tool-specSkill
- architecture-rfc-reviewSkill
- bug-report-to-reproSkill
- changelog-to-upgrade-planSkill
- competitive-research-briefSkill
- customer-call-to-prdSkill
- database-migration-risk-reviewSkill
- dependency-upgrade-risk-reviewSkill
- incident-log-to-postmortemSkill
- messy-notes-to-decision-logSkill
- privacy-data-flow-mapSkill
- prompt-to-eval-rubricSkill
- repo-launch-readiness-reviewSkill
- roadmap-to-release-planSkill
- security-review-checklistSkill
- telemetry-to-product-insightsSkill
- tool-permission-auditSkill
- workflow-automation-scoutSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 14
Use when support evidence should become a reusable regression or model-behavior eval.
- Distinctive: line 15
Do not use when the ticket only needs a one-off human reply and no future check is useful.
- Distinctive: line 28
Write assertions that can be checked by a human or automated judge.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Support Ticket To Eval
Purpose
Turn real support pain into repeatable eval cases for product, support, or agent behavior.
Fit
- Use when support evidence should become a reusable regression or model-behavior eval.
- Do not use when the ticket only needs a one-off human reply and no future check is useful.
Inputs
- Support ticket, complaint, chat transcript, or failed assistant output.
- Product area, user role, and known expected behavior if available.
- Any sensitive details that must be redacted.
Workflow
- Redact private data while preserving the failure pattern.
- Identify the user goal, current behavior, expected behavior, and severity.
- Convert the case into a minimal input that reproduces the behavior.
- Write assertions that can be checked by a human or automated judge.
- Record variants that should be tested later.
Output
Produce Markdown with:
- Eval Scenario
- Source Evidence
- Test Input
- Expected Behavior
- Failure Mode
- Assertions
- Variants
Validation
- The eval preserves the original user pain without exposing secrets.
- Assertions are concrete enough to judge.
- Expected behavior is not invented when product intent is unknown.
- Severity is justified by evidence.
- The case can be reused without the full ticket context.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.