agentleFS
joeywhelanjoeywhelan/judged-feedback/.claude/skills/feedback-triage/SKILL.md

feedback-triage

Skill0 starsChanged 51 days agoMITView on GitHub

Triage judged product feedback in Elasticsearch for a first-party retailer. Detect fit skew, attribute contradictions, safety clusters, quality clusters, and watch items from the feedback-judgments index; corroborate with raw feedback and exchanges; classify each finding into a closed action catalog; and write a findings report plus proposed elastic commands. Use when asked to analyze feedback-judgments or produce the feedback triage report.

4 other projects use a version of this.

Adopted by

Most recently taken up by

  • Other versions4 projects use a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 4

    1. civitai/civitai★ 7.3ksame name
    2. danjdewhurst/story-skills★ 417same name
    3. fullstackhouse/skills★ 1same name
    4. euroconic/pm-claude-skills★ 0same name

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repositoryOne other file it gives its agents.

The file

We split this skill into the rules it gives an agent

2 rules. 0 are common, 1 is distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 4 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use a choice answer only when its .confidence >= 0.8.

    line 39
  2. Distinctive:

    Run them in this order. A product gets at most one finding: if a later detection matches a product or line already covered, skip it.

    line 45

“Common” is measured: kept by at least 6 in 10 of the 4 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Feedback Triage

You analyze. You do not change anything. Every query runs through the elastic CLI in the current context, which is read-only. Your only outputs are two files: out/report.md and out/actions.ndjson. Thresholds live in this file, not in your judgment: a finding exists when a threshold below is crossed, and not otherwise.

Data

IndexOne document perUse it for
feedback-judgmentsfeedback item × rubric versiondetection: every threshold is computed here
feedback-rawreview or return, as submittedevidence: the customer's words (body) by feedback_id
productsproductcontext: listed_attributes, current fit_note, available

Judgment fields (ES|QL addresses object fields with dotted names):

  • Yes/no probabilities, 0 to 1: about_product, contradicts_attribute, image_mismatch, safety_concern, actionable
  • Choices: primary_issue.label (fit, quality, appearance_mismatch, functionality, damage, none), fit_direction.label (runs_small, true_to_size, runs_large, not_stated), which_attribute.label (color, material, fit, none), each with .confidence
  • Ordinal: severity.score (fractional, 0 to 3; nearest whole number is the level) and severity.confidence
  • Deterministic: product_id, category, type (review, return), rating, verified_purchase, exchange_size_delta (returns only: sizes between ordered and exchanged, +1 = one size up), submitted_at, rubric_version

A product line is the first six characters of product_id (LEFT(product_id, 6)): JKT-4801 to JKT-4805 are line JKT-48.

Filters that apply to every detection

  • rubric_version == "v1"
  • about_product > 0.8, except for safety. Feedback about shipping, delivery, or service is not product evidence.
  • Use a choice answer only when its .confidence >= 0.8.
  • Severity levels from the fractional score: level 2 or more is severity.score >= 1.5; level 3 is severity.score >= 2.5.
  • Analyze all judged feedback. The data is a fixed snapshot ending 2026-09-20, so do not filter relative to today.

Detections and thresholds

Run them in this order. A product gets at most one finding: if a later detection matches a product or line already covered, skip it.

1. Safety (SAFETY)

Threshold: at least 3 items for one product with safety_concern > 0.9.

FROM feedback-judgments
| WHERE rubric_version == "v1" AND safety_concern > 0.9
| STATS reports = COUNT(*), mean_severity = ROUND(AVG(severity.score), 2) BY product_id
| WHERE reports >= 3
| SORT reports DESC

Action: merchandising_escalation. It always goes first in the report.

2. Fit skew across a line (FIT)

Threshold, per line: at least 30 confident fit answers (fit_direction not not_stated, confidence ≥ 0.8), runs_small (or runs_large) share of at least 0.40, and at least 1.5× the same share for the rest of the category. A category baseline of 0 counts as crossed.

FROM feedback-judgments
| WHERE rubric_version == "v1" AND about_product > 0.8
    AND fit_direction.confidence >= 0.8 AND fit_direction.label != "not_stated"
| EVAL line = LEFT(product_id, 6),
       small = CASE(fit_direction.label == "runs_small", 1, 0),
       large = CASE(fit_direction.label == "runs_large", 1, 0)
| STATS fit_answers = COUNT(*), runs_small_share = ROUND(AVG(small), 2),
        runs_large_share = ROUND(AVG(large), 2) BY category, line
| SORT runs_small_share DESC

Baseline for a candidate line (substitute <category> and <line>):

FROM feedback-judgments
| WHERE rubric_version == "v1" AND about_product > 0.8 AND category == "<category>"
    AND LEFT(product_id, 6) != "<line>"
    AND fit_direction.confidence >= 0.8 AND fit_direction.label != "not_stated"
| EVAL small = CASE(fit_direction.label == "runs_small", 1, 0),
       large = CASE(fit_direction.label == "runs_large", 1, 0)
| STATS fit_answers = COUNT(*), runs_small_share = ROUND(AVG(small), 2),
        runs_large_share = ROUND(AVG(large), 2)

Corroboration (required in the report, not a threshold): exchanges by size direction for the line.

FROM feedback-judgments
| WHERE rubric_version == "v1" AND type == "return" AND exchange_size_delta IS NOT NULL
    AND LEFT(product_id, 6) == "<line>"
| STATS returns = COUNT(*), sized_up = SUM(CASE(exchange_size_delta > 0, 1, 0)),
        sized_down = SUM(CASE(exchange_size_delta < 0, 1, 0)),
        mean_delta = ROUND(AVG(exchange_size_delta), 2)

Action: content_update on every product in the line.

3. Attribute contradiction (CONTRA)

Threshold, per product and attribute: at least 15 product-feedback items, and at least 25% of them with contradicts_attribute > 0.8 naming that attribute (which_attribute.label, confidence ≥ 0.8).

FROM feedback-judgments
| WHERE rubric_version == "v1" AND about_product > 0.8
| EVAL hit = contradicts_attribute > 0.8 AND which_attribute.confidence >= 0.8,
       color = CASE(hit AND which_attribute.label == "color", 1, 0),
       material = CASE(hit AND which_attribute.label == "material", 1, 0),
       fit = CASE(hit AND which_attribute.label == "fit", 1, 0),
       photo = CASE(hit AND image_mismatch > 0.5, 1, 0)
| STATS items = COUNT(*), color = SUM(color), material = SUM(material), fit = SUM(fit),
        mentions_photos = SUM(photo) BY product_id
| EVAL color_rate = ROUND(color * 1.0 / items, 2), material_rate = ROUND(material * 1.0 / items, 2),
       fit_rate = ROUND(fit * 1.0 / items, 2)
| WHERE items >= 15 AND (color_rate >= 0.25 OR material_rate >= 0.25 OR fit_rate >= 0.25)
| SORT items DESC

Pull the product's listed_attributes to name the listed value. Action: content_ticket. Never propose changing a listed attribute: customers may be reacting to the photo, the attribute, or both (mentions_photos tells you how many cite photos), and only a person with the physical product can settle which is wrong.

4. Quality cluster (QUALITY)

Threshold, per product: at least 3 items with primary_issue.label == "quality" (confidence ≥ 0.8) and severity level 2 or more, at a rate of at least 2× the rest of the category.

FROM feedback-judgments
| WHERE rubric_version == "v1" AND about_product > 0.8
| EVAL q = CASE(primary_issue.label == "quality" AND primary_issue.confidence >= 0.8
                AND severity.score >= 1.5, 1, 0)
| STATS items = COUNT(*), quality = SUM(q) BY category, product_id
| EVAL rate = ROUND(quality * 1.0 / items, 3)
| WHERE quality >= 3
| SORT rate DESC

Baseline for a candidate (substitute <category> and <product_id>):

FROM feedback-judgments
| WHERE rubric_version == "v1" AND about_product > 0.8 AND category == "<category>"
    AND product_id != "<product_id>"
| EVAL q = CASE(primary_issue.label == "quality" AND primary_issue.confidence >= 0.8
                AND severity.score >= 1.5, 1, 0)
| STATS items = COUNT(*), quality = SUM(q)
| EVAL rate = ROUND(quality * 1.0 / items, 3)

Action: route_to_review. A quality cluster is real but the fix (supplier, materials, returns policy) is not in the catalog, so a person reads the items.

5. Watch (WATCH)

Threshold, per product with no finding above: at least 5 confident complaints (primary_issue.label != "none", confidence ≥ 0.8) spread across at least 3 issue types.

FROM feedback-judgments
| WHERE rubric_version == "v1" AND about_product > 0.8
    AND primary_issue.label != "none" AND primary_issue.confidence >= 0.8
| STATS complaints = COUNT(*), issue_types = COUNT_DISTINCT(primary_issue.label) BY product_id
| WHERE complaints >= 5 AND issue_types >= 3
| SORT complaints DESC

Action: watch. No command, no ticket: a scattered, low-volume pattern is noted so the next run can compare.

Route to review (report section, not a finding)

List, by feedback_id, every item of a QUALITY finding, plus any item with severity level 3 and low confidence that is not already in a finding:

FROM feedback-judgments
| WHERE rubric_version == "v1" AND severity.score >= 2.5 AND severity.confidence < 0.8
| KEEP feedback_id, product_id, severity.score, severity.confidence

Investigating a finding

For each finding, before classifying it:

  1. Examples: at most five, the highest actionable among the finding's items. Get IDs from feedback-judgments (KEEP feedback_id … | SORT actionable DESC | LIMIT 5), then the words from feedback-raw:

    FROM feedback-raw | WHERE feedback_id IN ("<id1>", "<id2>") | KEEP feedback_id, type, rating, body
  2. Corroboration: mean rating of the finding's reviews, and for fit findings the exchange query above.
  3. Product: listed_attributes, fit_note, and available from products.

Action catalog (closed)

ActionForProposesCommand in actions.ndjson
merchandising_escalationSAFETYavailable: false and a known_issues entryyes
content_updateFITfit_note, fit_guidance, and a known_issues entry on every product in the lineyes
content_ticketCONTRAa ticket for the content team: listed value, what customers report, photo mentionsno
route_to_reviewQUALITYthe feedback IDs, in the Route to review sectionno
watchWATCHnothing; the counts, for comparison next runno

The only fields a proposal may change are fit_note (one of runs_small, true_to_size, runs_large), fit_guidance, known_issues, and available. If a finding seems to need anything else, describe it in the finding's text; do not invent an action.

Commands

Products have no document IDs you can rely on; always select them by product_id with update-by-query. Every command runs in the admin context, so it includes --use-context admin. known_issues entries are {"type": <finding type, lowercase>, "source_finding": <finding ID>, "added": <today, YYYY-MM-DD>}. Confirm flags with elastic es update-by-query --help before writing a command. --use-context is a global option accepted by every command; do not test it, and never run a command that includes it (you are not permitted to). Write it into the proposal as shown.

Content update for a fit line:

elastic es update-by-query --index products --use-context admin --refresh true --json --query '{"prefix":{"product_id":"<line>"}}' --script '{"lang":"painless","source":"ctx._source.fit_note = params.fit_note; ctx._source.fit_guidance = params.fit_guidance; ctx._source.known_issues.add(params.issue)","params":{"fit_note":"runs_small","fit_guidance":"<one sentence of sizing advice>","issue":{"type":"fit","source_finding":"<finding ID>","added":"<date>"}}}'

Merchandising escalation for a safety finding:

elastic es update-by-query --index products --use-context admin --refresh true --json --query '{"term":{"product_id":"<product_id>"}}' --script '{"lang":"painless","source":"ctx._source.available = false; ctx._source.known_issues.add(params.issue)","params":{"issue":{"type":"safety","source_finding":"<finding ID>","added":"<date>"}}}'

Outputs

Finding IDs are F-<TYPE>-<product_id or line>, for example F-FIT-JKT-48, F-SAFETY-CKW-1102.

out/actions.ndjson: one JSON object per line, one line per command, only for merchandising_escalation and content_update:

{"finding_id": "F-FIT-JKT-48", "action": "content_update", "command": "elastic es update-by-query ..."}

out/report.md:

# Feedback triage — rubric v1

<one paragraph: how many findings of each type, and the one that needs attention first>

## Findings

### F-SAFETY-… — <product name> (<product_id>)
- **Type / action:** SAFETY → merchandising_escalation
- **Evidence:** <the threshold numbers, and the threshold they crossed>
- **Corroboration:** <ratings, exchanges, product fields>
- **Confidence:** <how confident the underlying answers are, and any caveat>
- **Examples:** <up to five quoted bodies with feedback_id>
- **Proposed action:** <in words; command in actions.ndjson>

(… one section per finding: SAFETY first, then by severity, then by volume; WATCH last)

## Route to review
<feedback_id, product_id, and one line of the body, per item>

The report contains findings only: no commentary on the rubric, the judge, or the data pipeline, and no recommendations outside the action catalog.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.