agentleFS
shuji-bonjishuji-bonji/ai-agent-architecture/.claude/skills/translation-quality/SKILL.md

translation-quality

Skill8 starsChanged 2 months agoMITView on GitHub

Bilingual (ja/en) documentation translation quality evaluation using xCOMET. The source language is resolved per page pair from .claude/translation.json, not fixed.

1 other project uses a version of this.

Adopted by

Most recently taken up by

  • Other versions1 project uses a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 1

    1. senshinji/claude-translation-skill★ 39same name

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repository3 other files it gives its agents.

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 0 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use xcomet:xcomet_batch_evaluate for multi-segment evaluation

    line 48
  2. Very specific:

    Read .claude/translation.json before scoring, and set source_lang / target_lang from the direction resolved for that pair

    line 49
  3. Very specific:

    Use deepl:translate-text for re-translation when --fix is specified

    line 57

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Translation Quality Evaluation Skill

Purpose

Evaluate and ensure high-quality Japanese → English translations using automated quality metrics (xCOMET) with clear decision criteria and actionable outputs.

Why This Skill?

  • Manual translation review is time-consuming and subjective
  • xCOMET provides objective, reproducible quality scores
  • Clear thresholds enable automated decision-making
  • Results can be directly embedded into documentation

Inputs

InputTypeDescription
page pairtwo .md filesOne page in both locales. Resolved from .claude/translation.json — see Pairing and Direction below
source languageja or enWhich side of the pair is the original. Resolved per pair, never assumed
thresholdnumber (0-1)Minimum acceptable score (default: from config, 0.85)

Pairing and Direction

This repository keeps locales in directories, not filename suffixes: docs/ is English and docs/ja/ is Japanese. docs/ja/part-2/layers.md pairs with docs/part-2/layers.md. Only files listed under filePairs (currently README.ja.md ↔ README.md) use the suffix form.

The original is not always Japanese. .claude/translation.json holds defaultSource for the repository and a sourceOverrides list of globs for pages written the other way round. Resolve the direction per pair before scoring; scoring a translation as if it were the source inverts the metric.

Outputs

OutputTypeDescription
quality_reportMarkdownEmbeddable report with scores and recommendations
segment_detailsTablePer-segment scores and issues
action_itemsListSpecific corrections needed

Constraints

MUST

  • Use xcomet:xcomet_batch_evaluate for multi-segment evaluation
  • Read .claude/translation.json before scoring, and set source_lang / target_lang from the direction resolved for that pair
  • Generate report in embeddable Markdown format
  • Include both summary and detailed segment scores

SHOULD

  • Evaluate at least 10 representative segments per document
  • Include segment text in reports for segments scoring < 0.90
  • Use deepl:translate-text for re-translation when --fix is specified

MUST NOT

  • Mark documents as "Excellent" if any segment scores < 0.85
  • Edit the source side of a pair to raise a score. Only the translation side is revised
  • Lower threshold or add a sourceOverrides entry to make a failing pair pass
  • Skip segments containing only code blocks
  • Include raw JSON in final report (format as Markdown tables)

Workflow

Step 1: File Detection

Resolve pairs from .claude/translation.json
- localeRoots: docs/ja/X  <->  docs/X   (English side is docs/ excluding docs/ja/)
- filePairs:   explicit pairs such as README.ja.md <-> README.md
- exclude:     glob list to skip (redirect stubs, .vitepress, public)
- If a single file is given, find its counterpart on the other side
- If a directory is given, walk it and collect every pair
Then resolve the direction: defaultSource, overridden by sourceOverrides globs

Step 2: Segment Extraction

Extract translatable segments from both files:
1. Document title (# heading)
2. Section headings (## / ### headings)
3. Key paragraphs (first 2-3 sentences per section)
4. Table content
5. List items with substantive content

Skip:
- Code blocks (```)
- URLs and links
- Mermaid diagrams
- Pure formatting elements

Step 3: Quality Evaluation

Call xCOMET MCP with extracted pairs:

mcp: xcomet
tool: xcomet_batch_evaluate
params:
  pairs: [{ source: "日本語", translation: "English" }, ...]
  source_lang: "<resolved source>"   # ja or en, per pair
  target_lang: "<the other side>"
  response_format: "markdown"

Step 4: Result Classification

Apply decision criteria to scores and generate report.

Step 5: Report Generation

Output embeddable Markdown with:

  • Summary table
  • Quality criteria reference
  • Segment details (for scores < 0.95)
  • Recommendations

Decision Criteria

ConditionActionRationale
Average ≥ 0.95✅ Excellent - PublishProfessional quality
0.90 ≤ Average < 0.95✅ Good - Minor reviewMay benefit from polish
0.85 ≤ Average < 0.90⚠️ Review requiredCheck flagged segments
0.70 ≤ Average < 0.85🔄 Re-translateParameter adjustment needed
Average < 0.70❌ Manual revisionSignificant issues present

Per-Segment Criteria

ScoreLabelAction
≥ 0.95ExcellentNo action needed
0.90-0.94GoodReview if time permits
0.85-0.89AcceptableConsider revision
< 0.85ProblematicMust revise

Examples

Example 1: Single File Evaluation

Input:

/check-translation docs/skills/overview.md

Process:

  1. Resolve pair: docs/ja/skills/overview.md ↔ docs/skills/overview.md, source ja
  2. Extract 15 segments
  3. Call xcomet:xcomet_batch_evaluate
  4. Average score: 0.978
  5. Generate report

Output:

## Translation Quality Report

### Summary

| Metric | Value |
|--------|-------|
| Files checked | 1 |
| Segments evaluated | 15 |
| Average score | **0.978** |
| Status | ✅ Excellent |

### Quality Criteria

| Score | Rating | Action |
|-------|--------|--------|
| ≥ 0.95 | Excellent | Publish |
| 0.90-0.94 | Good | Minor review |
| 0.85-0.89 | Acceptable | Review |
| < 0.85 | Problematic | Revise |

### Segment Details

All 15 segments scored ≥ 0.95. No issues detected.

### Recommendation

✅ Ready for publication without changes.

Example 2: Directory Batch Evaluation

Input:

/check-translation docs/

Process:

  1. Find 8 file pairs in docs/
  2. Extract 40 total segments
  3. Batch evaluate all segments
  4. Generate comprehensive report

Output:

## Translation Quality Report

### Summary

| Metric | Value |
|--------|-------|
| Files checked | 8 |
| Segments evaluated | 40 |
| Average score | **0.972** |
| Status | ✅ Excellent |
| Critical errors | 0 |

### Per-File Results

| File | Score | Status |
|------|-------|--------|
| README.md | 0.975 | ✅ Excellent |
| concepts/01-vision.md | 0.978 | ✅ Excellent |
| skills/anti-patterns.md | 0.957 | ✅ Excellent |
| workflows/patterns.md | 0.977 | ✅ Excellent |

### Segments Needing Attention (< 0.95)

| Source | Translation | Score |
|--------|-------------|-------|
| MCPサーバーの開発・運用... | Organizing security risks... | 0.898 |

### Recommendation

✅ Overall excellent quality. Optional: Review 1 segment above.

Anti-Patterns

Pattern: Raw Score Reporting

Problematic Approach:

Score: 0.9750812888145447

Why It's Problematic: Overly precise numbers suggest false accuracy

Correct Approach:

Score: **0.975** (Excellent)

Pattern: Missing Context

Problematic Approach:

3 segments below threshold

Why It's Problematic: Reviewer cannot act without seeing the text

Correct Approach:

| Source | Translation | Score |
|--------|-------------|-------|
| 実際のテキスト | Actual text | 0.84 |

Related MCPs

MCPUsageRequired
xcomet-mcp-serverxcomet_batch_evaluate, xcomet_evaluateRequired
deepl-mcptranslate-text (for --fix option)Optional

References

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.