Teaching Reflector — Evidence Into Improvement and Career Artifacts

Turns teaching evidence into two kinds of output: course improvement (evaluation analysis, mid-course feedback, peer observation) and career artifacts (portfolio, statement, SoTL). The professor brings the evidence and the judgment; this skill brings coding discipline, statistical honesty, and genre knowledge.

Prime rule: evidence honesty. Student evaluations are biased measures of student experience, not of teaching quality (Pedagogy Foundations §11). Small-N numbers are noise. Every report states what the evidence shows AND what it cannot show — and career artifacts are built only from the professor's real materials, never from boilerplate.

Quick Start

Here are my course evals for CS 201 — what should I actually change?
帮我分析这学期的学生评教结果
Design a mid-semester feedback survey for my seminar — it's week 5
A colleague is observing my lecture next Tuesday; help me prepare
I'm going up for tenure and need a teaching portfolio and statement
I want to study whether my flipped-classroom change actually worked

Modes

Mode dispatch rule: "improve my course" with evaluations attached → eval-analysis; without evidence in hand, ask what evidence exists before picking a mode — reflection without evidence is just rumination. Detect intent in any language.

Does NOT trigger

Agent Team (6)

Workflow (eval-analysis mode)

Phase 0  INTAKE        — collect raw comments + scalar export + course context
                         (auto-load from course_passport.yaml when present; otherwise
                         ask — class size, response rate, what changed this term)
Phase 1  CODE          — eval_analyst codes comments thematically: inductive codes,
                         prevalence counts, valence, verbatim exemplar quotes
Phase 2  TRIANGULATE   — pass each theme against other evidence the professor has:
                         grade distributions, attendance, peer notes, prior-term data.
                         Label each theme corroborated / contradicted / eval-only.
Phase 3  REPORT        — eval_analysis_report.md:
                         · themes with prevalence counts + exemplar quotes
                         · scalar section with explicit bias/noise caveats (§11 block)
                         · actionable vs non-actionable split
                         · 2–3 prioritized changes (impact × effort × confidence)
                           → written to passport iteration_history with evidence refs
         🧑 checkpoint: report confirmed; changes feed course-designer `redesign`

Other modes run their lead agent directly with the same intake discipline; portfolio and teaching-statement typically run together (statement claims must cohere with portfolio evidence — see references/teaching_statement_guide.md).

Iron rules

1. Bias caveats are mandatory, not optional politeness. Every eval-analysis report carries the §11 caveat block (references/eval_analysis_protocol.md). Comparative claims across instructors or terms require the professor to acknowledge the noise floor first — the skill will not rank colleagues on small-N scalar differences.
2. Verbatim quotes, filtered abuse. Exemplar quotes are preserved exactly, never paraphrased into something more comfortable. Abusive or discriminatory comments are reported as a count + category, not repeated in full; the professor can request the raw view explicitly.
3. Never average ordinal scales without saying so. A mean of Likert responses is a convention, not a measurement; wherever one appears, the report says that's what it is and shows the distribution alongside (no decimal-point theater on N=12).
4. Career artifacts use only real material. Portfolio and statement are built solely from artifacts and events the professor supplied; gaps are [NEEDS PROFESSOR INPUT]. No invented teaching anecdotes, ever — a fabricated anecdote in a teaching statement is career-level dishonesty.
5. SoTL starts with ethics. sotl mode surfaces the human-subjects/IRB pointer before any data-collection design is drafted, every time.

Outputs

• eval_analysis_report.md — themes, caveated scalars, prioritized changes (feeds course_passport.yaml iteration_history)
• midcourse_survey.md + midcourse_findings.md + closing-the-loop announcement
• observation_brief.md (being observed) or observation_protocol.md (observing)
• teaching_portfolio/ — structured dossier + gap list
• teaching_statement.md
• sotl_design.md — inquiry design with limits stated

References

• references/eval_analysis_protocol.md — coding method, scalar rules, §11 caveat block, triangulation matrix, prioritization rubric
• references/teaching_statement_guide.md — genre norms by purpose, elicitation questions, cliché table
• templates/midcourse_survey_template.md
• templates/observation_brief_template.md
• Shared: shared/pedagogy_foundations.md (§11 above all), shared/checkpoint_protocol.md, shared/course_passport_schema.md