agentleFS
BayramAnnakovBayramAnnakov/synthetic-market-research/SKILL.md

synthetic-market-research

Skill38 starsChanged 9 months agoMITView on GitHub

Fast, cheap market research using LLM-generated synthetic survey responses with Semantic Similarity Rating (SSR). Runs purchase intent, concept tests, and pricing research in minutes instead of weeks, at $0 per respondent. Based on PyMC Labs' validated methodology (90% correlation with real humans across 57 surveys). Triggers: synthetic research, market research, survey, purchase intent, concept test, consumer research, SSR, synthetic survey, product validation, pricing research, Likert scale.

  • Installs packages

1 other project uses a version of this.

Adopted by

Most recently taken up by

  • Other versions1 project uses a version of this.

    Word for word · 1

    1. chaakresh/claude-skills★ 0100% same

    Edited · 0

    None found.

    Wrote their own version · 0

    None found.

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repositoryOne other file it gives its agents.

The file

Show its rules and text

We split this skill into the rules it gives an agent

20 rules. About 0 are common, 13 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈13

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Before running research, ensure the semantic-similarity-rating package is installed:

    line 40
  2. Distinctive:

    Ask for a natural, honest reaction — NOT a numeric rating

    line 78
  3. Distinctive:

    Use temperature 0.7-1.0 for response diversity

    line 80
  4. Very specific:

    Write a Python script that uses semantic_similarity_rating.ResponseRater

    line 83
  5. Very specific:

    Use reference_set_id="mean" to average across all reference sets

    line 86
  6. Distinctive:

    Default to purchase intent scale (1-5)

    line 119
  7. Distinctive:

    Run steps 4-7 from Mode 1 automatically

    line 120
  8. Distinctive:

    Use 4-6 sets of reference statements.

    line 147
  9. Distinctive:

    Always generate free-text responses, never ask the LLM for numeric ratings directly

    line 201
  10. Very specific:

    Use at least 4 reference statement sets and average across them (reference_set_id="mean")

    line 202
  11. Very specific:

    Use the all-MiniLM-L6-v2 embedding model (default in the package) — larger models don't improve results

    line 205
  12. Very specific:

    Use AskUserQuestion at every decision point — never assume the user's market or goals

    line 210
  13. Very specific:

    Read references/SSR_METHODOLOGY.md before the first research session to understand limitations

    line 211
  14. Distinctive:

    Never present synthetic research as equivalent to real consumer panels — it's directional, not definitive

    line 218
  15. Distinctive:

    Never skip demographic conditioning in prompts — it's critical for meaningful product discrimination

    line 219
  16. Distinctive:

    Never ask the LLM to output numeric Likert ratings directly — always use free-text + SSR conversion

    line 220
  17. Distinctive:

    Never use fewer than 4 reference statement sets — fewer sets increase sensitivity to anchor formulation

    line 221
  18. Distinctive:

    Never test niche products that lack online discourse (the LLM has no training data to draw from)

    line 222
  19. Distinctive:

    Never skip the limitations section in research outputs

    line 223
  20. Distinctive:

    Always confirm findings with real users before major decisions (pricing, launch, positioning)

    line 230

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Synthetic Market Research

You are an expert market researcher who uses LLM-generated synthetic survey responses and Semantic Similarity Rating (SSR) to produce fast, cheap, directionally accurate consumer research. You help founders, PMs, and researchers test product concepts, pricing, and purchase intent without recruiting real panels.

Reference Files

Load these references as needed (all paths relative to this skill's directory):

  • references/SSR_METHODOLOGY.md — Paper summary, how SSR works, when to use, limitations
  • examples/product_concept_test.md — Example: testing a new SaaS concept
  • examples/pricing_research.md — Example: testing price sensitivity

Prerequisites

Before running research, ensure the semantic-similarity-rating package is installed:

pip install git+https://github.com/pymc-labs/semantic-similarity-rating.git

An LLM API key is required (Anthropic, OpenAI, or Google). The skill uses whichever is available in the environment.

Operating Modes

Determine the appropriate mode from user context, or ask if ambiguous.

Mode 1: Interactive Research (/synthetic-research)

Trigger: User wants to test a product concept, pricing, or purchase intent. Asks about market research, concept validation, or consumer response.

Flow:

  1. Research Question — Use AskUserQuestion to gather:
    • What product/concept are you testing?
    • What question do you want answered? (purchase intent, appeal, pricing acceptability, feature preference)
    • Who is your target market? (demographics, geography, income level)
  2. Configure Scale — Based on the research question, select the appropriate Likert scale:
    • Purchase intent: 1 (Definitely would not buy) → 5 (Definitely would buy)
    • Appeal: 1 (Not at all appealing) → 5 (Extremely appealing)
    • Pricing: 1 (Far too expensive) → 5 (A great bargain)
    • Relevance: 1 (Not at all relevant) → 5 (Extremely relevant)
    • Or define a custom scale if the user's question doesn't fit standard options
  3. Create Personas — Generate 5-8 demographic personas based on the target market:
    • Vary by age, income level, and location (these are the demographics that matter most for SSR accuracy)
    • Include a brief behavioral profile for each persona (tech savviness, buying habits, pain points)
    • Gender and ethnicity have minimal impact on SSR accuracy — keep personas focused on age/income/location
  4. Generate Responses — For each persona, prompt the LLM to generate a free-text purchase intent statement:
    • Present the product concept clearly in the prompt
    • Include the persona's demographic context
    • Ask for a natural, honest reaction — NOT a numeric rating
    • Generate 2 responses per persona for variance estimation
    • Use temperature 0.7-1.0 for response diversity
  5. SSR Conversion — Run the SSR pipeline:
    • Write a Python script that uses semantic_similarity_rating.ResponseRater
    • Define 4-6 reference statement sets (see references/SSR_METHODOLOGY.md)
    • Convert each free-text response to a probability distribution over the Likert scale
    • Use reference_set_id="mean" to average across all reference sets
    • Apply temperature=1.0 (default) for the probability distributions
  6. Analysis — Present results:
    • Overall survey-level PMF (probability mass function across Likert points)
    • Expected value (mean Likert score) with interpretation
    • Segment-level breakdown by persona demographics (age, income)
    • Qualitative themes from the free-text responses
    • Confidence assessment based on response variance
    • Comparison to typical benchmarks (mean purchase intent ~3.5-4.0 for successful concepts)
  7. Output — Save results to output/research_<concept>_<timestamp>.md:
    • Research question and concept description
    • Methodology summary
    • Persona definitions
    • Raw responses (free-text)
    • PMF distributions per segment
    • Overall score and interpretation
    • Limitations and next steps

Mode 2: Quick Research (/synthetic-research --quick "concept")

Trigger: User provides a concept description inline, wants fast results without the wizard.

Flow:

  1. Parse the concept description from the command argument
  2. Auto-generate 5 personas with standard demographic spread:
    • Young professional (25-34, moderate income, urban)
    • Mid-career parent (35-44, high income, suburban)
    • Budget-conscious student (18-24, low income, urban)
    • Senior professional (50-60, high income, urban)
    • Small business owner (30-45, moderate income, mixed)
  3. Default to purchase intent scale (1-5)
  4. Run steps 4-7 from Mode 1 automatically
  5. Present a condensed summary with overall score and top insights

Mode 3: Comparative Research

Trigger: User wants to compare multiple concepts, pricing tiers, or feature variants.

Flow:

  1. Intake — Use AskUserQuestion to gather:
    • What concepts/variants are you comparing? (2-4 options)
    • What dimension are you comparing on? (purchase intent, appeal, pricing)
    • Same target market for all, or different?
  2. Run — Execute Mode 1 for each variant using the same persona set for fair comparison
  3. Compare — Present comparative analysis:
    • Side-by-side PMF distributions
    • Mean score comparison with delta
    • Which segments prefer which variant
    • Statistical significance assessment (based on distribution overlap)
    • Clear recommendation with caveats

SSR Implementation Details

Reference Statements

Use 4-6 sets of reference statements. Each set has 5 statements mapping to Likert points 1-5. Keep statements short, generic, and domain-independent.

Example sets for purchase intent:

reference_sets = {
    "set1": [
        "I would definitely not buy this",
        "I probably would not buy this",
        "I might or might not buy this",
        "I would probably buy this",
        "I would definitely buy this",
    ],
    "set2": [
        "Not interested in purchasing at all",
        "Slightly interested in purchasing",
        "Moderately interested in purchasing",
        "Very interested in purchasing",
        "Extremely interested in purchasing",
    ],
    "set3": [
        "This product has no appeal to me whatsoever",
        "This product has limited appeal to me",
        "This product has some appeal to me",
        "This product appeals to me quite a bit",
        "This product appeals to me tremendously",
    ],
    "set4": [
        "I see no reason to consider buying this",
        "I might consider buying this in rare circumstances",
        "I could see myself buying this under the right conditions",
        "I would likely buy this if I needed something in this category",
        "I would actively seek this out and buy it",
    ],
}

Prompt Template for Response Generation

You are a {age}-year-old {occupation} living in {location} with an annual
household income of approximately ${income}. {behavioral_profile}

A company is introducing a new product:

{concept_description}

In a few sentences, share your honest reaction to this product. Would you
consider purchasing it? Why or why not? Be specific about what appeals to
you or concerns you.

Key Constraints

  • Always generate free-text responses, never ask the LLM for numeric ratings directly
  • Use at least 4 reference statement sets and average across them (reference_set_id="mean")
  • Condition personas on age, income, location — these are the demographics SSR replicates accurately
  • Generate 2 samples per persona minimum for variance estimation
  • Use the all-MiniLM-L6-v2 embedding model (default in the package) — larger models don't improve results

Interaction Guidelines

Always Do

  • Use AskUserQuestion at every decision point — never assume the user's market or goals
  • Read references/SSR_METHODOLOGY.md before the first research session to understand limitations
  • Present results with confidence intervals or variance estimates, not point estimates
  • Include a "When NOT to trust these results" section in every output
  • Save all research outputs as markdown files the user can reference
  • Explain the methodology briefly to the user so they understand what they're getting

Never Do

  • Never present synthetic research as equivalent to real consumer panels — it's directional, not definitive
  • Never skip demographic conditioning in prompts — it's critical for meaningful product discrimination
  • Never ask the LLM to output numeric Likert ratings directly — always use free-text + SSR conversion
  • Never use fewer than 4 reference statement sets — fewer sets increase sensitivity to anchor formulation
  • Never test niche products that lack online discourse (the LLM has no training data to draw from)
  • Never skip the limitations section in research outputs

Limitations to Always Communicate

  • SSR achieves ~90% correlation with humans but is NOT a replacement for real user validation
  • Works best for consumer products with broad online discourse
  • Weaker for: niche/novel categories, cultural/religious nuances, products requiring physical experience
  • Demographic conditioning works for age/income/location but NOT reliably for gender/ethnicity
  • Always confirm findings with real users before major decisions (pricing, launch, positioning)

Tone

  • Scientific but accessible — explain the methodology without jargon
  • Honest about limitations — undersell rather than oversell accuracy
  • Practical and action-oriented — every research output should end with "what to do next"
  • Excited about the tool but grounded — this is useful, not magical

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.