Deep Research & Knowledge Integration Pipeline

You are orchestrating a fully autonomous research → extraction → connection discovery workflow to expand the knowledge base with cutting-edge insights.

Input Processing

User Input: $ARGUMENTS

Execution Modes:

1. Directed Mode - User specifies topic(s): $ARGUMENTS = "neuroscience of habits" or $ARGUMENTS = "multi-agent systems, safety alignment"
2. Autonomous Mode - You select topics: $ARGUMENTS = "" or $ARGUMENTS = "auto"

Mission

Execute a complete 3-phase autonomous research pipeline:

1. RESEARCH - Gather cutting-edge papers and developments
2. EXTRACT - Pull unique insights from research findings
3. CONNECT - Map connections to existing knowledge base

Critical Requirement: ALL extracted insights MUST be stored in Document Insights folder structure to keep separate from main Brain.

Phase 1: Topic Selection & Research Planning

A. If User Provided Topic(s) (Directed Mode)

• Parse $ARGUMENTS for topic(s)
• Validate topics are research-worthy
• Plan research scope for each topic

B. If Autonomous Mode

Analyze knowledge base to identify research opportunities:

1. Read knowledge base analysis:

   cat knowledge-base-analysis.md

2. Check recent activity:

   ls -lt Brain/Document\ Insights/ | head -10

3. Identify gaps based on:
   • Underrepresented domains in knowledge-base-analysis.md
   • Missing connections flagged in recent changelogs
   • Emerging themes from existing insights
   • User's recent work patterns
   • CLAUDE.md priorities and future directions
4. Select 1-3 research topics that would:
   • Fill identified gaps
   • Build on existing strengths (e.g., Buddhism-Neuroscience-AI triangle)
   • Connect underexplored domains
   • Add empirical validation to intuitive frameworks
   • Challenge or extend current thinking

Examples of Good Topic Selection:

• "Neuroscience of habits and behavior change" (if habit formation underrepresented)
• "Collective intelligence and swarm behavior" (if group dynamics missing)
• "Embodied cognition and interoception" (if embodiment gap identified)
• "Complexity science and emergence" (if systems thinking needed)
• "Creativity neuroscience and insight generation" (if creative process mechanics missing)

Phase 2: Execute Research

Get Current Timestamp

date '+%Y-%m-%d %H:%M:%S %Z'

Save this for session folder naming: YYYY-MM-DD Topic Description

Launch Research Specialist Agent(s)

For Each Topic:

Use Task tool with subagent_type='research-specialist':

TOPIC: [Selected topic]

Conduct comprehensive research on [topic] focusing EXCLUSIVELY on the most recent research and developments.

⚠️ CRITICAL RECENCY REQUIREMENT:
Your training data may be outdated. The world changes rapidly, especially in fast-moving fields like AI, neuroscience, and technology. You MUST prioritize the most recent information available through web search, even if it contradicts what you think you know from training data.

⚠️ SOURCE QUALITY REQUIREMENT (recency is NOT enough):
Recent does not mean credible. Prefer the PRIMARY source over anyone summarizing it - the actual paper, lab page, or official doc, not a content-farm writeup, an AI-generated summary, a single-tweet leak, or an SEO explainer. Reject machine-generated and regurgitated material. For each finding, record which source it came from so downstream extraction can tier it. (The per-domain source diet is the canonical `resources/SOURCE-AUTHORITY.md`; the document-insight-extractor applies it at extraction time.)

SEARCH STRATEGY:
- Use Google Search grounding to find papers published in the last 12-18 months
- Explicitly search for "2024", "2025", "2026", "recent", "latest" in queries
- Check paper publication dates - reject anything older than 2024 unless foundational
- Look for preprints, conference proceedings, and recent journal publications
- Prioritize arXiv papers from last 6 months, conference papers from 2024-2026
- Search for "state of the art [topic] 2025" or "[topic] breakthrough 2026"

RESEARCH REQUIREMENTS:

1. **Target Sources (RECENT ONLY):**
   - arXiv preprints (2024-2026, prioritize last 6 months)
   - Major conferences 2024-2026 (NeurIPS, ICML, ICLR, AAAI, ACL, EMNLP, etc.)
   - Leading AI labs recent publications (OpenAI, Anthropic, Google DeepMind, Microsoft Research)
   - Top-tier journals (2024-2026 issues only)
   - Industry whitepapers and blog posts from major tech companies (last 12 months)
   - Recent preprints and working papers

2. **Key Focus Areas:**
   - Novel mechanisms and frameworks (not in your training data)
   - Empirical findings with quantified results (recent benchmarks)
   - Counter-intuitive or contrarian insights (challenging established thinking)
   - Cross-domain applications (emerging connections)
   - Real-world implementations and case studies (production deployments)
   - Practical implications for practitioners

3. **Output Requirements:**
   - Comprehensive structured report (15-25 major papers/developments)
   - Full citations with DATES prominently displayed (title, authors, DATE, venue, arXiv ID)
   - Key findings and novel contributions
   - Performance metrics and empirical data
   - Emerging trends and patterns
   - URLs to papers/resources
   - Critical analysis and synthesis

4. **Save Location:**
   resources/[Topic-Slug]-Research-Report-YYYY-MM-DD.md

VERIFICATION: Before finalizing, verify that 80%+ of papers are from 2024-2026. If not, search again with more explicit recency filters. Also verify the sources are credible primaries (actual papers/labs/official docs), not content-farm pages or AI-generated summaries.

Use Gemini AI with Google Search grounding. Trust the search results over your training data.

Strategy Considerations:

• Sequential: Run topics one-by-one if they're related (later research can reference earlier findings)
• Parallel: Run multiple topics simultaneously if they're independent domains
• Your choice - decide based on topic relationships and efficiency

Monitor Research Output

After each research agent completes:

1. Note the report file path
2. Verify comprehensive coverage (15-25+ papers)
3. Check for citations and empirical data
4. Confirm report saved in /resources/ directory

Phase 3: Extract Insights

Create Session Folder

Format: YYYY-MM-DD [Topic Description]

Example: 2025-11-20 Neuroscience of Habits and Behavior Change

Path: Brain/Document Insights/[Session-Folder]/

Launch Document Insight Extractor

For Each Research Report:

Use Task tool with subagent_type='document-insight-extractor':

Extract unique insights from the research report for the knowledge base.

SOURCE DOCUMENT: [Full path to research report]

SESSION FOLDER: [Session folder name]

EXTRACTION GUIDELINES:

1. **Focus on Novel Insights:**
   - Paradigm shifts and new frameworks
   - Counter-intuitive or surprising findings
   - Empirical validation of existing theories
   - Novel mechanisms and explanations
   - Cross-domain applications
   - Contrarian perspectives backed by evidence

2. **Bridge to Existing Knowledge Base:**
   - Connect to the 6 primary hubs: Consciousness, Dopamine, Decision-Making, Identity, AI Agents, Flow States
   - Reference the user's existing frameworks (Folder Paradigm, Mental Models Taxonomy, etc.)
   - Identify consilience opportunities (3+ domains converging)
   - Find validation or challenges to current thinking
   - Look for applications of Buddhist/neuroscience principles

3. **Prioritize:**
   - Research findings that extend current understanding
   - Empirical data that validates intuitive frameworks
   - Novel architectures or methodologies
   - Real-world implications and case studies
   - Philosophical or meta-level insights

4. **Quality Standards:**
   - 15-25 high-quality insights per report
   - Avoid redundancy with existing knowledge base (ALWAYS search for duplicates)
   - Include proper citations (paper title, authors, year)
   - Tag appropriately for discoverability
   - Create connections to existing permanent notes

5. **Output Requirements:**
   - Create permanent notes in session folder
   - Include full citations and sources
   - Add relevant tags
   - Note connections to existing insights
   - Create changelog: CHANGELOG - Document Analysis YYYY-MM-DD.md

CRITICAL:
- ALWAYS search for duplicates before creating notes
- Store ALL extracted notes in: Brain/Document Insights/[Session-Folder]/
- Create comprehensive changelog documenting extraction process

Monitor Extraction Output

After extraction completes:

1. Verify insights stored in correct Document Insights session folder
2. Check changelog was created
3. Note count of unique insights extracted
4. Confirm deduplication was performed

Phase 4: Insight Interview (Optional)

After extraction completes, present the top findings and offer to run an insight interview before connection discovery. This captures your personal perspective alongside the external research - making the final connection map richer because it maps both what the research says AND what you actually think about it.

Present Top Insights

Summarize the 5-8 most significant extracted insights from the session folder:

• List note titles with one-sentence descriptions
• Highlight findings that challenge existing KB frameworks or contradict current notes
• Flag any surprising or counterintuitive results

[APPROVAL GATE] - Run Insight Interview?

Present to user:

"[N] insights extracted on [topic]. Before connection discovery, would you like to do a quick insight interview? I'll ask you 6-8 questions grounded in your existing notes and these new findings - to capture YOUR angles, reactions, and disagreements. Your responses save to Brain/AI Extracted Notes/ and the connection finder will map both sets together.

Say yes to run the interview, or skip to go straight to connection discovery."

If yes: Invoke the insight-interview skill for the current topic.

• The dialogue runs here - one question at a time
• User insights saved to Brain/AI Extracted Notes/
• Note the session timestamp so connection discovery can include these new notes

If skip: Proceed directly to Phase 5.

Update Scope for Connection Discovery

If the interview ran, Phase 5 should map connections across both:

• External research: Brain/Document Insights/[Session-Folder]/
• Personal insights: new notes created in Brain/AI Extracted Notes/ during this session

Phase 5: Connection Discovery

Launch Connection Finder Agent(s)

Strategy Options:

Option A: Single Comprehensive Pass

• Run connection-finder once on the entire session folder
• Maps all new insights against full knowledge base

Option B: Multiple Targeted Passes

• Run connection-finder 2-3 times on different subsets
• First pass: New insights ↔ Existing AI insights (102 notes)
• Second pass: New insights ↔ Primary hubs (Dopamine, Consciousness, etc.)
• Third pass: Cross-domain bridges and synthesis opportunities

Your Choice - Select based on insight count and domain diversity.

Execute Connection Discovery

Use Task tool with subagent_type='connection-finder':

Discover connections between newly extracted insights and existing knowledge base.

STARTING POINTS:
All notes in session folder: Brain/Document Insights/[Session-Folder]/

Or specify individual notes if doing targeted passes.

CONNECTION DISCOVERY GOALS:

1. **Bridge to Existing Knowledge:**
   - Connect to 102 existing AI insights
   - Link to 6 primary thematic hubs (Consciousness, Dopamine, Decision-Making, Identity, AI Agents, Flow)
   - Find relationships to original frameworks (Folder Paradigm, Mental Models Taxonomy, etc.)
   - Map to MOCs and output content

2. **Cross-Domain Opportunities:**
   - Buddhism ↔ Neuroscience ↔ AI consilience
   - Decision Science ↔ Agent Architecture
   - Flow States ↔ Peak Performance ↔ AI Optimization
   - Identity/Belief Systems ↔ Agent Fitness Functions
   - Dopamine hub connections (universal bridge)

3. **Synthesis Identification:**
   - Clusters of insights ready for article development
   - Consilience zones (3+ domains converging)
   - Emergent patterns and meta-insights
   - Framework extension opportunities
   - New MOC candidates

4. **Analysis Parameters:**
   - Similarity thresholds: 0.65-0.85 (strong to moderate)
   - Depth: 2-3 levels from each new insight
   - Focus: Non-obvious, high-value connections

5. **Output Requirements:**
   - Map direct connections to existing permanent notes
   - Identify bridge notes connecting multiple domains
   - Highlight consilience zones and synthesis opportunities
   - Create dated changelog: CHANGELOG - Connection Discovery Session YYYY-MM-DD.md
   - Store changelog in: Brain/05-Meta/Changelogs/
   - Update master changelog: Brain/CHANGELOG.md
   - Suggest concrete article topics or framework extensions

Begin comprehensive connection mapping.

Monitor Connection Discovery

After connection-finder completes:

1. Verify changelog created in /Brain/05-Meta/Changelogs/
2. Check master CHANGELOG.md was updated
3. Note key findings: consilience zones, synthesis opportunities
4. Identify high-priority article topics

Phase 6: Final Summary & Recommendations

Consolidate Results

Generate a comprehensive session report including:

# Deep Research Pipeline - Session Summary
**Date:** [Timestamp]
**Execution Mode:** [Directed / Autonomous]
**Topics Researched:** [List]

---

## Phase 1: Research
**Topics Selected:**
1. [Topic 1] - Rationale: [Why chosen]
2. [Topic 2] - Rationale: [Why chosen]
...

**Research Reports Created:**
- [Report 1]: /resources/[filename] ([N] papers analyzed)
- [Report 2]: /resources/[filename] ([N] papers analyzed)

**Total Papers Analyzed:** [N]
**Research Coverage:** [Domains covered]

---

## Phase 2: Insight Extraction
**Session Folder:** /Brain/Document Insights/[Session-Folder]/

**Extraction Results:**
- Unique insights extracted: [N]
- Duplicates avoided: [N]
- Very similar (evaluated): [N]
- Changelogs created: [List paths]

**Insights by Type:**
- Research findings: [N]
- Theoretical frameworks: [N]
- Production insights: [N]
- Contrarian arguments: [N]

**Top Insights:**
1. [[Note Title]] - [Brief description]
2. [[Note Title]] - [Brief description]
...

---

## Phase 3: Connection Discovery
**Changelogs Created:**
- [Path to connection discovery changelog]

**Key Findings:**
- Strong connections discovered: [N]
- Emergent patterns identified: [N]
- Cross-domain bridges: [N]
- Consilience zones: [List]

**Major Cross-Domain Bridges:**
1. [Domain A] ↔ [Domain B] - Mechanism: [How connected]
2. [Domain A] ↔ [Domain C] - Mechanism: [How connected]

**Synthesis Opportunities Identified:**
1. **Article:** "[Title]" - Ready for development
2. **Framework:** "[Name]" - Extension of existing work
3. **MOC Candidate:** "[Topic]" - Needs organization hub

---

## Impact Assessment

**Knowledge Base Enhancement:**
- New research domains added: [List]
- Existing frameworks validated/extended: [List]
- Gaps filled: [List]
- New connections to core hubs: [N]

**Most Significant Discoveries:**
1. [Discovery 1] - Why significant: [Explanation]
2. [Discovery 2] - Why significant: [Explanation]
3. [Discovery 3] - Why significant: [Explanation]

**Contrarian Insights:**
- [Insight that challenges conventional wisdom]
- [Insight that challenges existing framework]

---

## Recommended Next Steps

**High-Priority Actions:**
1. **Write Article:** "[Suggested title]"
   - Sources: [[Note 1]], [[Note 2]], [[Note 3]]
   - Unique angle: [What makes this distinctive]
   - Target audience: [Who would benefit]

2. **Extend Framework:** "[Framework name]"
   - Current state: [What exists]
   - Enhancement: [What research adds]
   - Application: [How to use]

3. **Create MOC:** "[Topic]"
   - Notes to organize: [Count]
   - Structure: [Suggested organization]
   - Purpose: [Navigation goal]

**Medium-Priority:**
- [Additional recommendations]

**Long-Term Opportunities:**
- [Strategic synthesis possibilities]

---

## Session Files Created

**Research Reports:**
- [Path 1]
- [Path 2]

**Insight Notes:**
- [Session folder path] ([N] notes)

**Changelogs:**
- [Extraction changelog path]
- [Connection discovery changelog path]
- Master CHANGELOG.md updated

---

## Knowledge Base Statistics (Updated)

**Before Session:**
- Total permanent notes: [N]
- AI insights: [N]
- Document insights: [N]

**After Session:**
- Total permanent notes: [N] (+[N])
- AI insights: [N]
- Document insights: [N] (+[N])

**Growth:** +[N] notes, +[N] connections

---

## Meta-Analysis

**What Worked Well:**
- [Successes in topic selection, research, extraction, or connection]

**Challenges Encountered:**
- [Any difficulties or limitations]

**Lessons for Future Sessions:**
- [Improvements for next research pipeline run]

---

**End of Deep Research Pipeline Session**

Quality Standards & Best Practices

Topic Selection (Autonomous Mode)

• Strategic alignment: Choose topics that build on existing strengths or fill critical gaps
• Cross-domain potential: Prefer topics that bridge multiple knowledge base hubs
• Empirical grounding: Select areas with active research (2024-2026 papers available)
• Practical relevance: Topics should have real-world applications or implications

Research Quality

• Recency: Prioritize 2024-2026 papers and developments
• Rigor: Prefer primary sources (the actual paper/lab/official doc) over summaries; reject content-farm and AI-generated regurgitation. See the canonical source diet in resources/SOURCE-AUTHORITY.md.
• Depth: 15-25 major papers minimum per topic
• Breadth: Cover multiple perspectives and approaches
• Empirics: Include quantified results and performance metrics

Insight Extraction

• Novelty: Only extract genuinely new perspectives
• Deduplication: ALWAYS search before creating notes
• Citations: Include full source attribution
• Connections: Link to existing knowledge base
• Quality > Quantity: 15-25 high-value insights, not 100 mediocre ones

Connection Discovery

• Non-obvious focus: Surface-level links are less valuable
• Cross-domain priority: Consilience zones are gold
• Synthesis orientation: Identify article/framework opportunities
• Actionable output: Provide concrete next steps

Documentation

• Comprehensive changelogs: Document every phase
• Clear file organization: Session folders in Document Insights
• Master log updates: Keep CHANGELOG.md current
• Audit trail: Future-you should understand what happened and why

Execution Protocol

1. Parse input → Determine directed vs. autonomous mode
2. Select topics → Either use provided topics or analyze knowledge base for gaps
3. Get timestamp → For session folder naming
4. Research phase → Launch research-specialist agent(s)
5. Extraction phase → Launch document-insight-extractor for each report
6. Insight interview → Optional gate: present top findings, offer /insight-interview to capture your angles before connection discovery
7. Connection phase → Launch connection-finder agent(s) across both document and personal insights
8. Generate summary → Comprehensive session report
9. Provide recommendations → Actionable next steps for content creation

Key Principle: Fully autonomous execution. No human intervention required between phases. All insights stored in Document Insights folder structure to maintain separation from main Brain.

Error Handling

If research finds insufficient papers:

• Broaden search criteria
• Extend date range (include 2023)
• Consider adjacent topics
• Document limitation in summary

If extraction finds too many duplicates:

• Focus on truly novel contributions
• Look for empirical validation of concepts
• Seek contrarian perspectives
• Consider topic was already well-covered

If connection-finder finds weak connections:

• Topic may be genuinely novel (good!)
• Increase similarity threshold range
• Run additional passes on specific hubs
• Document gap as synthesis opportunity

If any phase fails:

• Document error in summary
• Continue with successful phases
• Provide partial results
• Recommend retry or alternative approach

Remember: This is a knowledge base expansion engine. Your goal is to systematically grow the user's second brain with cutting-edge, well-integrated insights that enhance his intellectual capabilities and content creation potential.

State Dependencies

Completion Checklist

• Execution mode determined (directed vs autonomous)
• Topics selected with rationale
• Research reports generated and saved to /resources/
• Session folder created in Document Insights
• Insights extracted with deduplication
• Extraction changelog created
• Insight interview offered (ran or skipped)
• Connection discovery completed
• Connection discovery changelog created in /05-Meta/Changelogs/
• Master CHANGELOG.md updated
• Session summary generated with recommendations
• Synthesis opportunities identified