File Organization & Standardization Runbook

A fully generalized, context-aware runbook for auditing, deduplicating, categorizing, and standardizing personal, family, and institutional file archives.

1. Universal Naming Formulas

All file names are generated using consistent, predictable formulas derived from internal document content, not download, print, or scan artifacts.

A. General & Identity Files (Default: Kebab-Case)

[<date>-][<issuer>-]<document-type>[-<qualifier>][-<status>].<ext>

Generalized Category Patterns:

• Identity & Legal:
  • [<issuer>-]<type>[-<variant>][-<status>].<ext> → pan.pdf, pan-attested.pdf, aadhaar-pvc.pdf, voter-id.pdf
  • passport[-<year>].pdf → passport.pdf, passport-2024.pdf, photo-passport-size.jpg
  • driving-license[-<variant>].pdf → driving-license.pdf, driving-license-learners.pdf
• Academic & Career:
  • <level>-marksheet.pdf → 10th-marksheet.pdf, 12th-marksheet.pdf, degree-transcript.pdf
  • <type>-certificate.pdf → degree-certificate.pdf, birth-certificate.pdf, provisional-certificate.pdf
  • <document-type>.pdf → resume.pdf, offer-letter.pdf, relieving-letter.pdf
• Banking, Financial & Tax:
  • statement-<period>.pdf → statement-2023-q4.pdf, statement-2024-03.pdf
  • passbook[-<issuer>].pdf → passbook-sbi.pdf, passbook.pdf
  • receipt-<type>[-<year>].pdf → property-tax-2023.pdf, fd-receipt.pdf
  • tax-return-<period>.pdf → itr-acknowledgement-ay-2023-24.pdf

B. Medical & Clinical Documents (Mandatory Exception)

To preserve clinical chronological records across multiple doctors, hospitals, and labs, medical files use a strict human-readable pattern:

YYYY-MM-DD - <Patient Name> - <Record Type> - <Provider or Test Name>.pdf

Event Date Rule: Always use the date the medical encounter or test took place, never the scan date, download date, or report printing date.

2. Generalization & Deduplication Principles

A. Path-Context Deduplication (Eliminate Redundancy)

Never repeat information that is already unambiguous from the folder path:

1. Never repeat person/owner names: Inside any personal or individual directory (e.g., <Workspace>/Identity/Passport/), use passport-2024.pdf, NEVER <Person Name> Passport 2024.pdf.
2. Never repeat parent folder tokens: Inside <Workspace>/Identity/Aadhaar/, use aadhaar.pdf or pvc.pdf, NEVER aadhaar-aadhaar.pdf. Inside <Workspace>/Identity/PAN/, use pan.pdf, NEVER pan-card.pdf.

B. Content-First Ground Truth

Never trust existing file names or system-assigned metadata:

1. Inspect internal text via CLI tools (pdftotext, strings) or render the first page to image (pdftoppm) for OCR inspection.
2. Verify:
   • True subject / owner of the document.
   • Issuing organization or provider.
   • Actual transaction, exam, or encounter date.
   • Document classification.

C. Normalization & Hygiene

• No spaces in non-medical filenames: Always use kebab-case (lowercase-with-hyphens).
• Sanitize corruptions: Strip duplicate extensions (.pdf.pdf, .jpg.pdf, .png.pdf).
• Remove trailing spaces: Fix spaces before extensions (name .pdf → name.pdf).
• Correct spelling errors: Normalize typos (Propety → property, Finacials → financials).

3. Directory Layout Archetypes

A. Individual Workspace (<Root>/<Individual-Identifier>/)

<Individual-Root>/
├── Career/                 # Resumes, offers, relieving letters, contracts
├── Certificates/           # Academic transcripts, degrees, diplomas, awards
├── Financial/
│   ├── Banking/            # Statements, passbooks, check records
│   ├── Investments/        # Fixed deposits, mutual funds, equity portfolios
│   ├── Property-Tax/       # Municipal tax receipts, property assessments
│   └── Taxes/              # Annual tax returns (ITR, Form 16, AIS, 26AS)
├── Identity/
│   ├── Aadhaar/            # Regular, PVC, e-Aadhaar
│   ├── Driving-License/    # Permanent DL, learner's license
│   ├── PAN/                # PAN card copies, e-PAN
│   └── Passport/           # Passports, visa stamp copies
├── Medical/                # Clinical records, chronologically organized
│   ├── Lab-Reports/
│   ├── Prescriptions/
│   └── Vaccines/
├── Vehicles/               # Registration certificates (RC), insurance, emission
└── Travel/                 # Visas, itineraries, tickets

B. Shared / Group Archives (<Root>/Shared/ or <Root>/Family/)

Store records that naturally encompass multiple individuals in shared domain roots:

• Shared/Insurance/: Multi-member floater health policies, umbrella policies.
• Shared/Property/: Joint title deeds, sale agreements, floor plans, mutations.
• Shared/Reference/: General manuals, health guides, emergency contacts.

4. Deduplication Protocol

1. Size Grouping: Index files by exact byte size to quickly isolate candidate collisions.
2. Binary Hashing: Compute MD5 / SHA-256 checksums (md5 -r <file>).

   Cloud Storage / FileProvider Note: When files are stored in on-demand cloud storage (e.g. Google Drive, iCloud, OneDrive), use native shell commands (md5, cmp) to trigger transparent on-demand hydration.

3. Collision Resolution:
   • Prune download duplicates (e.g., (1), (2), -copy).
   • Prune OS prefixes (e.g., Copy of ..., DUPLICATE_...).
   • Prune zero-byte and corrupted remnants.
   • Retain the cleanest, most descriptive filename and remove identical binary duplicates.

5. Encrypted PDF Handling & Password Removal

Personal archives frequently contain password-protected PDFs (e-Aadhaar, bank statements, salary slips, Form 16, insurance policies). Encrypted PDFs cannot be inspected via pdftotext or indexed.

A. Security Requirements

1. Never hardcode, store, or log passwords: Passwords must never appear in transcripts, commit messages, or plaintext log files.
2. Hidden input: Always acquire passwords interactively via terminal masked prompts (getpass) so characters are not echoed to stdout.
3. Session reuse: Within a single execution session, cache verified passwords in memory to unlock other files using the same credentials (e.g. repeated DOB or PAN) without repeated prompts.

B. Bundled Utility: scripts/unlock_pdf.py

The skill provides a turnkey decryption utility:

# 1. Prerequisite (fastest engine):
# macOS: brew install qpdf
# Linux: sudo apt install qpdf
# Fallback: pip install pypdf

# 2. Scan for locked PDFs without prompting (dry run):
python3 scripts/unlock_pdf.py /path/to/archive --dry-run

# 3. Securely unlock in-place (prompts via getpass; caches session password):
python3 scripts/unlock_pdf.py /path/to/archive --in-place

6. Execution Workflow

[Scan & Index Directory Tree]
        │
        ▼
[Detect Encrypted PDFs ──▶ Run unlock_pdf.py (Secure getpass Prompt)]
        │
        ▼
[Inspect Content via pdftotext / Page Render]
        │
        ▼
[Resolve True Owner & Route to Individual Workspace]
        │
        ▼
[Deduplicate via Exact Binary Hash]
        │
        ▼
[Apply Universal Naming Formula & Strip Redundancy]
        │
        ▼
[Relocate into Domain Archetype Hierarchy]