TimeCapsuleLLM
llms.txt2k starsChanged yesterdayMITView on GitHub
A research project for training language models from scratch on carefully bounded historical corpora. The primary current focus is English-language material associated with London, published from 1800 to 1875. Fine-tuning or LoRA adapts a model that already contains modern-language knowledge. TimeCapsuleLLM instead starts from random initialization and trains only on the selected corpus. The objective is not to make a modern model perform a historical style; it is to study a model whose learned language and knowledge come from the…
The file
We split this llms.txt into the rules it gives an agent
4 rules. About 0 are common, 3 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 11
Not intended use: Reliable factual question answering, historical authority, or production-grade text generation.
- Distinctive: line 60
Do not treat generated text as evidence about historical events or people without independently verifying it against sources.
- Very specific: line 64
Use the citation in CITATION.cff or the README when referring to the dataset or models.
- Distinctive: line 68
Read README.md for examples, model-behavior notes, detailed FAQ answers, related work, and acknowledgements.
≈ “Common” is an estimate: at least 1 in 5 of the 3,727 llms.txt files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
# TimeCapsuleLLM > A research project for training language models from scratch on carefully bounded historical corpora. The primary current focus is English-language material associated with London, published from 1800 to 1875. ## Project summary - Name: TimeCapsuleLLM / TimeCapsule LLM - License: MIT (see `LICENSE`) - Core idea: Selective Temporal Training (STT). Training data is restricted to a specified historical place and time window so the model is not pre-trained on modern text and then made to imitate the past. - Intended use: Research into historical language modeling, corpus curation, and period-bounded generative models. - Not intended use: Reliable factual question answering, historical authority, or production-grade text generation. Outputs can hallucinate, reproduce OCR artifacts, and be incoherent. - Model collection: https://huggingface.co/collections/haykgrigorian/timecapsulellm-1800-1875-london ## Why train from scratch? Fine-tuning or LoRA adapts a model that already contains modern-language knowledge. TimeCapsuleLLM instead starts from random initialization and trains only on the selected corpus. The objective is not to make a modern model perform a historical style; it is to study a model whose learned language and knowledge come from the bounded corpus. ## Current corpus and models The documented corpus focus is London, 1800-1875. It contains public-domain books, legal documents, newspapers, and other writings. Corpus coverage, data quality, and exact composition vary by release. | Release | Parameters | Corpus / training note | | --- | ---: | --- | | v0 | 16M | About 187 MB; early nanoGPT-based experiment. | | v0.5-london | 123M | About 435 MB; improved grammar, but still hallucinates and may reproduce OCR noise. | | v1-london | 700M | About 6.25 GB. | | v2mini-eval1-london | 300M | 15 GB sample of the v2 corpus; trained for 10K steps; known tokenization issue. | | v2mini-eval2-london | 200M | Same 15 GB sample; trained for 10K steps. | | v2-london | 1.2B | 90 GB raw corpus / 112 GB tokenized; 136,344 documents; trained for 182K steps. | ## Repository map - `experiments/`: versioned experiment material, grouped first by corpus and then model version. - `experiments/london-1800-1850/v0/`: v0 data-preparation, tokenizer-training, training, and sampling scripts. - `experiments/london-1800-1875/v0.5/`: v0.5 scripts, custom tokenizer artifacts, and example outputs. - `experiments/london-1800-1875/v2/`: v2 sample outputs and launch script. - `experiments/london-1800-1875/v2mini-eval1/`: v2 evaluation scripts, bias report, and charts. - `scripts/`: reusable acquisition and dataset utilities. - `data/manifests/`: small, versioned source-document and Internet Archive ID lists; not the raw corpus. - `assets/`: brand assets and documentation images. - `docs/`: repository-maintenance documentation, including `REPOSITORY_STRUCTURE.md`. - `CITATION.cff`: citation metadata. - `CONTRIBUTING.md`: contribution guidance. ## Reproducing the approach 1. Choose a defensible historical time and geographic window. 2. Collect public-domain plain-text sources that fall inside that window. 3. Clean boilerplate, modern annotations, and OCR errors where possible. 4. Train a tokenizer on the cleaned corpus (examples: `train_tokenizer.py` and `train_tokenizer_hf.py` under `experiments/london-1800-1850/v0/`). 5. Train a causal language model from scratch using the tokenizer and corpus. This repository does not aim to be a general end-to-end training framework; consult the relevant model implementation or nanoGPT for architecture and training details. 6. Evaluate outputs for fluency, period leakage, factual hallucination, OCR contamination, geographic bias, and temporal bias. ## Important limitations - A corpus cutoff reduces exposure to later material; it does not make outputs historically correct or neutral. - Historical sources can contain social biases, unequal representation, and errors from their era. - OCR artifacts can become learned output patterns. - Small or partially trained checkpoints may be grammatically weak or incoherent. - Do not treat generated text as evidence about historical events or people without independently verifying it against sources. ## Citation and research context Use the citation in `CITATION.cff` or the README when referring to the dataset or models. The project was initiated independently and is conducted under academic supervision, with affiliated research collaboration at Muhlenberg College and Georgia State University. ## Human-readable documentation Read `README.md` for examples, model-behavior notes, detailed FAQ answers, related work, and acknowledgements.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

