maths-cs-ai-compendium

# Maths, CS & AI Compendium

> An open, intuition-first textbook covering mathematics, computer science, and artificial intelligence from the ground up. Written for curious practitioners, not exam survivors.

## About
- Author: Henry Ndubuaku
- Repository: https://github.com/HenryNdubuaku/maths-cs-ai-compendium
- Website: https://henryndubuaku.github.io/maths-cs-ai-compendium/
- License: Open source educational resource
- Audience: AI/ML engineers, researchers, and students who want deep understanding

## Chapters

### Chapter 1: Vectors
- [Vector Spaces](chapter 01: vectors/01. vector spaces.md): Vector spaces, dimensions, subspaces, closure properties, and their role as the foundation of ML
- [Vector Properties](chapter 01: vectors/02. vector properties.md): Magnitude, direction, parallelism, orthogonality, linear independence, sparsity, unit vectors
- [Norms and Metrics](chapter 01: vectors/03. norms and metrics.md): L1, L2, Lp, infinity norms, distance metrics, cosine similarity
- [Products](chapter 01: vectors/04. products.md): Dot product, cross product, outer product, triple products, cosine similarity
- [Basis and Duality](chapter 01: vectors/05. basis and duality.md): Basis vectors, change of basis, dual spaces, covectors

### Chapter 2: Matrices
- [Matrix Properties](chapter 02: matrices/01. matrix properties.md): Transpose, trace, rank, determinant, inverse, condition number, norms, positive definiteness
- [Matrix Types](chapter 02: matrices/02. matrix types.md): Identity, diagonal, symmetric, orthogonal, sparse, Toeplitz, circulant, Hermitian, permutation
- [Operations](chapter 02: matrices/03. operations.md): Matrix multiplication, Hadamard product, outer product, systems of equations, pseudo-inverse
- [Linear Transformations](chapter 02: matrices/04. linear transformations.md): Rotation, reflection, scaling, shearing, affine transforms, homogeneous coordinates
- [Decompositions](chapter 02: matrices/05. decompositions.md): Gaussian elimination, LU, Cholesky, eigendecomposition, QR, SVD, PCA, NMF

### Chapter 3: Calculus
- [Differential Calculus](chapter 03: calculus/01. differential calculus.md): Limits, derivatives, chain rule, differentiation rules, L'Hopital's rule
- [Integral Calculus](chapter 03: calculus/02. integral calculus.md): Definite and indefinite integrals, fundamental theorem, u-substitution, integration by parts
- [Multivariate Calculus](chapter 03: calculus/03. multivariate calculus.md): Partial derivatives, gradients, Jacobians, Hessians, directional derivatives
- [Function Approximation](chapter 03: calculus/04. function approximation.md): Taylor series, Maclaurin series, Fourier series, polynomial approximation
- [Optimisation](chapter 03: calculus/05. optimisation.md): Critical points, convexity, Newton's method, gradient descent, Lagrange multipliers, KKT

### Chapter 4: Statistics
- [Fundamentals](chapter 04: statistics/01. fundamentals.md): Random variables, distributions, expectation, moments, variance
- [Measures](chapter 04: statistics/02. measures.md): Dispersion, quartiles, z-scores, correlation, outlier detection
- [Sampling](chapter 04: statistics/03. sampling.md): Sampling methods, central limit theorem, bootstrap
- [Hypothesis Testing](chapter 04: statistics/04. hypothesis testing.md): t-tests, p-values, confidence intervals, ANOVA
- [Inference](chapter 04: statistics/05. inference.md): Statistical inference, MLE, MAP estimation

### Chapter 5: Probability
- [Counting](chapter 05: probability/01. counting.md): Permutations, combinations, factorial, multinomial, inclusion-exclusion
- [Probability Concepts](chapter 05: probability/02. probability concepts.md): Axioms, conditional probability, Bayes theorem, independence
- [Distributions](chapter 05: probability/03. distributions.md): Bernoulli, binomial, Poisson, Gaussian, exponential, beta, Dirichlet
- [Bayesian Methods](chapter 05: probability/04. bayesian.md): Prior, posterior, likelihood, conjugate priors, MCMC
- [Information Theory](chapter 05: probability/05. information theory.md): Entropy, cross-entropy, KL divergence, mutual information

### Chapter 6: Machine Learning
- [Classical ML](chapter 06: machine learning/01. classical machine learning.md): Naive Bayes, decision trees, random forests, SVM, K-means, GMM
- [Gradient ML](chapter 06: machine learning/02. gradient machine learning.md): Linear/logistic regression, SGD, Adam, loss functions, regularisation, evaluation
- [Deep Learning](chapter 06: machine learning/03. deep learning.md): MLPs, CNNs, RNNs, LSTMs, attention, transformers, ViT, autoencoders, VAEs
- [Reinforcement Learning](chapter 06: machine learning/04. reinforcement learning.md): MDPs, Q-learning, policy gradient, actor-critic, PPO, RLHF, DPO
- [Distributed Training](chapter 06: machine learning/05. distributed deep learning.md): Data/model/pipeline/tensor parallelism, mixed precision, scaling laws, MoE

### Chapter 7: Computational Linguistics
- [Linguistic Foundations](chapter 07: computational linguistics/01. linguistic foundations.md): Morphology, syntax, semantics, pragmatics, phonology
- [Text Processing](chapter 07: computational linguistics/02. text processing and classic NLP.md): Tokenisation, TF-IDF, n-grams, NER, POS tagging
- [Embeddings and Sequence Models](chapter 07: computational linguistics/03. embeddings and sequence models.md): Word2Vec, GloVe, RNNs, LSTMs, seq2seq
- [Transformers and LMs](chapter 07: computational linguistics/04. transformers and language models.md): Self-attention, BERT, GPT, T5, positional encoding
- [Advanced Text Generation](chapter 07: computational linguistics/05. advanced text generation.md): MoE, SSMs, Mamba, GQA, MLA, modern LLM architectures, text diffusion

### Chapter 8: Computer Vision
- [Image Fundamentals](chapter 08: computer vision/01. image fundamentals.md): Pixels, colour spaces, filtering, edge detection, SIFT, feature extraction
- [Convolutional Networks](chapter 08: computer vision/02. convolutional networks.md): Convolution, pooling, LeNet, AlexNet, VGG, ResNet, EfficientNet, MobileNet
- [Detection and Segmentation](chapter 08: computer vision/03. object detection and segmentation.md): YOLO, SSD, Faster R-CNN, RetinaNet, U-Net, Mask R-CNN, panoptic
- [Vision Transformers and Generation](chapter 08: computer vision/04. vision transformers and generation.md): ViT, DeiT, Swin, DINO, MAE, GANs, diffusion, flow matching
- [Video and 3D Vision](chapter 08: computer vision/05. video and 3D vision.md): Optical flow, SlowFast, tracking, stereo depth, NeRF, 3D Gaussian Splatting, SLAM

### Chapter 9: Audio & Speech
- [Digital Signal Processing](chapter 09: audio and speech/01. digital signal processing.md): Waveforms, sampling, Fourier transform, spectrograms, MFCCs
- [Automatic Speech Recognition](chapter 09: audio and speech/02. automatic speech recognition.md): CTC, RNN-T, Conformer, Whisper, wav2vec
- [Text to Speech](chapter 09: audio and speech/03. text to speech and voice.md): WaveNet, Tacotron, VITS, voice conversion
- [Speaker and Audio Analysis](chapter 09: audio and speech/04. speaker and audio analysis.md): Speaker recognition, diarisation, VAD
- [Source Separation](chapter 09: audio and speech/05. source separation and noise.md): Conv-TasNet, active noise cancellation

### Chapter 10: Multimodal Learning
- [Multimodal Representations](chapter 10: multimodal learning/01. multimodal representations.md): Fusion strategies, CLIP, ALIGN, SigLIP, contrastive learning, InfoNCE
- [Vision Language Models](chapter 10: multimodal learning/02. vision language models.md): VQA, Flamingo, LLaVA, PaLI, grounding, document understanding
- [Image and Video Tokenisation](chapter 10: multimodal learning/03. image and video tokenisation.md): VQ-VAE, VQ-GAN, residual quantisation, video tokenisers
- [Cross-Modal Generation](chapter 10: multimodal learning/04. cross-modal generation.md): DALL-E, Stable Diffusion, Imagen, text-to-video, text-to-audio
- [Unified Architectures](chapter 10: multimodal learning/05. unified multimodal architectures.md): Gemini, GPT-4o, multimodal agents, world models

### Chapter 11: Autonomous Systems
- [Perception](chapter 11: autonomous systems/01. perception.md): Sensors, sensor fusion, BEVFusion, 3D detection, depth estimation, occupancy networks
- [Robot Learning](chapter 11: autonomous systems/02. robot learning.md): Kinematics, dynamics, PID/MPC/impedance control, imitation learning, sim-to-real, world models
- [Vision-Language-Action Models](chapter 11: autonomous systems/03. vision-language-action models.md): VLAs, RT-2, Octo, OpenVLA, Pi-0, action tokenisation
- [Self-Driving](chapter 11: autonomous systems/04. self-driving.md): Driving stack, motion prediction, planning, end-to-end driving, world models, safety, SAE levels
- [Space and Extreme Robotics](chapter 11: autonomous systems/05. space and extreme robotics.md): Planetary rovers, communication constraints, underwater, swarm robotics, HRI

### Chapter 12: Graph Neural Networks
- [Geometric Deep Learning](chapter 12: graph neural networks/01. geometric deep learning.md): Symmetry groups, invariance, equivariance, five geometric domains
- [Graph Theory](chapter 12: graph neural networks/02. graph theory.md): Adjacency, Laplacian, spectral theory, community detection
- [Graph Neural Networks](chapter 12: graph neural networks/03. graph neural networks.md): Message passing, GCN, GraphSAGE, GIN, over-smoothing, heterogeneous graphs, link prediction
- [Graph Attention Networks](chapter 12: graph neural networks/04. graph attention networks.md): GAT, GATv2, Graph Transformers, Graphormer, GPS, temporal graphs
- [3D Graph Networks](chapter 12: graph neural networks/05. 3d graph networks.md): SE(3)-equivariance, SchNet, DimeNet, EGNN, MACE, graph generation, drug discovery

### Chapter 13: Computing and OS
- [Discrete Maths](chapter 13: computing and OS/01. discrete maths.md): Logic, proofs, sets, relations, functions, graph theory, recurrences, computability, P vs NP
- [Computer Architecture](chapter 13: computing and OS/02. computer architecture.md): Number systems, IEEE 754, logic gates, CPU, ISAs, pipelining, memory hierarchy, virtual memory, I/O, DMA
- [Operating Systems](chapter 13: computing and OS/03. operating systems.md): Processes, threads, scheduling, memory management, file systems, networking, containers, security
- [Concurrency and Parallelism](chapter 13: computing and OS/04. concurrency and parallelism.md): Synchronisation, deadlock, lock-free, OpenMP, MPI, async/await, Amdahl's law
- [Programming Languages](chapter 13: computing and OS/05. programming languages.md): Paradigms, type systems, memory management, compilation, JIT, closures, pattern matching

### Chapter 14: Data Structures and Algorithms
- [Foundations](chapter 14: data structures and algorithms/00. foundations.md): Big O notation, recursion, backtracking, dynamic programming, pattern recognition for interviews
- [Arrays and Hashing](chapter 14: data structures and algorithms/01. arrays and hashing.md): Arrays, hash tables, two pointers, sliding window, prefix sums (with NeetCode problems)
- [Linked Lists, Stacks, and Queues](chapter 14: data structures and algorithms/02. linked lists, stacks, and queues.md): Fast/slow pointers, monotonic stack, heaps (with NeetCode problems)
- [Trees](chapter 14: data structures and algorithms/03. trees.md): BSTs, tries, Union-Find, segment/Fenwick trees (with NeetCode problems)
- [Graphs](chapter 14: data structures and algorithms/04. graphs.md): BFS, DFS, Dijkstra, topological sort, SCCs (with NeetCode problems)
- [Sorting and Search](chapter 14: data structures and algorithms/05. sorting and search.md): Merge/quick sort, binary search, greedy, DP, backtracking (with NeetCode problems)

### Chapter 15: Production Software Engineering
- [Linux and CMD](chapter 15: production software engineering/01. linux and CMD.md): Shell, file system, permissions, processes, SSH, essential ML commands
- [Git and Version Control](chapter 15: production software engineering/02. git and repository management.md): Branching, merging, PRs, git for ML, experiment tracking
- [Codebase Design](chapter 15: production software engineering/03. codebase design.md): Project structure, clean code, design patterns, config management, API design, AI coding agents
- [Testing and QA](chapter 15: production software engineering/04. testing and quality assurance.md): pytest, mocking, testing ML code, CI/CD, linting, code review
- [Deployment and DevOps](chapter 15: production software engineering/05. deployment and devops.md): Docker, model serving, experiment tracking, reproducibility, monitoring, feature stores, orchestration

### Chapter 16: SIMD and GPU Programming
- [Why C++ and How ML Frameworks Work](chapter 16: SIMD and GPU programming/00. why C++ and how ML frameworks work.md): Python/C++ architecture, NumPy/PyTorch/JAX internals, C++ fundamentals, pybind11
- [Hardware Fundamentals](chapter 16: SIMD and GPU programming/01. hardware fundamentals.md): Moore's law, SIMD concept, roofline model, chip architectures
- [ARM and NEON](chapter 16: SIMD and GPU programming/02. ARM and NEON.md): NEON intrinsics, I8MM, SME2, SVE, Apple Silicon, auto-vectorisation, Cactus engine
- [x86 and AVX](chapter 16: SIMD and GPU programming/03. x86 and AVX.md): AVX2/AVX-512/AMX intrinsics, alignment, profiling
- [GPU Architecture and CUDA](chapter 16: SIMD and GPU programming/04. GPU architecture and CUDA.md): GPU design, CUDA C++, warps, shared memory tiling, advanced optimisations, GPU generations
- [Triton, TPUs, and Pallas](chapter 16: SIMD and GPU programming/05. triton, TPUs and pallax.md): Triton kernels, Flash Attention, TPU architecture, JAX/Pallas
- [RISC-V and Embedded Systems](chapter 16: SIMD and GPU programming/06. RISC-V and embedded systems.md): RISC-V V extension, TinyML, TFLM, edge deployment constraints
- [Vulkan Compute](chapter 16: SIMD and GPU programming/07. vulkan compute and cross-platform GPU.md): Vulkan pipeline, GLSL compute shaders, Kompute, WebGPU

### Chapter 17: AI Inference
- [Quantisation](chapter 17: AI inference/01. quantisation.md): PTQ, QAT, GPTQ, AWQ, QuIP#, HQQ, AQLM, BitNet, FP8, MX formats, KV-cache quantisation
- [Efficient Architectures](chapter 17: AI inference/02. efficient architectures.md): StreamingLLM, sparse/linear attention, MQA/GQA/MLA, Mamba, Flash Attention, Ring Attention, pruning
- [Serving and Batching](chapter 17: AI inference/03. serving and batching.md): Prefill vs decode, continuous batching, PagedAttention/vLLM, constrained generation, request routing
- [Edge Inference](chapter 17: AI inference/04. edge inference.md): On-device runtimes, compiler stack, NPUs, on-device LLMs, federated learning, Cactus engine
- [Scaling and Deployment](chapter 17: AI inference/05. scaling and deployment.md): Tensor/pipeline parallelism, speculative decoding, prefix caching, KV-cache eviction, inference frameworks, cost optimisation

### Chapter 18: ML Systems Design
- [Systems Design Fundamentals](chapter 18: ML systems design/01. systems design fundamentals.md): Client-server, networking, DNS, load balancing, caching, databases, CAP theorem, message queues, API design
- [Cloud Computing](chapter 18: ML systems design/02. cloud computing.md): IaaS/PaaS/SaaS, Kubernetes, storage, networking, serverless, cost management, multi-region, IaC
- [Large Scale Infrastructure](chapter 18: ML systems design/03. large scale infrastructure.md): Distributed systems, microservices, data pipelines, GPU clusters, InfiniBand, fault tolerance, vector search, observability
- [ML Systems Design](chapter 18: ML systems design/04. ML systems design.md): ML lifecycle, data management, feature stores, A/B testing, feedback loops, fairness, monitoring
- [ML Design Examples](chapter 18: ML systems design/05. ML design examples.md): Recommendation systems, search ranking, ads, fraud detection, content moderation, RAG chatbot, image search

### Chapter 19: Applied AI
- [AI for Finance](chapter 19: applied AI/01. AI for finance.md): Time series, algorithmic trading, portfolio optimisation, risk modelling
- [Protein Design](chapter 19: applied AI/02. protein design.md): AlphaFold, protein structure prediction, inverse folding
- [Drug Discovery](chapter 19: applied AI/03. drug discovery.md): Molecular representations, virtual screening, binding affinity prediction
- [Agentic Systems](chapter 19: applied AI/04. agentic systems.md): AI agents, tool use, planning, multi-agent systems
- [Healthcare](chapter 19: applied AI/05. healthcare.md): Medical imaging, clinical NLP, drug safety, health monitoring

### Chapter 20: Bleeding Edge AI
- [Quantum Machine Learning](chapter 20: bleeding edge AI/01. quantum machine learning.md): Qubits, quantum gates, variational circuits, quantum kernels
- [Neuromorphic Computing](chapter 20: bleeding edge AI/02. neuromorphic computing.md): Spiking neural networks, neuromorphic hardware, event-driven vision
- [Datacentres in Space](chapter 20: bleeding edge AI/03. datacentres in space.md): Orbital computing, latency, radiation, power constraints
- [Decentralised AI](chapter 20: bleeding edge AI/04. decentralised AI.md): Federated learning, blockchain ML, distributed training
- [Brain Machine Interfaces](chapter 20: bleeding edge AI/05. brain machine interfaces.md): Neural decoding, BCI architectures, neural signal processing