Books, courses, papers, and tools for each stage
The foundational textbook covering neural networks, optimization, and generative models.
deeplearningbook.org →Comprehensive NLP textbook — covers n-grams to Transformers. Free draft online.
Stanford SLP3 →Interactive deep learning textbook with code in PyTorch, JAX, and TensorFlow.
d2l.ai →Hands-on guide to building a GPT-like LLM from scratch with PyTorch.
Manning →Practical guide to building NLP apps with Hugging Face — covers fine-tuning, summarization, QA, and deployment.
O'Reilly →Visual, intuitive deep-dive into LLM internals — embeddings, generation, fine-tuning, and semantic search.
O'Reilly →End-to-end ML system design — data engineering, feature stores, deployment, monitoring. Applicable to LLM systems.
O'Reilly →Linear algebra, calculus, probability, and optimization — the math backbone for understanding Transformers. Free PDF.
mml-book.github.io →Comprehensive, rigorous coverage of modern ML from a probabilistic perspective. Free draft online.
probml.github.io →Concise yet thorough ML primer — perfect for quick reference or refreshing fundamentals.
themlbook.com →Classic deep learning fundamentals — backprop, CNNs, training tips.
cs231n.stanford.edu →The gold-standard NLP course covering word vectors, Transformers, and LLMs.
cs224n.stanford.edu →Build a GPT from scratch in a video series. Superb for deep understanding.
karpathy.ai →Top-down, practical approach to deep learning. Great for building intuition quickly.
course.fast.ai →Free, hands-on course covering the Transformers library and modern NLP tasks.
huggingface.co →Practical LLM application development — prompt engineering, RAG, deployment.
fullstackdeeplearning.com →Short course on prompt engineering best practices — iterative development, summarizing, inferring, transforming.
deeplearning.ai →Learn when and how to fine-tune LLMs — data preparation, training, evaluation.
deeplearning.ai →Build LLM apps with chains, memory, agents, and evaluation using LangChain.
deeplearning.ai →Dedicated LLM course covering modeling, training, adaptation, and societal impact.
stanford-cs324 →Fast-paced intro to deep learning — RNNs, Transformers, generative models with TensorFlow labs.
introtodeeplearning.com →Graduate-level NLP covering pre-training, generation, structured prediction, and multilingual models.
phontron.com →Free, comprehensive LLM course on GitHub — architecture, pre-training, SFT, RLHF, quantization, and deployment.
GitHub →Build multi-step systems — classification, moderation, chain-of-thought, and evaluation pipelines.
deeplearning.ai →Demonstrated emergent in-context learning at scale.
arXiv →RLHF paper that shaped modern chatbot alignment.
arXiv →Meta's open-weight model family that catalyzed open-source LLM research.
arXiv →Efficient fine-tuning by injecting trainable low-rank matrices.
arXiv →Simplified alignment without a separate reward model.
arXiv →Combining retrieval with generation for knowledge-intensive tasks.
arXiv →IO-aware exact attention that is 2–4× faster and uses 5–20× less memory.
arXiv →Introduced masked language modeling and bidirectional pre-training. Revolutionized NLP benchmarks.
arXiv →Showed that scaling up language models yields zero-shot task transfer.
OpenAI PDF →Discovered power-law relationships between model size, data, compute, and loss.
arXiv →Showed models should be trained on more data than previously thought. Reshaped LLM training budgets.
arXiv →Demonstrated that step-by-step reasoning prompts dramatically improve LLM problem-solving.
arXiv →Anthropic's approach to self-supervised alignment using a set of principles (a "constitution").
arXiv →Detailed the training recipe for Llama 2, including RLHF alignment and safety tuning.
arXiv →Efficient 7B model using sliding window attention and GQA, outperforming larger models.
arXiv →Simplified MoE with top-1 routing — scale to trillions of parameters efficiently.
arXiv →Interleaving reasoning traces with actions for grounded, tool-using LLM agents.
arXiv →Extends chain-of-thought with deliberate search over reasoning paths using BFS/DFS.
arXiv →Memory-efficient attention variant between MHA and MQA — used in Llama 2, Mistral, and beyond.
arXiv →Elegant rotary position encoding used in LLaMA, Mistral, Qwen — enables length extrapolation.
arXiv →Trains LMs to decide when/how to call APIs (calculator, search, calendar) autonomously.
arXiv →One-shot weight quantization to 3-4 bits with minimal accuracy loss. Foundational for local LLM deployment.
arXiv →Alternative to Transformers using selective state spaces — linear complexity, strong language modeling.
arXiv →Multi-head latent attention + DeepSeekMoE — high efficiency MoE architecture with strong performance.
arXiv →Comprehensive 100+ page survey covering pre-training, alignment, utilization, and evaluation of LLMs.
arXiv →State-of-the-art pretrained models with a unified API.
huggingface.co →Framework for building LLM-powered applications and agents.
langchain.com →Run LLMs locally with one command. Supports Llama, Mistral, Gemma, and more.
ollama.com →C/C++ inference engine for LLMs — runs quantized models on CPU/GPU with GGUF format.
GitHub →Streamlined fine-tuning tool — supports LoRA, QLoRA, full fine-tune with YAML configs.
GitHub →Deep learning optimization library — ZeRO stages, mixed-precision, pipeline parallelism for large-scale training.
GitHub →Data curation platform for LLMs — label, curate, and build instruction datasets with human feedback.
GitHub →Pre-train, fine-tune, and deploy 20+ LLM architectures with a unified, hackable codebase.
GitHub →Desktop app for discovering, downloading, and running local LLMs with a ChatGPT-like UI.
lmstudio.ai →Build ML web demos in minutes — chatbots, image classifiers, and more with Python.
gradio.app →Active community for open-source LLM news, benchmarks, and discussions.
Reddit →Forums, Discord, and model hub — the epicenter of open-source AI.
huggingface.co →Track state-of-the-art results, find implementations, and explore benchmarks.
paperswithcode.com →@karpathy, @ylecun, @AndrewYNg, @rasbt (Sebastian Raschka), @srush_nlp, @_jasonwei — stay current with frontier research.
@karpathy →Compare LLMs head-to-head. The gold standard for model evaluation with crowd-sourced Elo ratings.
chat.lmsys.org →Track and compare open-source model performance across standard benchmarks.
HF Leaderboard →Curated lists of LLM tools, papers, and resources kept up-to-date by the community.
Awesome-LLM →Weekly AI newsletter — curated news, research summaries, and practical insights from Andrew Ng.
deeplearning.ai →High-signal newsletter covering LLM research, fine-tuning techniques, and practical ML insights.
sebastianraschka.com →Grassroots AI research lab — open-source models (GPT-NeoX, Pythia), datasets, and active Discord community.
eleuther.ai →In-depth video explanations of the latest AI papers, benchmarks, and developments.
YouTube →Paper walkthroughs and ML news — one of the best channels for understanding new research.
YouTube →The largest ML subreddit — paper discussions, research AMAs, and industry news.
Reddit →