🧩 LLM Fundamentals is optional and covers fundamental knowledge about mathematics, Python, and neural networks.
🧩 LLM基础 (LLM Fundamentals) 是可选的，涵盖关于数学、Python和神经网络的基础知识。
🧑‍🔬 The LLM Scientist focuses on building the best possible LLMs using the latest techniques.
🧑‍🔬 LLM科学家 (LLM Scientist) 专注于使用最新的技术构建尽可能最好的LLM。
👷 The LLM Engineer focuses on creating LLM-based applications and deploying them.
👷 LLM工程师 (LLM Engineer) 专注于创建基于LLM的应用程序并部署它们。

Toggle section (optional)

Tools

Notebook	Description	Notebook
🧐 LLM AutoEval	Automatically evaluate your LLMs using RunPod
🥱 LazyMergekit	Easily merge models using MergeKit in one click.
🦎 LazyAxolotl	Fine-tune models in the cloud using Axolotl in one click.
⚡ AutoQuant	Quantize LLMs in GGUF, GPTQ, EXL2, AWQ, and HQQ formats in one click.
🌳 Model Family Tree	Visualize the family tree of merged models.
🚀 ZeroSpace	Automatically create a Gradio chat interface using a free ZeroGPU.
✂️ AutoAbliteration	Automatically abliteration models with custom datasets.
🧼 AutoDedup	Automatically deduplicate datasets using the Rensa library.

Fine-tuning

微调 (Fine-tuning)

Notebook	Description	Article	Notebook
Fine-tune Llama 3.1 with Unsloth	Ultra-efficient supervised fine-tuning in Google Colab.	Article
Fine-tune Llama 3 with ORPO	Cheaper and faster fine-tuning in a single stage with ORPO.	Article
Fine-tune Mistral-7b with DPO	Boost the performance of supervised fine-tuned models with DPO.	Article
Fine-tune Mistral-7b with QLoRA	Supervised fine-tune Mistral-7b in a free-tier Google Colab with TRL.
Fine-tune CodeLlama using Axolotl	End-to-end guide to the state-of-the-art tool for fine-tuning.	Article
Fine-tune Llama 2 with QLoRA	Step-by-step guide to supervised fine-tune Llama 2 in Google Colab.	Article

Quantization

量化 (Quantization)

Notebook	Description	Article	Notebook
Introduction to Quantization	Large language model optimization using 8-bit quantization.	Article
4-bit Quantization using GPTQ	Quantize your own open-source LLMs to run them on consumer hardware.	Article
Quantization with GGUF and llama.cpp	Quantize Llama 2 models with llama.cpp and upload GGUF versions to the HF Hub.	Article
ExLlamaV2: The Fastest Library to Run LLMs	Quantize and run EXL2 models and upload them to the HF Hub.	Article

Other

Notebook	Description	Article	Notebook
Merge LLMs with MergeKit	Create your own models easily, no GPU required!	Article
Create MoEs with MergeKit	Combine multiple experts into a single frankenMoE	Article
Uncensor any LLM with abliteration	Fine-tuning without retraining	Article
Improve ChatGPT with Knowledge Graphs	Augment ChatGPT's answers with knowledge graphs.	Article
Decoding Strategies in Large Language Models	A guide to text generation from beam search to nucleus sampling	Article

1. The LLM Architecture

Architectural overview: Understand the evolution from encoder-decoder Transformers to decoder-only architectures like GPT, which form the basis of modern LLMs. Focus on how these models process and generate text at a high level.
架构概述：了解从编码器-解码器 Transformers 到像 GPT 这样的仅解码器架构的演变，GPT 构成了现代 LLM 的基础。重点关注这些模型如何在较高层次上处理和生成文本。
Tokenization: Learn the principles of tokenization - how text is converted into numerical representations that LLMs can process. Explore different tokenization strategies and their impact on model performance and output quality.
分词：了解分词的原理 - 如何将文本转换为 LLM 可以处理的数字表示。探索不同的分词策略及其对模型性能和输出质量的影响。
Attention mechanisms: Master the core concepts of attention mechanisms, particularly self-attention and its variants. Understand how these mechanisms enable LLMs to process long-range dependencies and maintain context throughout sequences.
注意力机制：掌握注意力机制的核心概念，特别是自注意力及其变体。了解这些机制如何使 LLM 能够处理远程依赖关系并在整个序列中保持上下文。
Sampling techniques: Explore various text generation approaches and their tradeoffs. Compare deterministic methods like greedy search and beam search with probabilistic approaches like temperature sampling and nucleus sampling.
采样技术：探索各种文本生成方法及其权衡。比较确定性方法（如贪婪搜索和束搜索）与概率方法（如温度采样和 nucleus 采样）。

📚 References:

• Visual intro to Transformers by 3Blue1Brown: Visual introduction to Transformers for complete beginners.
LLM Visualization by Brendan Bycroft: Interactive 3D visualization of LLM internals.
Brendan Bycroft 的 LLM 可视化：LLM 内部结构的交互式 3D 可视化。
nanoGPT by Andrej Karpathy: A 2h-long YouTube video to reimplement GPT from scratch (for programmers). He also made a video about tokenization.
Andrej Karpathy 的 nanoGPT：一个 2 小时的 YouTube 视频，用于从头开始重新实现 GPT（适用于程序员）。他还制作了一个关于分词的视频。
Attention? Attention! by Lilian Weng: Historical overview to introduce the need for attention mechanisms.
Lilian Weng 的 Attention? Attention!：介绍注意力机制需求的背景概述。
Decoding Strategies in LLMs by Maxime Labonne: Provide code and a visual introduction to the different decoding strategies to generate text.
Maxime Labonne 的 LLM 解码策略：提供代码和可视化介绍不同的解码策略来生成文本。

2. Pre-Training Models

Data preparation: Pre-training requires massive datasets (e.g., Llama 3.1 was trained on 15 trillion tokens) that need careful curation, cleaning, deduplication, and tokenization. Modern pre-training pipelines implement sophisticated filtering to remove low-quality or problematic content.
数据准备：预训练需要海量数据集（例如，Llama 3.1 在 15 万亿个 token 上进行了训练），这些数据集需要仔细的整理、清理、去重和分词。现代预训练管道实施复杂的过滤以删除低质量或有问题的 内容。
Distributed training: Combine different parallelization strategies: data parallel (batch distribution), pipeline parallel (layer distribution), and tensor parallel (operation splitting). These strategies require optimized network communication and memory management across GPU clusters.
分布式训练：结合不同的并行化策略：数据并行（批处理分布）、流水线并行（层分布）和张量并行（运算分割）。这些策略需要跨 GPU 集群的优化网络通信和内存管理。
Training optimization: Use adaptive learning rates with warm-up, gradient clipping, and normalization to prevent explosions, mixed-precision training for memory efficiency, and modern optimizers (AdamW, Lion) with tuned hyperparameters.
训练优化：使用带有预热的自适应学习率、梯度裁剪和归一化来防止爆炸，使用混合精度训练以提高内存效率，并使用带有调整的超参数的现代优化器（AdamW、Lion）。
Monitoring: Track key metrics (loss, gradients, GPU stats) using dashboards, implement targeted logging for distributed training issues, and set up performance profiling to identify bottlenecks in computation and communication across devices.
监控：使用仪表板跟踪关键指标（损失值（loss）、梯度（gradients）、GPU 统计信息），针对分布式训练问题实施有针对性的日志记录，并设置性能分析，以识别跨设备的计算和通信瓶颈。

📚 References:

FineWeb by Penedo et al.: Article to recreate a large-scale dataset for LLM pretraining (15T), including FineWeb-Edu, a high-quality subset.
Penedo et al. 的 FineWeb：重建大规模 LLM 预训练数据集 (15T) 的文章，包括高质量子集 FineWeb-Edu。
RedPajama v2 by Weber et al.: Another article and paper about a large-scale pre-training dataset with a lot of interesting quality filters.
Weber et al. 的 RedPajama v2：另一篇关于大规模预训练数据集的文章和论文，包含许多有趣的质量过滤器。
nanotron by Hugging Face: Minimalistic LLM training codebase used to make SmolLM2.
Hugging Face 的 nanotron：用于制作 SmolLM2 的极简 LLM 训练代码库。
Parallel training by Chenyan Xiong: Overview of optimization and parallelism techniques.
Chenyan Xiong 的并行训练（Parallel training）：优化和平行化技术概述。
Distributed training by Duan et al.: A survey about efficient training of LLM on distributed architectures.
Duan et al. 的分布式训练（Distributed training）：关于在分布式架构上高效训练 LLM 的一项调查。
OLMo 2 by AI2: Open-source language model with model, data, training, and evaluation code.
AI2 的 OLMo 2：包含模型、数据、训练和评估代码的开源语言模型。
LLM360 by LLM360: A framework for open-source LLMs with training and data preparation code, data, metrics, and models.
LLM360 的 LLM360：一个开源 LLM 框架，包含训练和数据准备代码、数据、指标和模型。

3. Post-Training Datasets

Storage & chat templates: Because of the conversational structure, post-training datasets are stored in a specific format like ShareGPT or OpenAI/HF. Then, these formats are mapped to a chat template like ChatML or Alpaca to produce the final samples that the model is trained on.
存储与聊天模板：由于会话结构，后期训练数据集以特定格式存储，例如 ShareGPT 或 OpenAI/HF。然后，这些格式被映射到聊天模板（chat template），如 ChatML 或 Alpaca，以生成模型训练的最终样本。
Synthetic data generation: Create instruction-response pairs based on seed data using frontier models like GPT-4o. This approach allows for flexible and scalable dataset creation with high-quality answers. Key considerations include designing diverse seed tasks and effective system prompts.
合成数据生成：使用 GPT-4o 等前沿模型，根据种子数据创建指令-响应对。这种方法可以灵活且可扩展地创建具有高质量答案的数据集。关键考虑因素包括设计多样化的种子任务和有效的系统提示。
Data enhancement: Enhance existing samples using techniques like verified outputs (using unit tests or solvers), multiple answers with rejection sampling, Auto-Evol, Chain-of-Thought, Branch-Solve-Merge, personas, etc.
数据增强：使用经过验证的输出（使用单元测试或求解器）、具有拒绝采样的多个答案、Auto-Evol、思维链（Chain-of-Thought）、分支-解决-合并（Branch-Solve-Merge）、角色扮演（personas）等技术来增强现有样本。
Quality filtering: Traditional techniques involve rule-based filtering, removing duplicates or near-duplicates (with MinHash or embeddings), and n-gram decontamination. Reward models and judge LLMs complement this step with fine-grained and customizable quality control.
质量过滤：传统技术包括基于规则的过滤、删除重复或近重复项（使用 MinHash 或嵌入（embeddings））和 n-gram 去污。奖励模型（Reward models）和评判 LLM 通过细粒度和可定制的质量控制来补充此步骤。

📚 References:

Synthetic Data Generator by Argilla: Beginner-friendly way of building datasets using natural language in a Hugging Face space.
Argilla 的合成数据生成器（Synthetic Data Generator）：在 Hugging Face space 中使用自然语言构建数据集的初学者友好方式。
LLM Datasets by Maxime Labonne: Curated list of datasets and tools for post-training.
Maxime Labonne 的 LLM Datasets：后期训练数据集和工具的精选列表。
NeMo-Curator by Nvidia: Dataset preparation and curation framework for pre- and post-training data.
Nvidia 的 NeMo-Curator：用于预训练和后期训练数据的数据集准备和管理框架。
Distilabel by Argilla: Framework to generate synthetic data. It also includes interesting reproductions of papers like UltraFeedback.
Argilla 的 Distilabel：生成合成数据的框架。它还包括对 UltraFeedback 等论文的有趣重现。
Semhash by MinishLab: Minimalistic library for near-deduplication and decontamination with a distilled embedding model.
MinishLab 的 Semhash：一个极简的库，用于使用精馏嵌入模型进行近重复数据删除和去污。
Chat Template by Hugging Face: Hugging Face's documentation about chat templates.
Hugging Face 的聊天模板（Chat Template）：Hugging Face 关于聊天模板的文档。

4. Supervised Fine-Tuning

Training techniques: Full fine-tuning updates all model parameters but requires significant compute. Parameter-efficient fine-tuning techniques like LoRA and QLoRA reduce memory requirements by training a small number of adapter parameters while keeping base weights frozen. QLoRA combines 4-bit quantization with LoRA to reduce VRAM usage. These techniques are all implemented in the most popular fine-tuning frameworks: TRL, Unsloth, and Axolotl.
训练技术：完全微调会更新所有模型参数，但需要大量计算。参数高效微调技术（Parameter-efficient fine-tuning），如 LoRA 和 QLoRA，通过训练少量适配器参数（adapter parameters）同时保持基础权重冻结来减少内存需求。QLoRA 将 4 位量化与 LoRA 结合使用，以减少 VRAM 使用量。这些技术都在最流行的微调框架中实现：TRL、Unsloth 和 Axolotl。
Training parameters: Key parameters include learning rate with schedulers, batch size, gradient accumulation, number of epochs, optimizer (like 8-bit AdamW), weight decay for regularization, and warmup steps for training stability. LoRA also adds three parameters: rank (typically 16-128), alpha (1-2x rank), and target modules.
训练参数：关键参数包括带调度器的学习率（learning rate with schedulers）、批大小（batch size）、梯度累积（gradient accumulation）、epoch 数、优化器（如 8 位 AdamW）、用于正则化的权重衰减（weight decay）以及用于训练稳定性的预热步数（warmup steps）。LoRA 还添加了三个参数：秩（rank，通常为 16-128）、alpha（1-2 倍秩）和目标模块（target modules）。
Distributed training: Scale training across multiple GPUs using DeepSpeed or FSDP. DeepSpeed provides three ZeRO optimization stages with increasing levels of memory efficiency through state partitioning. Both methods support gradient checkpointing for memory efficiency.
分布式训练：使用 DeepSpeed 或 FSDP 在多个 GPU 上扩展训练。DeepSpeed 提供三个 ZeRO 优化阶段，通过状态分区（state partitioning）提高内存效率。这两种方法都支持梯度检查点（gradient checkpointing）以提高内存效率。
Monitoring: Track training metrics including loss curves, learning rate schedules, and gradient norms. Monitor for common issues like loss spikes, gradient explosions, or performance degradation.
监控：跟踪训练指标，包括损失曲线（loss curves）、学习率计划（learning rate schedules）和梯度范数（gradient norms）。监控常见问题，如损失峰值（loss spikes）、梯度爆炸（gradient explosions）或性能下降。

📚 References:

Fine-tune Llama 3.1 Ultra-Efficiently with Unsloth by Maxime Labonne: Hands-on tutorial on how to fine-tune a Llama 3.1 model using Unsloth.
Maxime Labonne 的使用 Unsloth 超高效微调 Llama 3.1（Fine-tune Llama 3.1 Ultra-Efficiently with Unsloth）：关于如何使用 Unsloth 微调 Llama 3.1 模型的实践教程。
Axolotl - Documentation by Wing Lian: Lots of interesting information related to distributed training and dataset formats.
Wing Lian 的 Axolotl - 文档：与分布式训练和数据集格式相关的许多有趣信息。
Mastering LLMs by Hamel Husain: Collection of educational resources about fine-tuning (but also RAG, evaluation, applications, and prompt engineering).
Hamel Husain 的掌握 LLM（Mastering LLMs）：关于微调（但也包括 RAG、评估、应用程序和提示工程）的教育资源集合。
LoRA insights by Sebastian Raschka: Practical insights about LoRA and how to select the best parameters.
Sebastian Raschka 的 LoRA 见解（LoRA insights）：关于 LoRA 以及如何选择最佳参数的实践见解。

5. Preference Alignment

Rejection sampling: For each prompt, use the trained model to generate multiple responses, and score them to infer chosen/rejected answers. This creates on-policy data, where both responses come from the model being trained, improving alignment stability.
拒绝采样：对于每个提示，使用经过训练的模型生成多个响应，并对它们进行评分以推断选定/拒绝的答案。这会创建 on-policy 数据，其中两个响应都来自正在训练的模型，从而提高对齐稳定性。
Direct Preference Optimization Directly optimizes the policy to maximize the likelihood of chosen responses over rejected ones. It doesn't require reward modeling, which makes it more computationally efficient than RL techniques but slightly worse in terms of quality. Great for creating chat models.
直接偏好优化（Direct Preference Optimization）：直接优化策略以最大化选定响应相对于拒绝响应的可能性。它不需要奖励建模（reward modeling），这使得它比 RL 技术在计算上更有效，但在质量方面略逊一筹。非常适合创建聊天模型。
Reward model: Train a reward model with human feedback to predict metrics like human preferences. It can leverage frameworks like TRL, verl, and OpenRLHF for scalable training.
奖励模型：使用人工反馈训练奖励模型，以预测诸如人类偏好之类的指标。它可以利用 TRL、verl 和 OpenRLHF 等框架进行可扩展的训练。
Reinforcement Learning: RL techniques like GRPO and PPO iteratively update a policy to maximize rewards while staying close to the initial behavior. They can use a reward model or reward functions to score responses. They tend to be computationally expensive and require careful tuning of hyperparameters, including learning rate, batch size, and clip range. Ideal for creating reasoning models.
强化学习：RL 技术（如 GRPO 和 PPO）迭代地更新策略以最大化奖励，同时保持接近初始行为。他们可以使用奖励模型或奖励函数来对响应进行评分。它们往往计算成本很高，并且需要仔细调整超参数，包括学习率、批大小和裁剪范围。非常适合创建推理模型。

📚 References:

Illustrating RLHF by Hugging Face: Introduction to RLHF with reward model training and fine-tuning with reinforcement learning.
使用 Hugging Face 说明 RLHF：RLHF 介绍，包括奖励模型训练和使用强化学习进行微调。
LLM Training: RLHF and Its Alternatives by Sebastian Raschka: Overview of the RLHF process and alternatives like RLAIF.
LLM 训练：RLHF 及其替代方案，作者 Sebastian Raschka：RLHF 流程的概述以及 RLAIF 等替代方案。
Preference Tuning LLMs by Hugging Face: Comparison of the DPO, IPO, and KTO algorithms to perform preference alignment.
使用 Hugging Face 进行偏好调整 LLM：比较 DPO (Direct Preference Optimization)、IPO (Identity Preference Optimization) 和 KTO (Kahneman-Tversky Optimizer) 算法以执行偏好对齐。
Fine-tune with DPO by Maxime Labonne: Tutorial to fine-tune a Mistral-7b model with DPO and reproduce NeuralHermes-2.5.
使用 DPO 进行微调，作者 Maxime Labonne：使用 DPO 微调 Mistral-7b 模型并复现 NeuralHermes-2.5 的教程。
Fine-tune with GRPO by Maxime Labonne: Practical exercise to fine-tune a small model with GRPO.
使用 GRPO 进行微调，作者 Maxime Labonne：使用 GRPO 微调小型模型的实践练习。
DPO Wandb logs by Alexander Vishnevskiy: It shows you the main DPO metrics to track and the trends you should expect.
Alexander Vishnevskiy 的 DPO Wandb 日志：它向您展示了要跟踪的主要 DPO 指标以及您应该期望的趋势。

6. Evaluation

Automated benchmarks: Evaluate models on specific tasks using curated datasets and metrics, like MMLU. It works well for concrete tasks but struggles with abstract and creative capabilities. It is also prone to data contamination.
自动基准测试 (Automated benchmarks)：使用经过整理的数据集和指标（如 MMLU）评估特定任务的模型。它适用于具体的任务，但在抽象和创造性能力方面表现不佳。它也容易受到数据污染。
Human evaluation: It involves humans prompting models and grading responses. Methods range from vibe checks to systematic annotations with specific guidelines and large-scale community voting (arena). It is more suited for subjective tasks and less reliable for factual accuracy.
人工评估 (Human evaluation)：它涉及人类提示模型并对响应进行评分。方法范围从氛围检查到具有特定指南的大规模社区投票（竞技场）的系统注释。它更适合主观任务，而对于事实准确性则不太可靠。
Model-based evaluation: Use judge and reward models to evaluate model outputs. It highly correlates with human preferences but suffers from bias toward their own outputs and inconsistent scoring.
基于模型的评估 (Model-based evaluation)：使用判断 (judge) 和奖励 (reward) 模型来评估模型输出。它与人类偏好高度相关，但容易对自己的输出产生偏差且评分不一致。
Feedback signal: Analyze error patterns to identify specific weaknesses, such as limitations in following complex instructions, lack of specific knowledge, or susceptibility to adversarial prompts. This can be improved with better data generation and training parameters.
反馈信号 (Feedback signal)：分析错误模式以识别特定弱点，例如在遵循复杂指令方面的局限性、缺乏特定知识或容易受到对抗性提示的影响。这可以通过更好的数据生成和训练参数来改进。

📚 References:

LLM evaluation guidebook by Hugging Face: Comprehensive guide about evaluation with practical insights.
Hugging Face 的 LLM 评估指南：关于评估的综合指南，包含实践见解。
Open LLM Leaderboard by Hugging Face: Main leaderboard to compare LLMs in an open and reproducible way (automated benchmarks).
Hugging Face 的 Open LLM 排行榜：以开放和可重复的方式比较 LLM 的主要排行榜（自动基准测试）。
Language Model Evaluation Harness by EleutherAI: A popular framework for evaluating LLMs using automated benchmarks.
EleutherAI 的语言模型评估工具 (Language Model Evaluation Harness)：一个流行的框架，用于使用自动基准测试评估 LLM。
Lighteval by Hugging Face: Alternative evaluation framework that also includes model-based evaluations.
Hugging Face 的 Lighteval：另一种评估框架，也包括基于模型的评估。
Chatbot Arena by LMSYS: Elo rating of general-purpose LLMs, based on comparisons made by humans (human evaluation).
LMSYS 的聊天机器人竞技场 (Chatbot Arena)：基于人类比较的通用 LLM 的 Elo 评分（人工评估）。

7. Quantization

Base techniques: Learn the different levels of precision (FP32, FP16, INT8, etc.) and how to perform naïve quantization with absmax and zero-point techniques.
基础技术：了解不同的精度级别（FP32、FP16、INT8 等）以及如何使用 absmax 和零点技术执行朴素量化。
GGUF & llama.cpp: Originally designed to run on CPUs, llama.cpp and the GGUF format have become the most popular tools to run LLMs on consumer-grade hardware. It supports storing special tokens, vocabulary, and metadata in a single file.
GGUF & llama.cpp：llama.cpp 和 GGUF 格式最初设计用于在 CPU 上运行，现已成为在消费级硬件上运行 LLM 的最流行工具。它支持在单个文件中存储特殊标记、词汇和元数据。
GPTQ & AWQ: Techniques like GPTQ/EXL2 and AWQ introduce layer-by-layer calibration that retains performance at extremely low bitwidths. They reduce catastrophic outliers using dynamic scaling, selectively skipping or re-centering the heaviest parameters.
GPTQ & AWQ：GPTQ/EXL2 和 AWQ 等技术引入了逐层校准，可以在极低的位宽下保持性能。它们使用动态缩放减少灾难性异常值，有选择地跳过或重新居中最重的参数。
SmoothQuant & ZeroQuant: New quantization-friendly transformations (SmoothQuant) and compiler-based optimizations (ZeroQuant) help mitigate outliers before quantization. They also reduce hardware overhead by fusing certain ops and optimizing dataflow.
SmoothQuant & ZeroQuant：新的量化友好型转换 (SmoothQuant) 和基于编译器的优化 (ZeroQuant) 有助于在量化之前减轻异常值。它们还通过融合某些操作和优化数据流来减少硬件开销。

📚 References:

Introduction to quantization by Maxime Labonne: Overview of quantization, absmax and zero-point quantization, and LLM.int8() with code.
Maxime Labonne 的量化介绍：量化、absmax 和零点量化以及带有代码的 LLM.int8() 的概述。
Quantize Llama models with llama.cpp by Maxime Labonne: Tutorial on how to quantize a Llama 2 model using llama.cpp and the GGUF format.
Maxime Labonne 的使用 llama.cpp 量化 Llama 模型：关于如何使用 llama.cpp 和 GGUF 格式量化 Llama 2 模型的教程。
4-bit LLM Quantization with GPTQ by Maxime Labonne: Tutorial on how to quantize an LLM using the GPTQ algorithm with AutoGPTQ.
Maxime Labonne 的使用 GPTQ 进行 4 位 LLM 量化：关于如何使用带有 AutoGPTQ 的 GPTQ 算法量化 LLM 的教程。
Understanding Activation-Aware Weight Quantization by FriendliAI: Overview of the AWQ technique and its benefits.
FriendliAI 的理解激活感知权重量化：AWQ 技术及其优势的概述。
SmoothQuant on Llama 2 7B by MIT HAN Lab: Tutorial on how to use SmoothQuant with a Llama 2 model in 8-bit precision.
MIT HAN Lab 的 Llama 2 7B 上的 SmoothQuant：关于如何在 8 位精度下将 SmoothQuant 与 Llama 2 模型一起使用的教程。
DeepSpeed Model Compression by DeepSpeed: Tutorial on how to use ZeroQuant and extreme compression (XTC) with DeepSpeed Compression.
DeepSpeed 的 DeepSpeed 模型压缩：关于如何将 ZeroQuant 和极限压缩 (XTC) 与 DeepSpeed 压缩一起使用的教程。

8. New Trends

Model merging: Merging trained models has become a popular way of creating performant models without any fine-tuning. The popular mergekit library implements the most popular merging methods, like SLERP, DARE, and TIES.
模型合并 (Model merging)：合并经过训练的模型已成为创建高性能模型而无需任何微调的流行方式。流行的 mergekit 库实现了最流行的合并方法，如 SLERP、DARE 和 TIES。
Multimodal models: These models (like CLIP, Stable Diffusion, or LLaVA) process multiple types of inputs (text, images, audio, etc.) with a unified embedding space, which unlocks powerful applications like text-to-image.
多模态模型 (Multimodal models)：这些模型（如 CLIP、Stable Diffusion 或 LLaVA）使用统一的嵌入空间处理多种类型的输入（文本、图像、音频等），从而释放了强大的应用程序（如文本到图像）。
Interpretability: Mechanistic interpretability techniques like Sparse Autoencoders (SAEs) have made remarkable progress to provide insights about the inner workings of LLMs. This has also been applied with techniques such as abliteration, which allow you to modify the behavior of models without training.
可解释性 (Interpretability)：像稀疏自动编码器 (SAE) 这样的机械可解释性技术取得了显著进展，可以提供有关 LLM 内部运作的见解。这也已应用于诸如消融 (abliteration) 之类的技术，这些技术允许您在不进行训练的情况下修改模型的行为。
Test-time compute: Reasoning models trained with RL techniques can be further improved by scaling the compute budget during test time. It can involve multiple calls, MCTS, or specialized models like a Process Reward Model (PRM). Iterative steps with precise scoring significantly improve performance for complex reasoning tasks.
测试时计算 (Test-time compute)：通过 RL 技术训练的推理模型可以通过在测试时缩放计算预算来进一步改进。它可能涉及多次调用、MCTS 或专用模型，如过程奖励模型 (PRM)。通过精确评分的迭代步骤可以显著提高复杂推理任务的性能。

📚 References:

Merge LLMs with mergekit by Maxime Labonne: Tutorial about model merging using mergekit.
Maxime Labonne 的使用 mergekit 合并 LLM：关于使用 mergekit 进行模型合并的教程。
Smol Vision by Merve Noyan: Collection of notebooks and scripts dedicated to small multimodal models.
Merve Noyan 的 Smol Vision：专门用于小型多模态模型的笔记本和脚本的集合。
Large Multimodal Models by Chip Huyen: Overview of multimodal systems and the recent history of this field.
大型多模态模型 (Large Multimodal Models) by Chip Huyen：多模态系统概述以及该领域的近期历史。
Unsensor any LLM with abliteration by Maxime Labonne: Direct application of interpretability techniques to modify the style of a model.
使用 Ablitration 技术取消 LLM 的审查 (Unsensor any LLM with abliteration) by Maxime Labonne：直接应用可解释性技术来修改模型的风格。
Intuitive Explanation of SAEs by Adam Karvonen: Article about how SAEs work and why they make sense for interpretability.
SAE 的直观解释 (Intuitive Explanation of SAEs) by Adam Karvonen：关于 SAE 如何工作以及为什么它们对可解释性有意义的文章。 （SAE: Sparse Autoencoder, 稀疏自编码器）
Scaling test-time compute by Beeching et al.: Tutorial and experiments to outperform Llama 3.1 70B on MATH-500 with a 3B model.
扩展测试时计算 (Scaling test-time compute) by Beeching et al.：教程和实验，使用 3B 模型在 MATH-500 上超越 Llama 3.1 70B。

1. Running LLMs

LLM APIs: APIs are a convenient way to deploy LLMs. This space is divided between private LLMs (OpenAI, Google, Anthropic, etc.) and open-source LLMs (OpenRouter, Hugging Face, Together AI, etc.).
LLM API：API 是部署 LLM 的便捷方式。该领域分为私有 LLM（OpenAI、Google、Anthropic 等）和开源 LLM（OpenRouter、Hugging Face、Together AI 等）。
Open-source LLMs: The Hugging Face Hub is a great place to find LLMs. You can directly run some of them in Hugging Face Spaces, or download and run them locally in apps like LM Studio or through the CLI with llama.cpp or ollama.
开源 LLM：Hugging Face Hub 是查找 LLM 的好地方。您可以直接在 Hugging Face Spaces 中运行其中的一些，或者下载并在本地应用程序（如 LM Studio）中运行，或者通过 CLI 使用 llama.cpp 或 ollama 运行。
Prompt engineering: Common techniques include zero-shot prompting, few-shot prompting, chain of thought, and ReAct. They work better with bigger models, but can be adapted to smaller ones.
提示工程 (Prompt engineering)：常见技术包括零样本提示 (zero-shot prompting)、少样本提示 (few-shot prompting)、思维链 (chain of thought) 和 ReAct。它们在大模型上效果更好，但可以适应较小的模型。
Structuring outputs: Many tasks require a structured output, like a strict template or a JSON format. Libraries like Outlines can be used to guide the generation and respect a given structure. Some APIs also support structured output generation natively using JSON schemas.
结构化输出 (Structuring outputs)：许多任务需要结构化输出，例如严格的模板或 JSON 格式。像 Outlines 这样的库可用于指导生成并遵守给定的结构。一些 API 还支持使用 JSON 模式 (JSON schemas) 本机生成结构化输出。

📚 References:

Run an LLM locally with LM Studio by Nisha Arya: Short guide on how to use LM Studio.
使用 LM Studio 在本地运行 LLM (Run an LLM locally with LM Studio) by Nisha Arya：关于如何使用 LM Studio 的简短指南。
Prompt engineering guide by DAIR.AI: Exhaustive list of prompt techniques with examples
DAIR.AI 的提示工程指南 (Prompt engineering guide by DAIR.AI)：带有示例的详尽的提示技术列表。
Outlines - Quickstart: List of guided generation techniques enabled by Outlines.
Outlines - 快速入门 (Outlines - Quickstart)：Outlines 启用的引导生成技术列表。
LMQL - Overview: Introduction to the LMQL language.
LMQL - 概述 (LMQL - Overview)：LMQL 语言简介。

2. Building a Vector Storage

Ingesting documents: Document loaders are convenient wrappers that can handle many formats: PDF, JSON, HTML, Markdown, etc. They can also directly retrieve data from some databases and APIs (GitHub, Reddit, Google Drive, etc.).
摄取文档 (Ingesting documents)：文档加载器 (Document loaders) 是便捷的包装器，可以处理多种格式：PDF、JSON、HTML、Markdown 等。它们还可以直接从某些数据库和 API（GitHub、Reddit、Google Drive 等）检索数据。
Splitting documents: Text splitters break down documents into smaller, semantically meaningful chunks. Instead of splitting text after n characters, it's often better to split by header or recursively, with some additional metadata.
拆分文档 (Splitting documents)：文本拆分器 (Text splitters) 将文档分解为更小、语义上有意义的块。与其在 n 个字符后拆分文本，不如按标题或递归方式拆分，并添加一些额外的元数据。
Embedding models: Embedding models convert text into vector representations. Picking task-specific models significantly improves performance for semantic search and RAG.
嵌入模型 (Embedding models)：嵌入模型将文本转换为向量表示。选择特定于任务的模型可以显着提高语义搜索和 RAG 的性能。
Vector databases: Vector databases (like Chroma, Pinecone, Milvus, FAISS, Annoy, etc.) are designed to store embedding vectors. They enable efficient retrieval of data that is 'most similar' to a query based on vector similarity.
向量数据库 (Vector databases)：向量数据库（如 Chroma、Pinecone、Milvus、FAISS、Annoy 等）旨在存储嵌入向量。它们可以有效地检索与基于向量相似性的查询“最相似”的数据。

📚 References:

LangChain - Text splitters: List of different text splitters implemented in LangChain.
LangChain - 文本拆分器 (LangChain - Text splitters)：LangChain 中实现的各种文本拆分器列表。
Sentence Transformers library: Popular library for embedding models.
Sentence Transformers 库：用于嵌入模型的流行库。
MTEB Leaderboard: Leaderboard for embedding models.
MTEB 排行榜 (MTEB Leaderboard)：嵌入模型的排行榜。
The Top 7 Vector Databases by Moez Ali: A comparison of the best and most popular vector databases.
Moez Ali 的前 7 大向量数据库 (The Top 7 Vector Databases by Moez Ali)：对最佳和最流行的向量数据库的比较。

3. Retrieval Augmented Generation

Orchestrators: Orchestrators like LangChain and LlamaIndex are popular frameworks to connect your LLMs with tools and databases. The Model Context Protocol (MCP) introduces a new standard to pass data and context to models across providers.
编排器 (Orchestrators)：像 LangChain 和 LlamaIndex 这样的编排器是将您的 LLM 与工具和数据库连接起来的流行框架。模型上下文协议 (Model Context Protocol, MCP) 引入了一个新的标准，用于跨提供商将数据和上下文传递给模型。
Retrievers: Query rewriters and generative retrievers like CoRAG and HyDE enhance search by transforming user queries. Multi-vector and hybrid retrieval methods combine embeddings with keyword signals to improve recall and precision.
检索器 (Retrievers)：查询重写器和生成式检索器 (generative retrievers)，如 CoRAG 和 HyDE，通过转换用户查询来增强搜索。多向量和混合检索方法 (Multi-vector and hybrid retrieval methods) 将嵌入与关键词信号相结合，以提高召回率和精确度。
Memory: To remember previous instructions and answers, LLMs and chatbots like ChatGPT add this history to their context window. This buffer can be improved with summarization (e.g., using a smaller LLM), a vector store + RAG, etc.
记忆 (Memory)：为了记住之前的指令和答案，LLM 和聊天机器人（如 ChatGPT）会将此历史记录添加到其上下文窗口 (context window)。可以使用摘要（例如，使用较小的 LLM）、向量存储 + RAG 等来改进此缓冲区。
Evaluation: We need to evaluate both the document retrieval (context precision and recall) and the generation stages (faithfulness and answer relevancy). It can be simplified with tools Ragas and DeepEval (assessing quality).
评估 (Evaluation)：我们需要评估文档检索（上下文精确度和召回率）和生成阶段（忠实度和答案相关性）。可以使用工具 Ragas 和 DeepEval（评估质量）来简化它。

📚 References:

Llamaindex - High-level concepts: Main concepts to know when building RAG pipelines.
Llamaindex - 高级概念 (Llamaindex - High-level concepts)：构建 RAG 管道时需要了解的主要概念。
Model Context Protocol: Introduction to MCP with motivate, architecture, and quick starts.
模型上下文协议 (Model Context Protocol)：MCP 简介，包括动机、架构和快速入门。
Pinecone - Retrieval Augmentation: Overview of the retrieval augmentation process.
Pinecone - 检索增强 (Pinecone - Retrieval Augmentation)：检索增强过程概述。
LangChain - Q&A with RAG: Step-by-step tutorial to build a typical RAG pipeline.
LangChain - 使用 RAG 进行问答 (LangChain - Q&A with RAG)：构建典型 RAG 管道的分步教程。
LangChain - Memory types: List of different types of memories with relevant usage.
LangChain - 记忆类型 (LangChain - Memory types)：各种记忆类型及其相关用法的列表。
RAG pipeline - Metrics: Overview of the main metrics used to evaluate RAG pipelines.
RAG流水线 - 指标：用于评估RAG流水线的主要指标概述 (RAG pipeline - Metrics: Overview of the main metrics used to evaluate RAG pipelines)。

4. Advanced RAG

Query construction: Structured data stored in traditional databases requires a specific query language like SQL, Cypher, metadata, etc. We can directly translate the user instruction into a query to access the data with query construction.
查询构建 (Query construction)：存储在传统数据库中的结构化数据需要特定的查询语言，例如SQL、Cypher、元数据等。我们可以直接将用户指令翻译成查询，以通过查询构建来访问数据。
Tools: Agents augment LLMs by automatically selecting the most relevant tools to provide an answer. These tools can be as simple as using Google or Wikipedia, or more complex, like a Python interpreter or Jira.
工具 (Tools)：智能体 (Agents) 通过自动选择最相关的工具来提供答案，从而增强大型语言模型 (LLMs) 的能力。这些工具可以像使用谷歌或维基百科一样简单，也可以像Python解释器或Jira一样复杂。
Post-processing: Final step that processes the inputs that are fed to the LLM. It enhances the relevance and diversity of documents retrieved with re-ranking, RAG-fusion, and classification.
后处理 (Post-processing)：处理馈送到大型语言模型 (LLM) 的输入的最后一步。它通过重新排序 (re-ranking)、RAG-fusion和分类来提高检索到的文档的相关性和多样性。
Program LLMs: Frameworks like DSPy allow you to optimize prompts and weights based on automated evaluations in a programmatic way.
程序化大型语言模型 (Program LLMs)：像DSPy这样的框架允许您基于程序化方式的自动评估来优化提示 (prompts) 和权重 (weights)。

📚 References:

LangChain - Query Construction: Blog post about different types of query construction.
LangChain - 查询构建 (Query Construction)：关于不同类型查询构建的博客文章。
LangChain - SQL: Tutorial on how to interact with SQL databases with LLMs, involving Text-to-SQL and an optional SQL agent.
LangChain - SQL：关于如何使用大型语言模型 (LLMs) 与SQL数据库交互的教程，包括Text-to-SQL和一个可选的SQL智能体 (agent)。
Pinecone - LLM agents: Introduction to agents and tools with different types.
Pinecone - 大型语言模型 (LLM) 智能体 (agents)：介绍具有不同类型的智能体 (agents) 和工具 (tools)。
LLM Powered Autonomous Agents by Lilian Weng: A more theoretical article about LLM agents.
Lilian Weng的《LLM Powered Autonomous Agents》：一篇关于大型语言模型 (LLM) 智能体 (agents) 的更理论性的文章。
LangChain - OpenAI's RAG: Overview of the RAG strategies employed by OpenAI, including post-processing.
LangChain - OpenAI的RAG：概述OpenAI采用的RAG策略，包括后处理 (post-processing)。
DSPy in 8 Steps: General-purpose guide to DSPy introducing modules, signatures, and optimizers.
DSPy in 8 Steps：通用的DSPy指南，介绍模块 (modules)、签名 (signatures) 和优化器 (optimizers)。

5. Agents

Agent fundamentals: Agents operate using thoughts (internal reasoning to decide what to do next), action (executing tasks, often by interacting with external tools), and observation (analyzing feedback or results to refine the next step).
智能体 (Agent) 基础知识：智能体 (Agents) 的运作方式是：思考（决定下一步做什么的内部推理）、行动（执行任务，通常通过与外部工具交互）和观察（分析反馈或结果以改进下一步）。
Agent protocols: Model Context Protocol (MCP) is the industry standard for connecting agents to external tools and data sources with MCP servers and clients. More recently, Agent2Agent (A2A) tries to standardize a common language for agent interoperability.
智能体 (Agent) 协议：模型上下文协议 (Model Context Protocol, MCP) 是将智能体 (agents) 连接到具有MCP服务器和客户端的外部工具和数据源的行业标准。最近，Agent2Agent (A2A) 尝试标准化智能体 (agent) 互操作性的通用语言。
Vendor frameworks: Each major cloud model provider has its own agentic framework with OpenAI SDK, Google ADK, and Claude Agent SDK if you're particularly tied to one vendor.
供应商框架 (Vendor frameworks)：每个主要的云模型提供商都有自己的智能体框架 (agentic framework)，包括OpenAI SDK、Google ADK和Claude Agent SDK，如果你特别依赖于某个供应商。
Other frameworks: Agent development can be streamlined using different frameworks like LangGraph (design and visualization of workflows) LlamaIndex (data-augmented agents with RAG), or custom solutions. More experimental frameworks include collaboration between different agents, such as CrewAI (role-based team workflows) and AutoGen (conversation-driven multi-agent systems).
其他框架 (Other frameworks)：可以使用不同的框架来简化智能体 (agent) 的开发，例如LangGraph（工作流程的设计和可视化）、LlamaIndex（具有RAG的数据增强智能体 (agents)）或自定义解决方案。更具实验性的框架包括不同智能体 (agents) 之间的协作，例如CrewAI（基于角色的团队工作流程）和AutoGen（会话驱动的多智能体系统）。

📚 References:

Agents Course: Popular course about AI agents made by Hugging Face.
Agents Course：Hugging Face制作的关于AI智能体 (agents) 的热门课程。
LangGraph: Overview of how to build AI agents with LangGraph.
LangGraph：概述如何使用LangGraph构建AI智能体 (agents)。
LlamaIndex Agents: Uses cases and resources to build agents with LlamaIndex.
LlamaIndex Agents：使用案例和资源来使用LlamaIndex构建智能体 (agents)。

6. Inference optimization

Flash Attention: Optimization of the attention mechanism to transform its complexity from quadratic to linear, speeding up both training and inference.
Flash Attention：注意力机制的优化，将其复杂度从二次方转换为线性，从而加速训练和推理。
Key-value cache: Understand the key-value cache and the improvements introduced in Multi-Query Attention (MQA) and Grouped-Query Attention (GQA).
键值缓存 (Key-value cache)：了解键值缓存 (key-value cache) 以及多查询注意力 (Multi-Query Attention, MQA) 和分组查询注意力 (Grouped-Query Attention, GQA) 中引入的改进。
Speculative decoding: Use a small model to produce drafts that are then reviewed by a larger model to speed up text generation. EAGLE-3 is a particularly popular solution.
推测解码 (Speculative decoding)：使用一个小型模型来生成草稿，然后由一个较大的模型进行审查，以加速文本生成。EAGLE-3 是一种特别流行的解决方案。

📚 References:

GPU Inference by Hugging Face: Explain how to optimize inference on GPUs.
GPU Inference by Hugging Face：解释如何在GPU上优化推理。
LLM Inference by Databricks: Best practices for how to optimize LLM inference in production.
LLM Inference by Databricks：关于如何在生产环境中优化大型语言模型 (LLM) 推理的最佳实践。
Optimizing LLMs for Speed and Memory by Hugging Face: Explain three main techniques to optimize speed and memory, namely quantization, Flash Attention, and architectural innovations.
Optimizing LLMs for Speed and Memory by Hugging Face：解释了优化速度和内存的三个主要技术，即量化 (quantization)、Flash Attention和架构创新。
Assisted Generation by Hugging Face: HF's version of speculative decoding. It's an interesting blog post about how it works with code to implement it.
Assisted Generation by Hugging Face：HF的推测解码 (speculative decoding) 版本。这是一篇关于如何使用代码来实现它的有趣博客文章。
EAGLE-3 paper: Introduces EAGLE-3 and reports speedups up to 6.5×.
EAGLE-3 paper：介绍了EAGLE-3，并报告了高达6.5倍的加速。
Speculators: Library made by vLLM for building, evaluating, and storing speculative decoding algorithms (e.g., EAGLE-3) for LLM inference.
Speculators：vLLM制作的用于构建、评估和存储用于大型语言模型 (LLM) 推理的推测解码 (speculative decoding) 算法（例如，EAGLE-3）的库。

7. Deploying LLMs

Local deployment: Privacy is an important advantage that open-source LLMs have over private ones. Local LLM servers (LM Studio, Ollama, oobabooga, kobold.cpp, etc.) capitalize on this advantage to power local apps.
本地部署 (Local deployment)：隐私是开源大型语言模型 (LLMs) 相对于私有大型语言模型 (LLMs) 的一个重要优势。本地大型语言模型 (LLM) 服务器（LM Studio、Ollama、oobabooga、kobold.cpp等）利用此优势来支持本地应用程序。
Demo deployment: Frameworks like Gradio and Streamlit are helpful to prototype applications and share demos. You can also easily host them online, for example, using Hugging Face Spaces.
演示部署 (Demo deployment)：像Gradio和Streamlit这样的框架有助于原型化应用程序和分享演示。您还可以轻松地在线托管它们，例如，使用Hugging Face Spaces。
Server deployment: Deploying LLMs at scale requires cloud (see also SkyPilot) or on-prem infrastructure and often leverages optimized text generation frameworks like TGI, vLLM, etc.
服务器部署：大规模部署大型语言模型 (LLM) 需要云（另见 SkyPilot）或本地基础设施，并且通常利用优化的文本生成框架，例如 TGI、vLLM 等。
Edge deployment: In constrained environments, high-performance frameworks like MLC LLM and mnn-llm can deploy LLM in web browsers, Android, and iOS.
边缘部署：在受限环境中，高性能框架（如 MLC LLM 和 mnn-llm）可以将大型语言模型 (LLM) 部署在 Web 浏览器、Android 和 iOS 中。

📚 References:

Streamlit - Build a basic LLM app: Tutorial to make a basic ChatGPT-like app using Streamlit.
Streamlit - 构建一个基本的 LLM 应用程序：使用 Streamlit 构建一个基本的类似 ChatGPT 的应用程序的教程。
HF LLM Inference Container: Deploy LLMs on Amazon SageMaker using Hugging Face's inference container.
HF LLM 推理容器 (Inference Container)：使用 Hugging Face 的推理容器在 Amazon SageMaker 上部署大型语言模型 (LLM)。
Philschmid blog by Philipp Schmid: Collection of high-quality articles about LLM deployment using Amazon SageMaker.
Philschmid 博客，作者 Philipp Schmid：关于使用 Amazon SageMaker 部署大型语言模型 (LLM) 的高质量文章集合。
Optimizing latence by Hamel Husain: Comparison of TGI, vLLM, CTranslate2, and mlc in terms of throughput and latency.
Hamel Husain 优化延迟 (Optimizing Latence)：从吞吐量和延迟的角度比较 TGI、vLLM、CTranslate2 和 mlc。

8. Securing LLMs

Prompt hacking: Different techniques related to prompt engineering, including prompt injection (additional instruction to hijack the model's answer), data/prompt leaking (retrieve its original data/prompt), and jailbreaking (craft prompts to bypass safety features).
提示词攻击 (Prompt hacking)：与提示词工程 (prompt engineering) 相关的不同技术，包括提示词注入 (prompt injection)（向模型答案添加额外指令来劫持模型）、数据/提示词泄露 (data/prompt leaking)（检索其原始数据/提示词）和越狱 (jailbreaking)（精心设计提示词以绕过安全功能）。
Backdoors: Attack vectors can target the training data itself, by poisoning the training data (e.g., with false information) or creating backdoors (secret triggers to change the model's behavior during inference).
后门 (Backdoors)：攻击向量可以针对训练数据本身，方法是污染训练数据（例如，使用虚假信息）或创建后门（在推理过程中更改模型行为的秘密触发器）。
Defensive measures: The best way to protect your LLM applications is to test them against these vulnerabilities (e.g., using red teaming and checks like garak) and observe them in production (with a framework like langfuse).
防御措施：保护大型语言模型 (LLM) 应用程序的最佳方法是针对这些漏洞对其进行测试（例如，使用红队测试 (red teaming) 和 garak 等检查工具）并在生产环境中观察它们（使用像 langfuse 这样的框架）。

OWASP LLM Top 10 by HEGO Wiki: List of the 10 most critical vulnerabilities seen in LLM applications.
HEGO Wiki 的 OWASP LLM Top 10：大型语言模型 (LLM) 应用程序中发现的 10 个最关键的漏洞列表。
Prompt Injection Primer by Joseph Thacker: Short guide dedicated to prompt injection for engineers.
Joseph Thacker 的提示词注入入门 (Prompt Injection Primer)：为工程师编写的关于提示词注入 (prompt injection) 的简短指南。
LLM Security by @llm_sec: Extensive list of resources related to LLM security.
@llm_sec 的 LLM 安全：与大型语言模型 (LLM) 安全相关的广泛资源列表。
Red teaming LLMs by Microsoft: Guide on how to perform red teaming with LLMs.
微软的 LLM 红队测试：关于如何使用大型语言模型 (LLM) 执行红队测试 (red teaming) 的指南。