Mixture of Insights.
← Writing

Series · 5 articles

Post-Training in Practice

Data generation, verifiers, GRPO, DPO, and self-play in ORBIT.

Follow ORBIT training trajectories from generation and filtering to reward validation, GRPO, DPO and self-play. The articles distinguish executable paths from design sketches and discuss where evaluation limits the conclusions.

Read in the order below: start with the constraints, then follow the implementation and debugging details.

In this series

  1. 01

    LLM post-training data: generation, verification and yield

    12 min read
  2. 02

    SFT cold-start before GRPO for constrained planning

    13 min read
  3. 03

    Planning-agent rewards: verifiers, reward models and reward hacking

    13 min read
  4. 04

    DPO for role-play: preference pairs and chosen likelihood

    12 min read
  5. 05

    OpenSpiel training data: MCTS, CFR and trajectory filtering

    13 min read