Reference vs Margin: Decoding the True Performance of DPO and SimPO

DPO and SimPO employ fundamentally different preference tuning strategies—one anchors optimization to a reference model while the other pursues margin-based improvements—yet practitioners rarely compare them rigorously. Without controlling for confounding factors like LoRA rank and validating against held-out metrics, it's impossible to determine whether performance gains reflect genuine algorithmic superiority or merely optimization artifacts. Effective preference tuning demands systematic evaluation under controlled conditions rather than assumption-based algorithm selection.