Quantifying the Chaos: How Lie Brackets Reveal Neural Network Order Dependence

This research provides a mathematical framework to quantify how the sequence of training data shifts a model's parameters, challenging the ideal Bayesian view of unordered datasets. By treating training examples as vector fields, Lie Brackets offer a way to measure and potentially optimize the non-commuting updates that define deep learning optimization.