Trajectory-Aware Low-Rank Approximation for Diffusion Language Models

Traj-MC improves compression for diffusion language models by formulating a low-rank objective over the entire inference trajectory rather than static activations. This approach uses Monte Carlo sampling to estimate the trajectory second moment, preserving mathematical reasoning during model compression.