Collaborative Personalized Preference Alignment for LLMs under Data Deficiency

Approximate Pareto Optimality (APO) mitigates gradient conflicts when training lightweight aligners for heterogeneous user preferences. The method groups users with compatible update directions to learn shared initializations, enabling effective few-shot adaptation despite scarce individual feedback and competing multi-objective constraints.