Grouped Query Experts Adds MoE Routing to GQA Query Heads Only

Grouped Query Experts (GQE) applies mixture-of-experts routing to query heads within each GQA group, selecting k active query-head experts per token while keeping all KV heads dense. This preserves KV cache efficiency from GQA while reducing active query-head compute, targeting the quadratic attention cost at long context lengths.