Kairos Proposes Byte-Level Foundation Model via LiZAttention2 and MoE

Kairos introduces a multi-modal architecture using bidirectional gated-DeltaNet linear attention and sliding-window attention over shared projections. It utilizes a multi-scale byte-level codec and sparse Mixture-of-Experts to train on a unified byte space encompassing text, image, audio, video, lidar, and control streams.