A 566GB FP8 MiMo-2.6-Pro model was successfully loaded and run on a 128GB Jetson AGX Thor. The execution achieved 0.646 tokens per second by reading model weights from disk mid-inference.
HOW THIS AFFECTS YOU
●
builderYou can technically run massive models on constrained edge hardware via disk-based inference, though latency is extremely high.