Mixture of Block Attention for Long-Context LLMs

Overview

MoBA provides an innovative approach to enhance long-context capabilities in large language models. It is designed for researchers and developers working on advanced LLM training and serving.