DeepSpeed is an easy-to-use deep learning optimization software suite that enables unprecedented scale and speed for Deep Learning Training and Inference. With DeepSpeed you can:

• Train/Inference dense or sparse models with billions or trillions of parameters
• Achieve excellent system throughput and efficiently scale to thousands of GPUs
• Train/Inference on resource constrained GPU systems
• Achieve unprecedented low latency and high throughput for inference
• Achieve extreme compression for an unparalleled inference latency and model size reduction with low costs

DeepSpeed is an important part of Microsoft’s new AI at Scale initiative to enable next-generation AI capabilities at scale, where you can find more information here.