Connor Holmes

Lead author of DeepSpeed-FastGen, a high-throughput LLM text generation system combining DeepSpeed-MII and DeepSpeed-Inference. Introduced Dynamic SplitFuse, a novel prompt and generation composition strategy delivering up to 2.3x higher effective throughput, 2x lower average latency, and 3.7x lower tail latency compared to existing systems.

Designed and implemented high-performance CUDA quantization kernels for ZeRO++, enabling 4x reduction in communication volume for distributed LLM training. The block-quantization approach was 3x more accurate and 5x faster than basic quantization, yielding up to 2.16x better throughput at 384 GPU scale.

Co-authored the DeepSpeed-Chat system that democratized RLHF training of ChatGPT-like models at all scales, combining training and inference optimizations to enable training of models with hundreds of billions of parameters at a fraction of the cost.

Created GRNN during his PhD at Colorado School of Mines, a CUDA library for low-latency, scalable RNN inference on GPUs. Achieved up to 17.5x speedup over state-of-the-art CPU inference and 9x over GPU inference libraries through novel data reorganization, thread mapping, and performance modeling. Published at EuroSys 2019.

Served as Systems Lead for Sora at OpenAI, building the systems infrastructure enabling generation of up to 1-minute 1080p videos. Supported the launches of both Sora (Feb 2024) and Sora 2 (Dec 2025).

Co-authored the DeepSpeed4Science initiative enabling large-scale scientific discovery, addressing challenges like memory explosion in protein-structure prediction (Evoformer) and very-long sequence support for virus evolutionary landscape modeling.