Kai Yi

Research Scientist at Meta AI, Sunnyvale, CA.

prof_pic.jpg

Meta AI

Sunnyvale, CA, USA

I am a Research Scientist at Meta AI in Sunnyvale, working on model compression and inference acceleration for large language models. My research focuses on low-bit quantization, quantization-aware training, post-training compression, and deployment-aware optimization, with the broader goal of making foundation models more efficient to train, adapt, and serve.

I received my Ph.D. in Computer Science from KAUST in June 2025, advised by Prof. Peter Richtárik. Before that, I completed my Master’s degree at KAUST under the supervision of Prof. Mohamed Elhoseiny and received my Bachelor of Engineering with honors from Xi’an Jiaotong University (XJTU).

Before joining Meta AI full-time, I gained research experience through internships and collaborations with Sony AI, Vector Institute, Tencent AI Lab, CMU Xulab, NUS CVML Group, and SenseTime Research.

My recent work studies efficient LLM compression, including extreme low-bit quantization, QAT/PTQ methods, and deployment-aware optimization. Representative works include JacQuant, WinQ, SymWanda and PV-Tuning. Previously, I worked on communication-efficient federated and distributed learning, including CohortSqueeze, FedP3, FedComLoc, and EF-BV.

Research Interests

  • LLM Model Compression: Post-Training Quantization/Pruning, Low-Bit Quantization-Aware Training, Extreme Quantization, Weight/Activation/KV Cache Compression, and Compression-Robust Fine-Tuning
  • LLM Inference Acceleration: Deployment-Efficient Optimization, Hardware-Aware Compression, Memory- and Bandwidth-Efficient Serving, Speculative Decoding, Long-Context Efficiency, and Efficient Attention/KV Management
  • Efficient Training and Adaptation: Efficient Pretraining, Parameter-Efficient Fine-Tuning, Quantization-Aware Adaptation, and Optimization for Resource-Constrained Foundation Models
  • Federated and Distributed Optimization: Communication-Efficient Training, Personalization, Sparsity, Data/Model Heterogeneity, and Scalable Distributed Learning

News

May 01, 2026 Our paper JacQuant is available on arXiv!
May 01, 2026 Gold Reviewer of ICML 2026.
May 01, 2026 WinQ has been accepted for publication in ICML 2026!
Sep 01, 2025 Invited to be the reviewer for AISTATS 2026 and ICLR 2026.
Sep 01, 2025 PhD Dissertation has been submitted to arXiv.
Sep 01, 2025 FedComLoc has been accepted for publication in TMLR!
Sep 01, 2025 Accepted to be the reviewer for ICLR 2026.
Sep 01, 2025 Program Committee Member of AAAI 2026.
Aug 01, 2025 Started as a Research Scientist at Meta, Sunnyvale!
Jun 01, 2025 Reviewer for NeurIPS 2025, TMC x2.
Apr 01, 2025 Scafflix has been accepted for publication in TMLR!
Apr 01, 2025 Successfully defended my PhD titled “Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization.” I’m truly grateful to my committee, colleagues, friends, and schoolmates for their amazing support on this special day!
Mar 01, 2025 SymWanda has been accepted for presentation at SLLM@ICLR 2025.
Feb 01, 2025 Invited as a reviewer for ACM Multimedia and NeurIPS 2025.
Jan 01, 2025 Our paper SymWanda is available on arXiv!
Jan 01, 2025 Reviewer for IEEE Transactions on Mobile Computing (TMC) x2.
Dec 01, 2024 Invited to serve as a reviewer for ICML 2025.
Dec 01, 2024 Attending NeurIPS 2024 in Vancouver, Canada. If you’re also here, let’s connect!
Oct 01, 2024 Our paper Cohort Squeeze has been accepted by NeurIPS FL@FM Workshop as an Oral!
Sep 01, 2024 Our paper PV-Tuning has been accepted by NeurIPS 2024 as an Oral (acceptance rate 0.4%, 61/15671)!
Aug 01, 2024 Invited as a reviewer for ICLR 2025.
Jul 01, 2024 Invited as a reviewer for IEEE Transactions on Signal Processing.
Jun 01, 2024 Our paper Cohort Squeeze is available on arXiv!
May 01, 2024 Our paper Sparse-ProxSkip is available on arXiv!
May 01, 2024 Our paper PV-Tuning is available on arXiv! Code has been released.
May 01, 2024 Reviewer for NeurIPS 2024.
May 01, 2024 Attending ICLR 2024 in Vienna, Austria.
Apr 01, 2024 Passed the PhD proposal defense with the title “Exploring Real-World Challenges in Federated Learning: Personalization, Sparsity, and Scalability”!
Apr 01, 2024 Our paper FedP3 is available on arXiv. Code has been released.
Mar 01, 2024 New paper “FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models” is available on arXiv.
Mar 01, 2024 An improved version of DACZSL is on arXiv.
Jan 01, 2024 Invited as a reviewer for ECCV 2024.
Jan 01, 2024 Our paper FedP3 has been accepted by ICLR24!
Nov 01, 2023 Reviewer for CVPR 2024.
Sep 01, 2023 Reviewer for AISTATS 2024.
Sep 01, 2023 Our paper DACZSL has been accepted by the ICCV23 OOD-CV workshop.
Aug 01, 2023 Reviewer for Computer Vision and Image Understanding (CVIU).
Jul 01, 2023 Our paper IGCZSL has been accepted by ICCV 2023!
Jul 01, 2023 Program Committee member for AAAI 2024.
Jun 01, 2023 Research Internship at SonyAI.
Jun 01, 2023 Reviewer for IJCV.
May 01, 2023 Reviewer for NeurIPS 2023, WACV 2024, BMVC 2023.
May 01, 2023 Our paper “Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning” is available on arXiv. Code is available here.
Mar 01, 2023 Reviewer for ICCV 2023, IJCV, T-PAMI.
Dec 01, 2022 Attending NeurIPS 2022 in New Orleans, LA.
Nov 01, 2022 Reviewer for ICLR 2023, CVPR 2023.
Oct 01, 2022 Reviewer for AISTATS 2023.
Sep 01, 2022 Two papers (EF-BV and VR-ProxSkip) accepted by NeurIPS 2022!
Sep 01, 2022 Program Committee member (PC) for AAAI 2023.
Aug 01, 2022 Serve as an Orientation Leader at KAUST 2022 Fall.
Jul 01, 2022 Our paper “Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning” is available on arXiv.
Jul 01, 2022 HGR-Net for large-scale zero-shot learning has been accepted by ECCV 2022! Code is available here.
Jun 01, 2022 Reviewer for WACV 2023, BMVC 2022.
Jun 01, 2022 I moved from Building-1 L4 to Building-1 L2.
Jun 01, 2022 Reviewer for NeurIPS 2022.
Jun 01, 2022 Continue serving as a Student Ambassador 2022-2023 at KAUST CEMSE!
May 01, 2022 Our paper “EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization” is available on arXiv!
Apr 01, 2022 Our paper “Creative Walk Adversarial Networks: Novel Art Generation with Probabilistic Random Walk Deviation from Style Norms” has been accepted by ICCC 2022!
Apr 01, 2022 Our short paper “Language-Guided Imaginative Walks: Generative Random Walk Deviation Loss for Unseen Class Recognition using Text” has been accepted by CVPR22 L3D-IVU Workshop!
Mar 01, 2022 HGR-Net for large-scale zero-shot learning is available on arXiv.
Mar 01, 2022 Our paper VisualGPT is accepted by CVPR22! code has been released.
Feb 01, 2022 Reviewer for ECCV22, CVPR22, ICML22, IJCV, TIP.
Feb 01, 2022 Teaching Assistant for CS283: Deep Generative Modeling.
Dec 01, 2021 DACZSL is available on arXiv.
Dec 01, 2021 Glad to be a PhD student at Optimization and Machine Learning Lab led by Prof. Peter Richtárik!
Dec 01, 2021 Graduated from KAUST with a Master degree. Thanks a lot to my MS supervisor Prof. Mohamed Elhoseiny and colleagues. Also congratulations to myself!
Nov 01, 2021 My Master’s thesis is available, homepage.
Nov 01, 2021 Successfully defended my Master’s thesis!
May 01, 2021 Our unsupervised open-set recognition work has been accepted to ICIP 2021!
Apr 28, 2021 Spotlight talk of CIZSL++ at KAUST AI Initiative. link.
Apr 01, 2021 Our imaginative walk paper is available on arXiv! –> homepage
Dec 01, 2020 Our paper VisualGPT is on arXiv. code.
Dec 01, 2020 Our paper CIZSL++ has been submitted to PAMI. arXiv paper and code are available.
Dec 01, 2020 Start research internship at Tencent AI Lab.
Aug 01, 2020 Technical report of Legendre decomposition in machine learning is available on arXiv.
May 01, 2020 Start research at KAUST Vision-CAIR group.
Apr 01, 2020 Glad to be a MS/PhD student at KAUST.
Feb 01, 2020 Join CMU Xulab as a remote research intern.

Selected Publications

  1. JacQuant.png
    JacQuant: STE-Free Quantization-Aware Training via Learned Jacobian Surrogates
    Kai Yi, Vignesh Vivekraja, Harshit Khaitan, and 1 more author
    2026
  2. WinQ.png
    WinQ: Accelerating Quantization-Aware Training of Language Models Around Saddle Points
    Dongyue Li, Zechun Liu, Kai Yi, and 6 more authors
    In International Conference on Machine Learning (ICML), 2026
  3. FedComLoc.png
    FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
    Kai Yi, Georg Meinhardt, Laurent Condat, and 1 more author
    Transactions on Machine Learning Research (TMLR), 2025
  4. SymWanda-Symmetric_Pruning.png
    Symmetric Pruning for Large Language Models
    Kai Yi and Peter Richtárik
    In ICLR Workshop on Sparsity in LLMs (SLLM), 2025
  5. PV_Tuning.png
    PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
    Vladimir Malinovskii, Denis Mazur, Ivan Ilin, and 5 more authors
    In Advances in Neural Information Processing Systems (NeurIPS), 2024
    Oral
  6. FedP3.png
    FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
    Kai Yi, Nidham Gazagnadou, Peter Richtárik, and 1 more author
    In International Conference on Learning Representations (ICLR), 2024
  7. HGR-Net_ECCV.png
    Exploring Hierarchical Graph Representation for Large-Scale Zero-Shot Image Classification
    Kai Yi, Xiaoqian Shen, Yunhao Gou, and 1 more author
    In European Conference on Computer Vision (ECCV), 2022
  8. EF-BV.png
    EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization
    Laurent Condat, Kai Yi, and Peter Richtárik
    In Advances in Neural Information Processing Systems (NeurIPS), 2022
  9. VisualGPT.png
    VisualGPT: Data-efficient Image Captioning by Balancing Visual Input and Linguistic Knowledge from Pretraining
    Jun Chen, Han Guo, Kai Yi, and 2 more authors
    In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022