AI & Fundamentals

Challenges and Opportunities for Federated Learning in the Age of Foundation Models - Han Yu, Associate Professor, Nanyang Technological University

Abstract:

The rise of large foundation models underscores the importance and relevance of federated learning as a key research direction. As LLMs become the mainstream in machine learning development, the research focus is shifting from model architecture design to addressing challenges related to privacy preservation and distributed learning in order to efficiently leverage privately owned valuable but sensitive data. Advances in federated learning as an infrastructure for collaborative foundation model training/finetuning has the potential to unlock the value of large models by enabling efficient and scalable training while safeguarding sensitive data. The long-term healthy development of this field requires continuously attracting high-quality data owners to collaboratively build models and share the benefits. In this talk, I will share some of the efforts in this emerging area from the Trustworthy Federated Ubiquitous Learning (TrustFUL) Lab at Nanyang Technological University, Singapore. These include methods for quantifying participant contributions in federated settings, ensuring fairness among diverse participants, establishing multi-agent automated data auctions to support federated training, as well as collaborative training of foundation models in federated settings. I will also share about 2 deployed case studies related to public sector service provision.

Bio:

Dr. Han Yu is a tenured Associate Professor in the College of Computing and Data Science (CCDS), Nanyang Technological University (NTU), Singapore. Between 2018 and 2024, he was a Nanyang Assistant Professor (NAP) in CCDS, NTU. He has been a Visiting Scholar at the Department of Computer Science and Engineering, Hong Kong University of Science and Technology (HKUST) from 2017 to 2018. Between 2015 and 2018, he held the prestigious Lee Kuan Yew Post-Doctoral Fellowship (LKY PDF) at NTU. He obtained his PhD from the School of Computer Science and Engineering, NTU in 2014. His work focuses on trustworthy federated learning. He has published over 300 research papers in leading international conferences and journals. He co-authored the book Federated Learning - the first monograph in this field. His research work has been recognized with multiple scientific awards. In 2021, he co-founded the Trustworthy Federated Ubiquitous Learning (TrustFUL) Research Lab (https://trustful.federated-learning.org/). He is an Associate Editor of IEEE TNNLS, IJCAI Sponsorship Officer General, and Vice Chair of the IEEE Computational Intelligence Society (CIS) Standards Committee. For his continued contributions to the field of trustworthy AI and real-world impact in the society, he has been identified as one of the World's Top 2% Scientists in AI, and selected as one of the JCI Ten Outstanding Young Persons (TOYP) of Singapore.