Lilin Xu

ICSL Lab
Columbia University
New York, NY

About Me

I am a first second-year PhD student at Columbia University, advised by Prof. Fred Jiang at the ICSL Lab. I obtained my M.E. degree from Control Science and Engineering College, Zhejiang University in 2024, advised by Prof. Chaojie Gu and Prof. Shibo He. I also work with Prof. Rui Tan at Nanyang Technological University. I received my B.E. degree from Control Science and Engineering College, Zhejiang University in 2021.

My research interests are centered around mobile computing and AIoT (AI + IoT), with a primary focus on developing intelligent sensing systems for practical applications. Recently, I have been working on integrating LLMs into sensing systems to effectively exploit the potential of sensing systems and real-world sensor data.

News

  • [2025.09]   One paper ContextAgent is accepted by NeurIPS 2025.
  • [2025.01]   I have been awarded CAA Outstanding Master's Thesis.
  • [2024.12]   One paper SocialMind is accepted by IMWUT.
  • [2024.11]   One paper SoundFace is accepted by ACM Transactions on Sensor Networks.
  • [2024.10]   One paper DrHouse is accepted by IMWUT.
  • [2024.07]   I have been selected for ICDCS'24 Travel Grant.
  • [2024.04]   One paper GesturePrint is accepted by ICDCS 2024.

Selected Publications [Google Scholar]

ContextAgent

ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptions
Bufang Yang*, Lilin Xu* (*equal contribution), Liekang Zeng, Kaiwei Liu, Siyang Jiang, Wenrui Lu, Hongkai Chen, Xiaofan Jiang, Guoliang Xing, Zhenyu Yan
The Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS), 2025
[Paper] [Poster]

SocialMind

SocialMind: LLM-based Proactive AR Social Assistive System with Human-like Perception for In-situ Live Interactions
Bufang Yang*, Yunqi Guo*, Lilin Xu* (*equal contribution), Zhenyu Yan, Hongkai Chen, Guoliang Xing, Xiaofan Jiang
ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2025
[Paper] [Poster]

GesturePrint

GesturePrint: Enabling User Identification for mmWave-based Gesture Recognition Systems
Lilin Xu, Keyi Wang, Chaojie Gu, Xiuzhen Guo, Shibo He, Jiming Chen
44th IEEE International Conference on Distributed Computing Systems (ICDCS), 2024
[Paper] [Slides]

MESEN

MESEN: Exploit Multimodal Data to Design Unimodal Human Activity Recognition with Few Labels
Lilin Xu, Chaojie Gu, Rui Tan, Shibo He, Jiming Chen
The 21th ACM Conference on Embedded Networked Sensor Systems (SenSys), 2023
[Paper] [Slides]

Exploring the Capabilities of LLMs for IMU-based Fine-grained Human Activity Understanding
Lilin Xu*, Kaiyuan Hou* (*equal contribution), Xiaofan Jiang
Proceedings of the 2nd International Workshop on Foundation Models for Cyber-Physical Systems & Internet of Things (FMSys), 2025
[Paper] [Slides]

TaskSense: A Translation-like Approach for Tasking Heterogeneous Sensor Systems with LLMs
Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, Zhenyu Yan
Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems (SenSys), 2025
[Paper]

DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge
Bufang Yang, Siyang Jiang, Lilin Xu, Kaiwei Liu, Hai Li, Guoliang Xing, Hongkai Chen, Xiaofan Jiang, Zhenyu Yan
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2024
[Paper]

Listen to your face: a face authentication scheme based on acoustic signals
Huimin Chen, Chaojie Gu, Lilin Xu, Rui Tan, Shibo He, Jiming Chen ACM Transactions on Sensor Networks (TOSN), 2025
[Paper]

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique
Bicheng Guo, Lilin Xu, Tao Chen, Peng Ye, Shibo He, Haoyu Liu, Jiming Chen
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2023
[Paper]

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search
Bicheng Guo, Tao Chen, Shibo He, Haoyu Liu, Lilin Xu, Peng Ye, Jiming Chen
ACM International Conference on Multimedia (ACM MM), 2022
[Paper]

Selected Awards

  • CAA Outstanding Master's Thesis, 2025.
  • Columbia University Presidential Fellowship, 2024.
  • MobiCom'24 Travel Grant, 2024.
  • ICDCS'24 Travel Grant, 2024.
  • Zhejiang University Sun Youxian Scholarship (Top 1%), 2024.
  • Zhejiang University Outstanding Graduate Student, 2024.
  • SenSys'23 SIG Travel Grant, 2023.
  • AI Studio 2022 CVPR Track2: Performance Estimation Track, Top 10 Award (8/190), 2022.
  • Academic Excellence First-prize Scholarship, 2022 and 2023.
  • Zhejiang University First-prize Scholarship (Top 3%), 2019.

Services

  • Shadow TPC of CoNEXT 2025
  • Web Chair of ACM SenSys 2024
  • Student Volunteer of IPSN 2024
  • Invited Reviewer for IEEE Internet of Things Journal

Education

Experience