Project Theme 1: Pervasive Sensing
Vibrations of Everything
Physical and physiological information is essential for indoor human monitoring applications. In order to capture such information non-intrusively, we utilize/design sensors that can capture the vibration induced by people/body parts to infer a variance of activities (over different scales, both infrastructure and wearable) in the physical world. We explore new opportunities to utilize complementary information over different scales (on-body, room, city, etc.) to achieve accurate inference.
In-Home Activities of Daily Living (ADL) Monitoring
Latest paper:
Zhizhang Hu, Yue Zhang, Tong Yu, and Shijia Pan. "VMA: Domain Variance-and Modality-Aware Model Transfer for Fine-Grained Occupant Activity Recognition." In 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), pp. 259-270. IEEE, 2022.
Retainer-Formed Sensor Capturing Oral Activities-Induced Vibration
Latest paper:
Zhizhang Hu, Amirmohammad Radmehr, Yue Zhang, Shijia Pan, and Phuc Nguyen. "IOTeeth: Intra-Oral Teeth Sensing System for Dental Occlusal Diseases Recognition." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, no. 1 (2024): 1-29.
Shijia Pan, Dong Yoon Lee, Jun Ho Lee, and Phuc Nguyen. "TeethVib: Monitoring Teeth Functional Occlusion Through Retainer Vibration Sensing." In 2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 92-96. IEEE, 2021.
Repurpose Vibrotactile Intervention for Fine-grained Muscle Profiling
Paper:
Shubham Rohal, Shreya Shriram, V. P. Nguyen, and Shijia Pan. "FinePose: Fine-Grained Postural Muscle Profiling via Haptic Vibration Signals." In Proceedings of the 2022 Workshop on Body-centric Computing Systems, pp. 19-24. 2022.
🏆 Best Poster Award IPSN'22
🚀 Alumni Spin-off: PosTrue
Structure-Augmented Intelligent Surfaces
To push the boundary of ambient information inference via surface sensing, we further explore methods and apparatus for augmenting the ability to acquire and recognize human-induced surface deformation by changing the physical properties of the surface.
Augment rigid surface with LEGO® bricks enabled metasurface
Latest paper:
Yue Zhang, Shikha Patel, Dong Yoon Lee, Paolo Celli, Amelie Bonde, and Shijia Pan. 2023. Poster Abstract: LEVO: LEGO® Bricks Enhanced Single-Point Vibration Sensing for Occupant Monitoring. In The 21st ACM Conference on Embedded Networked Sensor Systems (SenSys ’23), November 12–17, 2023, Istanbul, Turkiye. ACM, New York, NY, USA, 2 pages.
🏆 Best Poster Award SenSys'23
Augment elastic surface with Miura-Ori Origami metastructure
Latest paper:
Shubham Rohal, Carlos Ruiz, Yue Zhang, and Shijia Pan. "MOOCA: Muira-Ori Origami-Based Configurable Shelf-Liner for Autonomous Retail." In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 776-777. 2022.
🏆 Best Demo Award SenSys'22
Augment rigid surface with controlled propagation path
Latest paper:
Yue Zhang, Carlos Ruiz, Shubham Rohal, and Shijia Pan. "CPA: Cyber-Physical Augmentation for Vibration Sensing in Autonomous Retails." In Proceedings of the 24th International Workshop on Mobile Computing Systems and Applications, pp. 8-14. 2023.
Project Theme 2: Autonomous Adaptation via Cross-Modal Intelligence
Cross-Modal Association
Multiple co-located sensing modalities capture the same physical event from different aspects and provide complementary information for inference. However, to accurately associate signals from different modality capturing the same event is challenging, expecially when the modalities are significantly different.
Cross-Modal Association Using Humans as Shared Context
Paper:
Yue Zhang, Zhizhang Hu, Uri Berger, and Shijia Pan. "CMA: Cross-Modal Association Between Wearable and Structural Vibration Signal Segments for Indoor Occupant Sensing." In The 22nd International Conference on Information Processing in Sensor Networks, pp. 96-109. 2023.
Yue Zhang, Zhizhang Hu, Uri Berger, and Shijia Pan. "Poster Abstract: Integrating On-and Off-body Sensing for Young Adults Failure to Launch (FTL) Behavior Profiling." In Proceedings of the 22nd International Conference on Information Processing in Sensor Networks, pp. 320-321. 2023.
🏆 Best Poster Award Runner Up IPSN'23
Yue Zhang, Zhizhang Hu, Susu Xu, and Shijia Pan. "AutoQual: Task-Oriented Structural Vibration Sensing Quality Assessment Leveraging Co-located Mobile Sensing Context." CCF Transactions on Pervasive Computing and Interaction 3 (2021): 378-396.
Adaptation in Sensing
Data acquisition quality directly affects the information representative and the model accuracy in applications of real-world deployed cyber-physical systems. We combine physical and data-driven knowledge to design metrics and methods to assess the signal and dataset quality for particular sensing tasks.
Cross-Modal IoT System Auto Configuration
Paper:
Shubham Rohal, Yue Zhang, Carlos Ruiz, and Shijia Pan. "AutoLoc: Autonomous Sensor Location Configuration via Cross-Modal Sensing." Frontiers in Big Data 5 (2022): 835949.
Yu, Tong, Yue Zhang, Zhizhang Hu, Susu Xu, and Shijia Pan. "Vibration-based indoor human sensing quality reinforcement via thompson Sampling." In Proceedings of the First International Workshop on Cyber-Physical-Human System Design and Implementation, pp. 33-38. 2021.
Lixing He, Carlos Ruiz, Mostafa Mirshekari, and Shijia Pan. "SCSV2: physics-informed self-configuration sensing through vision and vibration context modeling." In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 532-537. 2020.
Adaptation in Learning
The heterogeneity of cyber-physical systems brings challenges and opportunities for various applications. Our goal is to explore new methods to combine the complementary characteristics of multiple sensing modalities for accurate fine-grained learning with limited (labeled) data.
Domain Variance- and Modality-Aware Model Transfer
Related paper:
Zhizhang Hu, Yue Zhang, Tong Yu, and Shijia Pan. "VMA: Domain Variance-and Modality-Aware Model Transfer for Fine-Grained Occupant Activity Recognition." In 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), pp. 259-270. IEEE, 2022.
Project Theme 3: Networked Collaborative Intelligence
AIoT Enabled Augmented Air Quality Monitoring
Coming Soon!
Papers:
Zhizhang Hu∗, Shangjie Du∗, Yuning Chen, Xuan Zhang, Wan Du, Asa Bradman, and Shijia Pan. 2023. Poster Abstract: Enhancing Fault Resilience of Air Quality Monitoring in San Joaquin Valley: A Data Equity Analysis. In The 21st ACM Conference on Embedded Networked Sensor Systems (SenSys ’23), November 12–17, 2023, Istanbul, Turkiye. ACM, New York, NY, USA, 2 pages.
Wang, Gang, Shijia Pan, and Susu Xu. "Decoupling the unfairness propagation chain in crowd sensing and learning systems for spatio-temporal urban monitoring." In Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pp. 200-203. 2021.
Smart Farm Data Quality Assessment via Spatial Consistency Modeling
Coming Soon!
Papers:
Yue Zhang, Abdias Tellez Benitez, Reza Ehsani, and Shijia Pan. "DaQual: Data Quality Assessment for Tree Trunk Relative Water Content Sensors in a Pomegranate Orchard." In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 1096-1101. 2022.