Project Theme 1: Pervasive Sensing

Vibrations of Everything

Physical​ and physiological information is essential for indoor human monitoring applications. In order to capture such information non-intrusively, we utilize/design sensors that can capture the vibration induced by people/body parts to infer a variance of activities (over different scales, both infrastructure and wearable) in the physical world. We explore new opportunities to utilize complementary information over different scales (on-body, room, city, etc.) to achieve accurate inference.

In-Home Activities of Daily Living (ADL) Monitoring

Latest paper:
Zhizhang Hu, Yue Zhang, Tong Yu, and Shijia Pan. "VMA: Domain Variance-and Modality-Aware Model Transfer for Fine-Grained Occupant Activity Recognition." In 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), pp. 259-270. IEEE, 2022.

Retainer-Formed Sensor Capturing Oral Activities-Induced Vibration

Latest paper:
Zhizhang Hu, Amirmohammad Radmehr, Yue Zhang, Shijia Pan, and Phuc Nguyen. "IOTeeth: Intra-Oral Teeth Sensing System for Dental Occlusal Diseases Recognition." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, no. 1 (2024): 1-29.

Shijia Pan, Dong Yoon Lee, Jun Ho Lee, and Phuc Nguyen. "TeethVib: Monitoring Teeth Functional Occlusion Through Retainer Vibration Sensing." In 2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 92-96. IEEE, 2021.

Repurpose Vibrotactile Intervention for Fine-grained Muscle Profiling 

Paper:
Shubham Rohal, Shreya Shriram, V. P. Nguyen, and Shijia Pan. "FinePose: Fine-Grained Postural Muscle Profiling via Haptic Vibration Signals." In Proceedings of the 2022 Workshop on Body-centric Computing Systems, pp. 19-24. 2022.

🏆 Best Poster Award IPSN'22

🚀 Alumni Spin-off: PosTrue

Structure-Augmented Intelligent Surfaces

To push the boundary of ambient information inference via surface sensing, we further explore methods and apparatus for augmenting the ability to acquire and recognize human-induced surface deformation by changing the physical properties of the surface.

Augment rigid surface with LEGO® bricks enabled metasurface

Latest  paper:
Yue Zhang, Shikha Patel, Dong Yoon Lee, Paolo Celli, Amelie Bonde, and Shijia Pan. 2023. Poster Abstract: LEVO: LEGO® Bricks Enhanced Single-Point Vibration Sensing for Occupant Monitoring. In The 21st ACM Conference on Embedded Networked Sensor Systems (SenSys ’23), November 12–17, 2023, Istanbul, Turkiye. ACM, New York, NY, USA, 2 pages.

🏆 Best Poster Award SenSys'23

Augment elastic surface with Miura-Ori Origami metastructure

Latest  paper:
Shubham Rohal, Carlos Ruiz, Yue Zhang, and Shijia Pan. "MOOCA: Muira-Ori Origami-Based Configurable Shelf-Liner for Autonomous Retail." In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 776-777. 2022.

🏆 Best Demo Award SenSys'22

Augment rigid surface with controlled propagation path

Latest  paper:
Yue Zhang, Carlos Ruiz, Shubham Rohal, and Shijia Pan. "CPA: Cyber-Physical Augmentation for Vibration Sensing in Autonomous Retails." In Proceedings of the 24th International Workshop on Mobile Computing Systems and Applications, pp. 8-14. 2023.

Project Theme 2: Autonomous Adaptation via Cross-Modal Intelligence

Cross-Modal Association

Multiple co-located sensing modalities capture the same physical event from different aspects and provide complementary information for inference. However, to accurately associate signals from different modality capturing the same event is challenging, expecially when the modalities are significantly different.

Cross-Modal Association Using Humans as Shared Context

Paper:
Yue Zhang, Zhizhang Hu, Uri Berger, and Shijia Pan. "CMA: Cross-Modal Association Between Wearable and Structural Vibration Signal Segments for Indoor Occupant Sensing." In The 22nd International Conference on Information Processing in Sensor Networks, pp. 96-109. 2023.

Yue Zhang, Zhizhang Hu, Uri Berger, and Shijia Pan. "Poster Abstract: Integrating On-and Off-body Sensing for Young Adults Failure to Launch (FTL) Behavior Profiling." In Proceedings of the 22nd International Conference on Information Processing in Sensor Networks, pp. 320-321. 2023.

🏆 Best Poster Award Runner Up IPSN'23

Yue Zhang, Zhizhang Hu, Susu Xu, and Shijia Pan. "AutoQual: Task-Oriented Structural Vibration Sensing Quality Assessment Leveraging Co-located Mobile Sensing Context." CCF Transactions on Pervasive Computing and Interaction 3 (2021): 378-396.

Adaptation in Sensing

Data acquisition quality directly affects the information representative and the model accuracy in applications of real-world deployed cyber-physical systems. We combine physical and data-driven knowledge to design metrics and methods to assess the signal and dataset quality for particular sensing tasks.

Cross-Modal IoT System Auto Configuration

Paper:
Shubham Rohal, Yue Zhang, Carlos Ruiz, and Shijia Pan. "AutoLoc: Autonomous Sensor Location Configuration via Cross-Modal Sensing." Frontiers in Big Data 5 (2022): 835949.

Yu, Tong, Yue Zhang, Zhizhang Hu, Susu Xu, and Shijia Pan. "Vibration-based indoor human sensing quality reinforcement via thompson Sampling." In Proceedings of the First International Workshop on Cyber-Physical-Human System Design and Implementation, pp. 33-38. 2021.

Lixing He, Carlos Ruiz, Mostafa Mirshekari, and Shijia Pan. "SCSV2: physics-informed self-configuration sensing through vision and vibration context modeling." In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 532-537. 2020.

Adaptation in Learning

The heterogeneity of cyber-physical systems brings challenges and opportunities for various applications. Our goal is to explore new methods to combine the complementary characteristics of multiple sensing modalities for accurate fine-grained learning with limited (labeled) data.

Domain Variance- and Modality-Aware Model Transfer

Related  paper:
Zhizhang Hu, Yue Zhang, Tong Yu, and Shijia Pan. "VMA: Domain Variance-and Modality-Aware Model Transfer for Fine-Grained Occupant Activity Recognition." In 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), pp. 259-270. IEEE, 2022.

Project Theme 3: Networked Collaborative Intelligence

AIoT Enabled Augmented Air Quality Monitoring

Coming Soon!

Papers:
Zhizhang Hu∗, Shangjie Du∗, Yuning Chen, Xuan Zhang, Wan Du, Asa Bradman, and Shijia Pan. 2023. Poster Abstract: Enhancing Fault Resilience of Air Quality Monitoring in San Joaquin Valley: A Data Equity Analysis. In The 21st ACM Conference on Embedded Networked Sensor Systems (SenSys ’23), November 12–17, 2023, Istanbul, Turkiye. ACM, New York, NY, USA, 2 pages.

Wang, Gang, Shijia Pan, and Susu Xu. "Decoupling the unfairness propagation chain in crowd sensing and learning systems for spatio-temporal urban monitoring." In Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pp. 200-203. 2021.

Smart Farm Data Quality Assessment via Spatial Consistency Modeling

Coming Soon!

Papers:
Yue Zhang, Abdias Tellez Benitez, Reza Ehsani, and Shijia Pan. "DaQual: Data Quality Assessment for Tree Trunk Relative Water Content Sensors in a Pomegranate Orchard." In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 1096-1101. 2022.