Mobile Systems

Adams, Roy, Nazir Saleheen, Edison Thomaz, Abhinav Parate, Santosh Kumar, and Benjamin Marlin. "Hierarchical Span-Based Conditional Random Fields for Labeling and Segmenting Events in Wearable Sensor Data Streams." International Conference on Machine Learning. 2016. Abstracticml2016_hns.pdf

The field of mobile health (mHealth) has the potential to yield new insights into health and behavior through the analysis of continuously recorded data from wearable health and activity sensors. In this paper, we present a hierarchical span-based conditional random field model for the key problem of jointly detecting discrete events in such sensor data streams and segmenting these events into high-level activity sessions. Our model includes higher-order cardinality factors and inter-event duration factors to capture domain-specific structure in the label space. We show that our model supports exact MAP inference in quadratic time via dynamic programming, which we leverage to perform learning in the structured support vector machine framework. We apply the model to the problems of smoking and eating detection using four real data sets. Our results show statistically significant improvements in segmentation performance at the p=0.005 level relative to a hierarchical pairwise CRF.

Dadkhahi, Hamid, Nazir Saleheen, Santosh Kumar, and Benjamin Marlin. "Learning Shallow Detection Cascades for Wearable Sensor-Based Mobile Health Applications." ICML On Device Intelligence Workshop. 2016. Abstractdadkhahi-icml-odi2017.pdf

The field of mobile health aims to leverage recent advances in wearable on-body sensing technology and smart phone computing capabilities to develop systems that can monitor health states and deliver just-in-time adaptive interventions. However, existing work has largely focused on analyzing collected data in the off-line setting. In this paper, we propose a novel approach to learning shallow detection cascades developed explicitly for use in a real-time wearable-phone or wearable-phone-cloud systems. We apply our approach to the problem of cigarette smoking detection from a combination of wrist-worn actigraphy data and respiration chest band data using two and three stage cascades.

Nguyen, Thai, Roy J. Adams, Annamalai Natarajan, and Benjamin M. Marlin Parsing Wireless Electrocardiogram Signals with the CRF-CFG Model. Conference on Uncertainty in Artificial Intelligence Machine Learning for Health Workshop., 2016. Abstractnguyen-uai-health2016.pdf

Recent advances in wearable sensor technology have made it possible to simultaneously collect multiple streams of physiological and context data from individuals as they go about their daily activities in natural environments. However, extracting reliable higher-level inferences from these raw data streams remains a key data analysis challenge. In this paper, we focus on the specific case of the analysis of data from wireless electrocardiogram (ECG) sensors. We present a new robust probabilistic approach to ECG morphology extraction using conditional random field context free grammar models, which have traditionally been applied to parsing problems in natural language processing. We introduce a robust context free grammar for parsing noisy ECG data, and show significantly improved performance on the ECG morphological labeling task.

Mayberry, Addison, Yamin Tun, Pan Hu, Duncan SmithFreedman, Deepak Ganesan, Benjamin Marlin, and Christopher Salthouse CIDER: Enabling RobustnessPower Tradeoffs on a Computational Eyeglass. 21st Annual International Conference on Mobile Computing and Networking., 2015. Abstractcider.pdf

The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. The challenges lie in: a) handling the complexity of continuous high-rate sensing from a camera and processing the image stream to estimate eye parameters, and b) dealing with the wide variability in illumination conditions in the natural environment. This paper explores the power--robustness tradeoffs inherent in the design of a wearable eye tracker, and proposes a novel staged architecture that enables graceful adaptation across the spectrum of real-world illumination. We propose, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared, b) error in estimating pupil center and pupil dilation, and c) model training procedures that involve zero effort from a user. We demonstrate that the system can estimate pupil center with error less than two pixels, and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.

Saleheen, Nazir, Amin Ali, Syed Monowar Hossain, Hillol Sarker, Soujanya Chatterjee, Benjamin Marlin, Emre Ertin, Mustafa al'Absi, and Santosh Kumar puffMarker : A Multi-Sensor Approach for Pinpointing the Timing of First Lapse in Smoking Cessation. 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing., 2015. Abstractpuff-marker.pdf

Smoking is the leading cause of preventable deaths. Mobile technologies can help to deliver just-in-time-interventions to abstinent smokers and assist them in resisting urges to lapse. Doing so, however, it requires identification of high-risk situations that may lead an abstinent smoker to relapse. In this paper, we propose an explainable model for detecting smoking lapses in newly abstinent smokers using respiration and 6-axis inertial sensors worn on wrists. We propose a novel method by identifying windows of data that represent the hand at the mouth. We then develop a model to classify into puff or non-puff. On the training data, the model achieves a recall rate of 98%, for a FP rate of 1.5%. When the model is applied to the data collected from 13 abstainers, the false positive rate is 0.3/hour. Among 15 lapsers, the model is able to pinpoint the timing of first lapse in 13 participants.

Adams, Roy J., Edison Thomaz, and Benjamin M. Marlin. "Hierarchical Nested CRFs for Segmentation and Labeling of Physiological Time Series." NIPS Workshop on Machine Learning in Healthcare. 2015. Abstractadams-nips-heath2015.pdf

In this paper, we address the problem of nested hierarchical segmentation
and labeling of time series data. We present a hierarchical
span-based conditional random field framework for this problem that
leverages higher-order factors to enforce the nesting constraints. The framework can
incorporate a variety of additional factors including higher order cardinality
factors. This research is motivated by hierarchical activity recognition problems
in the field of mobile Health (mHealth). We show that the specific model of interest in the mHealth setting supports exact MAP inference in quadratic time. Learning is accomplished in the structured support vector machine framework. We show positive results on real and synthetic data sets.

Natarajan, Annamalai, Edward Gaiser, Gustavo Angarita, Robert Malison, Deepak Ganesan, and Benjamin Marlin. "Conditional Random Fields for Morphological Analysis of Wireless ECG Signals." Proceedings of the 5th Annual conference on Bioinformatics, Computational Biology and Health Informatics. Newport Beach, CA: ACM, 2014. Abstractcrf_bcb2014.pdf

Thanks to advances in mobile sensing technologies, it has recently become practical to deploy wireless electrocardiograph sensors for continuous recording of ECG signals. This capability has diverse applications in the study of human health and behavior, but to realize its full potential, new computational tools are required to effectively deal with the uncertainty that results from the noisy and highly non-stationary signals collected using these devices. In this work, we present a novel approach to the problem of extracting the morphological structure of ECG signals based on the use of dynamically structured conditional random field (CRF) models. We apply this framework to the problem of extracting morphological structure from wireless ECG sensor data collected in a lab-based study of habituated cocaine users. Our results show that the proposed CRF-based approach significantly out-performs independent prediction models using the same features, as well as a widely cited open source toolkit.

Parate, Abhinav, Matthias Böhmer, David Chu, Deepak Ganesan, and Benjamin M. Marlin. "Practical prediction and prefetch for faster access to applications on mobile phones." UbiComp. 2013. 275-284. Abstractprefetch_ubicomp13_paper.pdf

Mobile phones have evolved from communication devices to indispensable accessories with access to real-time content. The increasing reliance on dynamic content comes at the cost of increased latency to pull the content from the Internet before the user can start using it. While prior work has explored parts of this problem, they ignore the bandwidth costs of prefetching, incur significant training overhead, need several sensors to be turned on, and do not consider practical systems issues that arise from the limited background processing capability supported by mobile operating systems. In this paper, we make app prefetch practical on mobile phones. Our contributions are two-fold. First, we design an app prediction algorithm, APPM, that requires no prior training, adapts to usage dynamics, predicts not only which app will be used next but also when it will be used, and provides high accuracy without requiring additional sensor context. Second, we perform parallel prefetch on screen unlock, a mechanism that leverages the benefits of prediction while operating within the constraints of mobile operating systems. Our experiments are conducted on long-term traces, live deployments on the Android Play Market, and user studies, and show that we outperform prior approaches to predicting app usage, while also providing practical ways to prefetch application content on mobile phones.

Parate, Abhinav, Meng-Chieh Chiu, Deepak Ganesan, and Benjamin M. Marlin. "Leveraging graphical models to improve accuracy and reduce privacy risks of mobile sensing." MobiSys. 2013. 83-96. Abstractsensing_mobisys13_paper.pdf

The proliferation of sensors on mobile phones and wearables has led to a plethora of context classifiers designed to sense the individual's context. We argue that a key missing piece in mobile inference is a layer that fuses the outputs of several classifiers to learn deeper insights into an individual's habitual patterns and associated correlations between contexts, thereby enabling new systems optimizations and opportunities. In this paper, we design CQue, a dynamic bayesian network that operates over classifiers for individual contexts, observes relations across these outputs across time, and identifies opportunities for improving energy-efficiency and accuracy by taking advantage of relations. In addition, such a layer provides insights into privacy leakage that might occur when seemingly innocuous user context revealed to different applications on a phone may be combined to reveal more information than originally intended. In terms of system architecture, our key contribution is a clean separation between the detection layer and the fusion layer, enabling classifiers to solely focus on detecting the context, and leverage temporal smoothing and fusion mechanisms to further boost performance by just connecting to our higher-level inference engine. To applications and users, CQue provides a query interface, allowing a) applications to obtain more accurate context results while remaining agnostic of what classifiers/sensors are used and when, and b) users to specify what contexts they wish to keep private, and only allow information that has low leakage with the private context to be revealed. We implemented CQue in Android, and our results show that CQue can i) improve activity classification accuracy up to 42%, ii) reduce energy consumption in classifying social, location and activity contexts with high accuracy(>90%) by reducing the number of required classifiers by at least 33%, and iii) effectively detect and suppress contexts that reveal private information.