The ten participating students each year will have opportunities to engage in a wide variety of research projects related to smart environments. Here are some examples of the projects that student's can participate over the course of the REU.
Smart Environment Research Testbed
The next generation of smart environments will make use of the unprecedented volume of data collected from sensors and other sources and will analyze and reason about this data in order to make decisions that improve the safety, health and productivity of the inhabitants in the environment. Research in smart environments is critical to develop and transfer the technological resources necessary to bring this vision to fruition. We are creating the Smart Environment Research Testbed (SERT) that will enable research in the area far beyond the capabilities provided by current technology. The SERT infrastructure consists of fixed sensors, smart kiosks, and mobile robots, as well as smartphones and smart watches for sensor data collection and user interaction. The research in sensor stream analysis, learning, robotics and decision-making enabled by this infrastructure will significantly advance the fields related to smart environments, provide outreach opportunities to promote engineering, and create the technologies necessary to replicate such a deployment in other environments. REU students will help develop, evaluate and deploy components of the SERT testbed.
Monitoring and Modeling Human Behavior
The goal of this research area is to design automated tools for monitoring and modeling human behavior by analyzing data that is collected by sensors embedded in buildings (using environmental sensors that monitor motion, door use, temperature, and lighting) or carried by individuals (e.g., smart phones, wearable sensors). Our current work has shown that activities can be effectively recognized from environmental sensors. REU students will refine and enhance machine learning algorithms to discover and recognize behavioral patterns and activity from raw sensor data. Using wearable sensors, we have pioneered work examining the relationship between sleep and everyday function in the older adult population. By combining information provided by smart environments (e.g., total movement, activity recognition) with data collected by wearable sensors (e.g., an actigraph energy-monitoring watch), REU students will design approaches to automatically estimate calorie consumption and total energy expenditure that is noninvasive and is more accurate than self-reporting.
Identify Correlations between Behavior and Health
Smart home technologies that produce continuous data can provide health care professionals with the opportunity to see variability and trends or trajectories, rather than absolute values, and to identify how daily activities impact traditional measures. Data collected through these technologies, combined with symptoms reported by patients in the clinician’s office, could increase health and well-being through early detection and treatment and increase the accuracy of diagnosis. We are collecting data in 40 smart homes with older adults and are performing monthly well-being assessment of the participants. We have already found correlation between observed task quality of these activities and cognitive health of the participants. REU students will design methods of finding latent correlations between these parameters and determining the ecological validity of standard self-report and laboratory-based assessments. REU students will research methods of identifying correlations between human behavior and resource utilization. Students will have access to real-time, whole-building energy usage data for 15,000 buildings in the Pullman area. We will predict energy consumption based on detected individual behavior and will identify clusters of similar consumption patterns based on building features, demographics, and behavior patterns.
Designed Interventions to Promote Aging in Place
In order to keep adults functioning independently at home, the smart environment needs to play a Cognitive Prosthesis role in extending or enhancing their ability to perform everyday activities using automated interventions. One type of intervention that is useful for an individual with limiting conditions is automated reminders. However, identifying the appropriate timing for a reminder is difficult. We used supervised machine learning techniques to identify contexts in which caregivers provide prompts, and used these contexts to train a machine learning algorithm to automate prompts in similar situations. REU students will interact with health care providers and observe caregiver interactions in adult day care and assisted care facilities to see what types of caregiver assistance are naturally provided and in what contexts caregivers typically intervene. A unique component of our proposed REU research is that our multidisciplinary team will investigate uses of the automation that are beneficial for health and sustainability in the smart environments. REU students will design methods that provide interventions to enhance the capabilities of the resident while still encouraging the residents to perform as many tasks for themselves as they are able. Students will also investigate machine learning approaches for giving these intervention methods the ability to adapt to the changing cognitive and physical abilities of the resident. A type of intervention that has been investigated by the community to assist adults with dementia is the use of computer and card games to assess and improve cognitive performance. REU students will participate in a Microsoft-sponsored project to design serious games. The purpose of these games will be to perform automated assessment of cognitive function and to teach compensatory strategy skills for individuals with cognitive impairment.
Integration of Advanced Sensors
We have extensive experience in deploying and utilizing sensors to accomplish recognition and prediction task in smart environments. Crucial to this work is the integration of new advanced sensors, as they become available. Leveraging our new Kinect One sensors and tools developed through our senior capstone projects, REU students will evaluate the state of the art in free-form voice-based communication with smart environments, as well as issues with human computer interaction. Results will be centered on the effectiveness of this kind of interface for residents in smart homes. The Panasonic GridEye sensor produces richer and more complex data than we have dealt with before. REU students would develop new methods to interpret the data and use established algorithms to determine what the sensor is "seeing."
HC-Search for Activity Recognition
Activity Recognition (AR) is a very important task in the context of smart environments. AR can be considered as an instance of structured prediction, where the sensor data sequence corresponds to the structured input, and the activity sequence corresponds to the structured output. HC-Search is a new framework for structured prediction that was shown to perform significantly better than standard approaches including Conditional Random Fields (CRFs) and recurrent sliding window classifiers on a number of benchmark problems. REU students will apply HC - Search for AR using the data from smart home infrastructure at WSU.
Decision-Theoretic Assistants for Smart Homes
We want to build intelligent assistants to help the people (agents) living in a smart home environment. In this project, REU students will leverage and build on the recent work on restricted classes of POMDPs called Hidden Goal MDPs (HGMDPs) and Helper Action MDPs (HAMDPs). The main challenge is that the goals of the agent are hidden from the assistant. Therefore, the assistant has to estimate the goal of the agent and then select assistive action(s) for a given state. We will evaluate the performance of the assistant with and without the prior-knowledge in our smart home infrastructure at WSU.
Interactive Framework for Reinforcement Learning
In our interactive RL framework, we assume the availability of a small amount of demonstration data to bootstrap our learning process, and then the learning agent tries to adapt to the needs of the end-user over time based on explicit user-feedback (e.g., critiquing the decisions taken by the agent) or by asking rich forms of queries (e.g., trajectory preference queries or action preference queries). REU students will evaluate alternate approaches that employ different learning from demonstration algorithms; different query forms; different querying strategies; and different ways of incorporating the user-feedback into the learning process.
Applying Clique Finder for Data Compression
We have developed a fast, parallel, branch-and-bound type algorithm for finding a clique of maximum size in a network. The need for finding maximum cliques arises in many applications ranging from systems biology to machine learning to scientific computing. The goal of this project is to apply the clique finder to a large social network (e.g., a Facebook graph) that will further be used to obtain a compressed representation of the network. Students will conduct research in high-performance graph algorithms and their application.
Network Analysis Tools
Networks have become a worthy modeling language for diverse phenomena in nature and society. Network science is emerging as an interdisciplinary field that seeks to understand the structure and behavior of complex networks and to reason about dynamic processes that take place on them. The goal of this project is to collect some of the most prominent tools available and assess them along several metrics, including capability, ease-of-use, and performance. Students will learn how to select a network analysis tool for a specific need and successfully use the tool to conduct analysis.
Embedded and Pervasive Systems
Embedded and pervasive systems, which utilize wearable and lightweight biosensors, allow for collection and analysis of physiological and contextual information about individuals and provide a means for continuous, real-time, and automated interventions. REU students will be involved in projects that provide them with both a hands-on experience in system integration and an opportunity to explore and address a research problem in biomedical signal processing, pervasive computing, distributed computing, or medical embedded system design. Students will also have a chance to get involved in clinical trials. Furthermore, the interdisciplinary nature of this research allows undergraduate students to participate in meetings with our clinical partners in medicine, nursing, pharmacy, and health sciences. REU students working in this area will be involved in various infrastructure development and hardware/software prototyping efforts related to design, development, and validation of a peripheral edema monitoring system, called Smart-Sock. The project develops a highly wearable sensor platform, embedded in a sock, and accompanying algorithms, software, back-end databases and data analyses techniques to measure lower-limb swelling (i.e., peripheral edema) and report severity of a medical condition. Peripheral edema is one of the early signs of volume overload in the body due to onset or exacerbation of a variety of systemic life threatening diseases. Our pilot application is congestive heart failure and we are currently collaborating with UCLA School of Medicine on this project.
Dynamic Graph Mining
We are developing new methods for pattern learning in dynamic graphs by analyzing only the changes to the dynamic graph, without having to build the entire graph periodically over time. We have developed such methods for streams of additive structural changes. Many approaches to mining graph streams utilize a windowing approach, which analyzes a time-window-based selection of the data stream. REU students will apply this windowing technique, but on the stream of changes, rather than graph snapshots. One promising target for graph mining is sensor data collected from smart environments. We have successfully designed machine learning algorithms to recognize resident activities from this information. While the algorithms typically use summary statistics of sensor events in a feature-vector based approach, there is a rich source of additional information in the relational structure of the event sequence. For example, we can build graphs from motion sensor event sequences, where each node in the graph is a motion sensor, and each edge represents a transition from one sensor to another. REU students will evaluate the use of these graph-based activity signatures to improve the accuracy of activity recognition either by including them as features or performing relational learning based on these structures. We want to scale our sensor data collection to a large collection of ~100 homes and provide longitudinal data that monitors behavioral patterns over multiple years, allowing REU researchers to thus examine behavioral cycles and trends. Finally, we want to include collections of smart environment data for target populations, specifically for older healthy adults and older adults with dementia who can benefit from health and behavior monitoring. The result of this effort will be a massive data collection. Our smart homes generate an average of 5,000 sensor events each day. When smart phone and power meter readings are added, the volume of data will dramatically increase. REU students will design graph-based representations of this data to improve recognition and prediction of activities and energy usage.
Parallel community detection
Graph-based computations are pervasive in a number of scientific applications and one of the most heavily sought after operation is that of identifying community structures within real world networks/graphs. We are working on parallelizing the community detection operation for massive real world networks. This work is focused on developing scalable/parallel algorithms and software that can take advantage of modern day multicore and manycore architectures. We propose to involve undergraduate student researchers to study the peculiarities involved in parallel programming on these emerging architectures, and pass on that learning experience to the project's graduate students who are working on designing efficient algorithms for these platforms. Such a task will also lead to training of the undergraduate student in high performance computing (which is part of an NSF-funded initiative under the TCPP/Parallel and Distributed Computing program).
Topological data analysis
Understanding the structure inherent in data is key to developing advanced analytic functions in domains containing complex data sets. We are investigating approaches that use recent mathematical results from the domain of algebraic topology in analyzing bioinformatics data sets that have shown to be highly complex both by their sheer scale and the number of dimensions they encode. Undergraduate student researchers will help implement the basic core routines of these analysis pipelines and will help graduate students in testing their analysis through data collection. This will also provide a valuable training for the undergraduate researchers in the broad area of data analytics while potentially motivating them to a research based career in this area.
REU students will perform side-by-side comparisons of automated functional assessment with assessment based on self reports and assessment based on laboratory testing. Both healthy older adults and individuals with cognitive impairments have completed scripted everyday activities in our smart home testbed (e.g., make oatmeal, fill a medication dispenser). They have also completed self-report questionnaires about their everyday activities and been assessed with clinical and performance-based measures of everyday activities in our laboratory. Where there are unexpected misalignments in assessment results between the three methods the REU students will further investigate to discover the sources of the inconsistency. The results will allow us to improve questionnaire measures and laboratory-based assessment and to design smart environment algorithms that more thoroughly assess the physical and cognitive well being of individuals.
REU students will experiment with prompt-based interventions that combine cognitive rehabilitation theory with automated interventions to deliver support for cognitive difficulties within the smart environment. REU students will participate in collecting and analyzing data relevant to several questions (a) Are participants more compliant when prompts are delivered during activity transitions rather than being time-based? (b) How can we incorporate motivators into our prompting system to produce high levels of compliance? (c) Do older adults and those with cognitive impairment prefer verbal prompts or multi-modal-prompts? And (d) Do older adults and those with cognitive impairment prefer tablet-based prompts or phone-based prompts? REU students will also have the opportunity to participate in our iterative participatory design studies with users to construct the best interfaces for our prompting interventions and for our digital memory notebook.
Effects of Instruction on Demonstration
Learning from Demonstration (LfD) is a common paradigm for humans to teach robots how to perform complex tasks. However, demonstrators are typically expert users. In order for robots to be more accessible to non-expert users, we are working to improve LfD methods so that they work well with non-expert users. This project will use different sets of instructions to students before they control a UAS. In the first treatment, subjects will be told the UAS is an expensive piece of equipment, while in the second, they will be told it is a toy sold on Amazon (both of which are true). We will measure the difference in task performance between the two groups and investigate how interface design can diminish or exacerbate the difference. The outcome will be to provide insights into human-robot interaction in general, and LfD for non-expert users in particular.
Learning and Sensing for Customized Gym Workouts
Given the recent advances in sensor technology, there are a growing number of high-quality, low-cost body monitors that can accurately detect the motion of a wearer. This project will use time-series machine learning techniques to identify exercises presented by the user, rather than those pre-programmed in the factory. By allowing the user to train the system, she will be able to specify custom or modified exercises that can be automatically recognized during workouts. This will allow the user to generate detailed activity logs of number of sets and repetitions of different exercises, as well as the time between sets and reps. Such data will not only be useful for the individual to track progress towards her goals, but can also be easily reviewed by coaches and trainers to suggest modifications to the workout regime. Eventually, such a system will also be able to provide on-line guidance to the user, such as alerting her that she is not resting long enough between sets or is performing repetitions too quickly.
Unmanned Aerial Systems for Autonomous Health Monitoring of Power Lines
Power lines need regular inspection for physical damage and changing environmental conditions (e.g., foliage overhanging the wires). Power lines also need inspection after a disaster to evaluate the extent of the damage. However, current methods rely on visual inspection via trucks or helicopters and result in significant costs for labor, fuel, and equipment rental. This project will continue work already started by a team of undergraduate students to develop a hybrid software-hardware solution to help automate the inspection process for both transmission and distribution lines. A UAS will automatically follow designated power lines via a combination of vision, altimeter, and GPS. While traveling near the line, visual anomaly detection software will be leveraged to automatically detect abnormalities resulting from physical damage or environmental problems. The UAS will record and analyze the health of the line, GPS tagging any potential problems. The goal of this project is an end-to-end monitoring system that can discover, report, and suggest possible solutions to discovered problems, along with confidence levels in its reasoning.
Cost-aware pattern matching
Networks are routinely queried and manipulated in emerging network analytics. With the growth of size and data complexity, resource consumption (e.g., time, space, energy, power) in large-scale network processing is of growing concern. The need for effective cost-aware graph processing is evident in data-intensive computing over complex networks. This project aims to develop resource-bounded algorithms and optimization techniques for graph pattern matching, i.e., finding similar counterparts of specific patterns as small graphs in a network. This is a class of cornerstone queries widely used in network analysis. The students shall (a) understand the basic graph pattern matching algorithms, (b) design a querying platform that supports user-specified resource bound such as running time or space usage, leveraging the idea of budgeted search, and (c) test the platform over real-life networks and verify the quality of answers. The outcome of the project leads to a network analytics system with flexible performance in accordance to dynamic computing environment.
Benchmarking reachability queries
With the rise of big graph data, there have been various available graph databases (Neo4j, TITAN, RDF3X, etc.). A comparative study is needed of various graph databases for fundamental queries such as reachability, i.e., finding the connectivity pattern between two sets of nodes in a network. Apart from conventional relational database benchmarks (e.g., TPC-H), a graph benchmarking system should fairly measure the performance of graph systems with query load, data storage and experimental settings. In this project, students shall first get familiar with two state-of-the-art graph databases (Neo4j and TITAN), and learn to develop algorithms for reachability queries using the built-in functions. They shall then build a benchmarking platform for benchmarking the performance of reachability query classes. REU students will learn the basics of graph database and benchmark design, and the pros and cons of state-of-the-art graph processing techniques. Further development of the benchmarking system shall enable integrated graph database systems for effective network analytics. Graph databases (e.g., Neo4j, TITAN) and analysis software (Gephi) will be provided with installation guidelines.