Abstract
In pervasive and ubiquitous computing systems, human activity recognition has immense potential in a large number of application domains. Current activity recognition techniques (i) do not handle variations in sequence, concurrency and interleaving of complex activities; (ii) do not incorporate context; and (iii) require large amounts of training data. There is a lack of a unifying theoretical framework which exploits both domain knowledge and data-driven observations to infer complex activities. In this article, we propose, develop and validate a novel Context-Driven Activity Theory (CDAT) for recognizing complex activities. We develop a mechanism using probabilistic and Markov chain analysis to discover complex activity signatures and generate complex activity definitions. We also develop a Complex Activity Recognition (CAR) algorithm. It achieves an overall accuracy of 95.73% using extensive experimentation with real-life test data. CDAT utilizes context and links complex activities to situations, which reduces inference time by 32.5% and also reduces training data by 66%.
- Robert P. Abelson. 1981. Psychological status of the script concept. American Psychologist 36, 7 (1981), 715--729.Google ScholarCross Ref
- Gregory D. Abowd and Elizabeth D. Mynatt. 2000. Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput.-Hum. Interact. 7, 1 (March 2000), 29--58. DOI: http://dx.doi.org/10.1145/344949.344988. Google ScholarDigital Library
- Fahd Albinali, Nigel Davies, and Adrian Friday. 2007. Structural learning of activities from sparse datasets. In Proceedings of the 5th International Conference on Pervasive Computing and Communications (PerCom'07). 221--228. Google ScholarDigital Library
- Oliver Amft and Gerhard Troster. 2008. Recognition of dietary activity events using on-body sensors. Artif. Intell. Med. 42, 2 (2008), 121--136. 0933-3657 Google ScholarDigital Library
- Android-Platform. 2010. Android Platform. Retrieved May 17, 2010 from http://www.android.com/.Google Scholar
- Ling Bao. 2003. Physical Activity Recognition from Acceleration Data under Semi-Naturalistic Conditions. Ph.D. dissertation. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science.Google Scholar
- L. Bao and S. Intille. 2004. Activity Recognition from User-Annotated Acceleration data. In Proc. Pervasive. 1--17.Google Scholar
- Roland Bennett. 2007. Time Tracker: PC Activity Monitor for Windows. Retrieved May 17, 2010 from http://ttracker.sourceforge.net/.Google Scholar
- O. Brdiczka, J. Crowley, and P. Reignier. 2007. Learning situation models for providing context-aware services. In Universal Access in Human-Computer Interaction. Ambient Interaction, Constantine Stephanidis (Ed.). Lecture Notes in Computer Science, Vol. 4555. Springer, Berlin, 23--32. Google ScholarDigital Library
- Oliver Brdiczka, Norman Makoto Su, and James Bo Begole. 2010. Temporal task footprinting: Identifying routine tasks by their temporal patterns. In Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI'10). ACM, New York, NY, 281--284. DOI: http://dx.doi.org/10.1145/1719970.1720011. Google ScholarDigital Library
- Liming Chen, C. D. Nugent, and Hui Wang. 2012. A knowledge-driven approach to activity recognition in smart homes. IEEE Transactions on Knowledge and Data Engineering 24, 6 (June 2012), 961--974. 1041-4347 DOI: http://dx.doi.org/10.1109/TKDE.2011.51. Google ScholarDigital Library
- T. Choudhury, S. Consolvo, B. Harrison, J. Hightower, A. LaMarca, L. LeGrand, A. Rahimi, A. Rea, G. Bordello, B. Hemingway, P. Klasnja, K. Koscher, J. A. Landay, J. Lester, D. Wyatt, and D. Haehnel. 2008. The mobile sensing platform: An embedded activity recognition system. Pervasive Computing, IEEE 7, 2 (2008), 32--41. Google ScholarDigital Library
- Sunny Consolvo, David W. McDonald, Tammy Toscos, Mike Y. Chen, Jon Froehlich, Beverly Harrison, Predrag Klasnja, Anthony LaMarca, Louis LeGrand, Ryan Libby, Ian Smith, and James A. Landay. 2008. Activity sensing in the wild: a field trial of ubifit garden. In Proceedings of the 26th Annual SIGCHI Conference on Human Factors in Computing Systems (CHI'08). ACM, New York, NY, 1797--1806. DOI: http://dx.doi.org/10.1145/1357054.1357335. Google ScholarDigital Library
- Nigel Davies, Daniel P. Siewiorek, and Rahul Sukthankar. 2008. Activity-based computing. Pervasive Computing, IEEE 7, 2 (2008), 20--21. Google ScholarDigital Library
- J. W. Deng and H. T. Tsui. 2000. An HMM-based approach for gesture segmentation and recognition. In Proceedings of the 15th International Conference on Pattern Recognition, Vol. 3. 679--682. Google ScholarDigital Library
- Anind Kumar Dey. 2000. Providing Architectural Support for Building Context-Aware Applications. Ph.D. Dissertation. Georgia Institute of Technology.Google Scholar
- GreenerBuildings. 2010. GreenerBuildings. Retrieved May 17, 2010 from http://www.greenerbuildings. eu/sites/greenerbuildings.eu/files/GreenerBuildingsFiche.pdf.Google Scholar
- G. Guerra-Filho and Y. Aloimonos. 2007. A language for human action. Computer 40, 5 (2007), 42--51. Google ScholarDigital Library
- R. Hamid, S. Maddi, A. Bobick, and M. Essa. 2007. Structure from statistics—unsupervised activity analysis using suffix trees. In Proceedings of the 11th International Conference on Computer Vision (ICCV'07). 1--8.Google Scholar
- Rim Helaoui, Mathias Niepert, and Heiner Stuckenschmidt. 2011a. Recognizing interleaved and concurrent activities: A statistical-relational approach. In Proceedings of the 2011 IEEE International Conference on Pervasive Computing and Communications (PerCom'11). IEEE Computer Society, 1--9. Google ScholarDigital Library
- Rim Helaoui, Mathias Niepert, and Heiner Stuckenschmidt. 2011b. Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships. Pervasive and Mobile Computing 7, 6 (2011), 660--670. DOI: http://dx.doi.org/10.1016/j.pmcj.2011.08.004. Google ScholarDigital Library
- I. Homed, A. Misra, M. Ebling, and W. Jerome. 2008. HARMONI: Context-aware filtering of sensor data for continuous remote health monitoring. In Proceedings of the 6th International Conference on Pervasive Computing and Communications (PerCom'08). 248--251. Google ScholarDigital Library
- Xin Hong, Chris Nugent, Maurice Mulvenna, Sally McClean, Bryan Scotney, and Steven Devlin. 2009. Evidential fusion of sensor data for activity recognition in smart homes. Pervasive and Mobile Computing 5, 3 (2009), 236--252. DOI: http://dx.doi.org/10.1016/j.pmcj.2008.05.002. Google ScholarDigital Library
- Derek Hao Hu and Qiang Yang. 2008. CIGAR: Concurrent and interleaving goal and activity recognition. In Proceedings of the 23rd National Conference on Artificial Intelligence, Volume 3. AAAI Press, Chicago, Illinois. Google ScholarDigital Library
- T. Huynh and B. Schiele. 2006. Unsupervised discovery of structure in activity data using multiple eigenspaces. In Proceedings of the International Conference on Location- and Context-Awareness (LoCA'06). 151--167. Google ScholarDigital Library
- E. Jovanov, A. O'Donnell Lords, D. Raskovic, P. G. Cox, R. Adhami, and F. Andrasik. 2003. Stress monitoring using a distributed wireless intelligent sensor system. IEEE Engineering in Medicine and Biology Magazine 22, 3 (2003), 49--55.Google ScholarCross Ref
- V. Kaptelinin, BA Nardi, and C Macaulay. 1999. Methods and tools: The activity checklist: A tool for representing the space of context. interactions 6, 4 (1999), 27--39. Google ScholarDigital Library
- T. Kasteren, G. Englebienne, and B. Krose. 2010. Transferring knowledge of activity recognition across sensor networks. In Pervasive Computing, Vol. 6030. Springer Berlin, 283--300. DOI: http://dx.doi.org/10.1007/978-3-642-12654-3_17. Google ScholarDigital Library
- Tim van Kasteren, Athanasios Noulas, Gwenn Englebienne, and Ben Krose. 2008. Accurate activity recognition in a home setting. In Proceedings of the 10th International Conference on Ubiquitous Computing (Ubicomp'08). ACM, Seoul, Korea. Google ScholarDigital Library
- Young-Seol Lee and Sung-Bae Cho. 2011. Activity recognition using hierarchical hidden Markov models on a smartphone with 3D accelerometer. In Hybrid Artificial Intelligent Systems. Lecture Notes in Computer Science, Vol. 6678. Springer, Berlin, 460--467. DOI: http://dx.doi.org/10.1007/978-3-642-21219-2_58. Google ScholarDigital Library
- Ian Li, Anind K. Dey, and Jodi Forlizzi. 2012. Using context to reveal factors that affect physical activity. ACM Trans. Comput.-Hum. Interact. 19, 1, Article 7 (May 2012), 21 pages. DOI: http://dx.doi.org/ 10.1145/2147783.2147790. Google ScholarDigital Library
- L. Liao, T. Choudhury, D. Fox, and H. Kautz. 2007. Training conditional random fields using virtual evidence boosting. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI'07). 2530--2535. Google ScholarDigital Library
- Paul Lukowicz, Jamie A. Ward, Holger Junker, Mathias Stäger, Gerhard Tröster, Amin Atrash, and Thad Starner. 2004. Recognizing workshop activity using body worn microphones and accelerometers. In Pervasive Computing. 18--32.Google Scholar
- Sven Meyer and Andry Rakotonirainy. 2003. A survey of research on context-aware homes. In Proceedings of the Australasian Information Security Workshop Conference on ACSW Frontiers 2003, Volume 21. 159--168. Google ScholarDigital Library
- S. P. Meyn and R. L. Tweedie. 1993. Markov Chains and Stochastic Stability. Springer-Verlag.Google Scholar
- D. Minnen, T. Starner, M. Essa, and C. Isbell. 2006. Discovering characteristic actions from on-body sensor data. In Proceedings of the 10th IEEE International Symposium on Wearable Computers (ISWC'06). 11--18.Google Scholar
- Mulle-v3. 2010. Mulle v3 Sensor Platform. Retrieved May 17, 2010 from http://www.eistec.se/.Google Scholar
- D. Ocone. 2010. Lecture Notes in Discrete and Probabilistic Models in Biology: Markov Chain Models. Retrieved May 17, 2010 from http://www.math.rutgers.edu/courses/338/coursenotes/chapter5.pdf.Google Scholar
- Nuria Oliver, Eric Horvitz, and Ashutosh Garg. 2002. Layered representations for human activity recognition. In Proceedings of the 4th IEEE International Conference on Multimodal Interfaces (ICMI'02). IEEE Computer Society. Google ScholarDigital Library
- Amir Padovitz, Seng Wai Loke, and Arkady Zaslavsky. 2004. Towards a theory of context spaces. In Proceedings of the 2nd IEEE Annual Conference on Pervasive Computing and Communications Workshops (PerCom'04). Google ScholarDigital Library
- Amir Padovitz, Seng Wai Loke, and Arkady B. Zaslavsky. 2008. Multiple-agent perspectives in reasoning about situations for context-aware pervasive computing systems. IEEE Transactions on Systems, Man, and Cybernetics, Part A 38, 4 (2008), 729--742. Google ScholarDigital Library
- Etienne Pardoux. 2008. Markov Processes and Applications: Algorithms, Networks, Genome and Finance. John Wiley and Sons Ltd.Google Scholar
- M. Philipose, K. P. Fishkin, M. Perkowitz, D. J. Patterson, D. Fox, H. Kautz, and D. Hahnel. 2004. Inferring activities from interactions with objects. Pervasive Computing, IEEE 3, 4 (2004), 50--57. Google ScholarDigital Library
- P. Rashidi, D. Cook, L. Holder, and M. Schmitter-Edgecombe. 2011. Discovering activities to recognize and track in a smart environment. IEEE Transactions on Knowledge and Data Engineering 23, 4 (2011), 527--539. Google ScholarDigital Library
- Parisa Rashidi and Diane J. Cook. 2011. Activity knowledge transfer in smart environments. Pervasive and Mobile Computing 7, 3 (2011), 331--343. Google ScholarDigital Library
- Daniele Riboni and Claudio Bettini. 2009. Context-aware activity recognition through a combination of ontological and statistical reasoning. In Proceedings of Conference on Ubiquitous Intelligence and Computing (UIC'09). 39--53. http://dx.doi.org/10.1007/978-3-642-02830-4_5. Google ScholarDigital Library
- Daniele Riboni and Claudio Bettini. 2011. COSAR: Hybrid reasoning for context-aware activity recognition. Personal Ubiquitous Comput. 15, 3 (March 2011), 271--289. DOI: http://dx.doi.org/10.1007/ s00779-010-0331-7. Google ScholarDigital Library
- M. S. Ryoo and J. K. Aggarwal. 2006. Recognition of composite human activities through context-free grammar based representation. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2. 1709--1718. Google ScholarDigital Library
- Saguna, Arkady Zaslavsky, and Dipanjan Chakraborty. 2011. Complex activity recognition using context driven activity theory in home environments. In Smart Spaces and Next Generation Wired/Wireless Networking, Sergey Balandin, Yevgeni Koucheryavy, and Honglin Hu (Eds.). Lecture Notes in Computer Science, Vol. 6869. Springer, Berlin, 38--50. DOI: http://dx.doi.org/10.1007/978-3-642-22875-9_4. Google ScholarDigital Library
- Roger C. Schank and Robert P. Abelson. 1975. Scripts, plans, and knowledge. In Proceedings of the 4th International Joint Conference on Artificial Intelligence, Volume 1(IJCAI'75). Morgan Kaufmann, San Francisco, CA, 151--157. http://dl.acm.org/citation.cfm?id=1624626.1624649. Google ScholarDigital Library
- Thomas Stiefmeier. 2008. Real-Time Spotting of Human Activities in Industrial Environments. Ph.D. Dissertation. ETH, Zürich.Google Scholar
- Thomas Stiefmeier, Daniel Roggen, and Gerhard Troster. 2007. Gestures are strings: Efficient online gesture spotting and classification using string matching. In Proceedings of the 2nd International Conference on Body Area Networks. Google ScholarDigital Library
- T. Stiefmeier, D. Roggen, G. Troster, G. Ogris, and P. Lukowicz. 2008. Wearable activity tracking in car manufacturing. Pervasive Computing, IEEE 7, 2 (2008), 42--50. Google ScholarDigital Library
- M. Stikic, T. Huynh, K. Van Laerhoven, and B. Schiele. 2008. ADL recognition based on the combination of RFID and accelerometer sensing. In Proceedings of the 2nd International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth'08). 258--263.Google Scholar
- Holger Storf, Thomas Kleinberger, Martin Becker, Mario Schmitt, Frank Bomarius, and Stephan Prueckner. 2009. An Event-Driven Approach to Activity Recognition in Ambient Assisted Living Ambient Intelligence. In Lecture Notes in Computer Science. Springer Berlin/Heidelberg, 123--132. http://dx.doi.org/10.1007/978-3-642-05408-2_16. Google ScholarDigital Library
- Gu Tao, Wu Zhanqing, Tao Xianping, Pung Hung Keng, and Lu Jian. 2009. epSICAR: An Emerging Patterns based approach to sequential, interleaved and Concurrent Activity Recognition. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom'09). 1--9. Google ScholarDigital Library
- Emmanuel Munguia Tapia, Stephen S. Intille, and Kent Larson. 2004. Activity recognition in the home using simple and ubiquitous sensors. In Proceedings of the 2nd International Conference on Pervasive. 158--175.Google ScholarCross Ref
- M. Tentori and J. Favela. 2008. Activity-aware computing for healthcare. Pervasive Computing, IEEE 7, 2 (2008), 51--57. Google ScholarDigital Library
- Huawei Tu, Xiangshi Ren, and Shumin Zhai. 2012. A comparative evaluation of finger and pen stroke gestures. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (CHI'12). ACM, New York, NY, 1287--1296. DOI: http://dx.doi.org/10.1145/2207676.2208584. Google ScholarDigital Library
- Kristof Van Laerhoven. 2001. Combining the self-organizing map and K-means clustering for on-line classification of sensor data. In Proceedings of the International Conference on Artificial Neural Networks. 464--469--. http://dx.doi.org/10.1007/3-540-44668-0_65. Google ScholarDigital Library
- K. Van Laerhoven and O. Cakmakci. 2000. What shall we teach our pants?. In Proceedings of the 4th International Symposium on Wearable Computers. 77--83. Google ScholarDigital Library
- Shiaokai Wang, William Pentney, Ana-Maria Popescu, Tanzeem Choudhury, and Matthai Philipose. 2007. Common sense based joint training of human activity recognizers. In Proceedings of the 20th International Joint Conference on Artificial Intelligence. Google ScholarDigital Library
- J. A. Ward, P. Lukowicz, G. Troster, and T. E. Starner. 2006. Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 10 (2006), 1553--1567. Google ScholarDigital Library
- Tsu-Yu Wu, Chia-Chun Lian, and Jane Hsu. 2007. Joint Recognition of multiple concurrent activities using factorial conditional random fields. In Proceedings of the 2007 AAAI Workshop on Plan, Activity, and Intent Recognition. AAAI Press, Menlo Park.Google Scholar
- Guang-Zhong Yang, Benny Lo, and Surapa Thiemjarus. 2006. Body Sensor Networks. Springer, London.Google Scholar
- Juan Ye, L. Coyle, S. Dobson, and P. Nixon. 2007. Using situation lattices to model and reason about context. In Proceedings of the 4th International Workshop on Modeling and Reasoning in Context (MRC'07). 1--12.Google Scholar
- Juan Ye, L. Coyle, S. Dobson, and P. Nixon. 2009. Using situation lattices in sensor analysis. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom'09). 1 --11. DOI: http://dx.doi.org/10.1109/PERCOM.2009.4912762. Google ScholarDigital Library
- Shumin Zhai and Victoria Bellotti. 2005. Introduction to sensing-based interaction. ACM Trans. Comput.-Hum. Interact. 12, 1 (March 2005), 1--2. DOI: http://dx.doi.org/10.1145/1057237.1057238. Google ScholarDigital Library
Index Terms
Complex activity recognition using context-driven activity theory and activity signatures
Recommendations
Complex activity recognition using context driven activity theory in home environments
NEW2AN'11/ruSMART'11: Proceedings of the 11th international conference and 4th international conference on Smart spaces and next generation wired/wireless networkingThis paper proposes a context driven activity theory (CDAT) and reasoning approach for recognition of concurrent and interleaved complex activities of daily living (ADL) which involves no training and minimal annotation during the setup phase. We ...
A knowledge-driven approach to composite activity recognition in smart environments
UCAmI'12: Proceedings of the 6th international conference on Ubiquitous Computing and Ambient IntelligenceKnowledge-driven activity recognition has recently attracted increasing attention but mainly focused on simple activities. This paper extends previous work to introduce a knowledge-driven approach to recognition of composite activities such as ...
Real world activity recognition with multiple goals
UbiComp '08: Proceedings of the 10th international conference on Ubiquitous computingRecognizing and understanding the activities of people from sensor readings is an important task in ubiquitous computing. Activity recognition is also a particularly difficult task because of the inherent uncertainty and complexity of the data collected ...
Comments