Arpan pal mobisys_wpa2015

Arpan pal mobisys_wpa2015

• 1.
  1Copyright © 2014Tata Consultancy Services Limited Fusing Personal Context with Physical and Physiological Context for creating value-added crowd-sensing applications 22nd May 2015 Arpan Pal Principal Scientist, Innovation Labs Tata Consultancy Services Ltd.
• 2.
  2  Pioneer &Leader in Indian IT TCS was established in 1968  One of the top ranked global software service provider  Largest Software service provider in Asia  300,000+ associates  USD 15Billion+ annual revenue  Global presence – 55+ countries, 119 nationalities  First Software R&D Center in India Tata Consultancy Services (TCS) at a Glance Bangalore, India1 Chennai, India2 Cincinnati, USA3 Delhi, India4 Hyderabad, India5 Kolkata, India6 Mumbai, India7 Peterborough, UK8 Pune, India9 2000+ Associates in Research, Development and Asset Creation Singapore10 Innovation @ TCS TCS Connected Universe Platform (TCUP) • M2M Communication • Distributed Computing • Sensor Integration and Management • Analytics Services Context-aware Applications • Healthcare • Insurance • Retail • Manufacturing • Smart Building / Campus • Smart Villages / Cities Overview 10 Corporate Innovation Labs Co-innovation Network (COIN) with Academia and Industry Internet-of-Things Research Three stage Innovation Process – Explore, Enable. Exploit
• 3.
  3 Agenda Context Discovery usingIoT Application Use Cases – Physical Context Evacuation, Insurance, Retail Physiological Sensing – Mobile and Wearable HRV, BP, EEG, GSR Behavioral Model from Physiological Sensing
• 4.
  4 The Internet ofEverything Humans Physical Objects and Infrastructure Computing Infrastructure Physical Context Discovery INTERNET OF EVERYTHING Physical Context Discovery What is happening, where and when People Context Discovery Who is doing what, where and when, who is thinking what Internet of Digital Internet of Things Internet of Humans ABI Research. May 7, 2014 New Business / Pricing Models
• 5.
  5 Understanding the PeopleContext Non-intrusive, un-obtrusive sensing Identity, Location, Activity, Physiology Understand Behavior – Individuals / Groups Quantified Self Customer becomes the focus, not the product or service – key is understanding the Customer, Extend B2B to B2B2C Using Wearable's and Nearables (mobile phone, camera, mic, ….)
• 6.
  6 Context Discovery -Multi-dimensional Fusion • Panic • Stress • Like / Dislike • Weather • Environment • Network • Likes and Dislikes • Location • Activity • Proximity Physical Social Media PhysiologySurroundings Contextual Information
• 7.
  7 Click to editMaster title styleApplication Use Cases
• 8.
  8 Application Use Cases •Floor plan based capacity planning • Location based recommendation • Behavioral Sensing – panic / proximity Emergency Evacuation • Hard Cornering / Braking / Harsh Acceleration from Accelerometer • Driver Scoring • Road / Traffic / Weather Condition • Behavioral Sensing - Stress Driving Behavior • User profiling from usage / social media • Location based Recommendation • Environmental Effect • Behavioral Sensing – Buying urge / group behavior Consumer Behavior Wearable sensing, nearable sensing and crowd sensing
• 9.
  9 Sensing Physical Contextof People – Location and Activity Indoor Localization – Bldg, Mall • Entry-Exit using RFID and Magnetometer • Zoning using Wi-Fi • Fine-grained positioning using Inertial Navigation Activity Detection - Wellness • Walking / Brisk Walking / Jogging / Running using Accelerometer Signature • Orientation and Placement agnostic • Calorie Burnt using Activity based models Magnetometer – Entry/Exit RFID Fusion WiFi -Zoning Bluetooth - Proximity 98% 99.7% 97% 96% (Accuracy ~2m) (Accuracy ~ 98%) Publications o Nasimuddim Ahmed et. al., ""SmartEvacTrak: A People Counting and Coarse-Level Localization Solution for Efficient Evacuation of Large Buildings“, CASPER'15 workshop of IEEE Percom 2013 o Vivek Chandel et.al., "AcTrak - Unobtrusive Activity Detection and Step Counting using Smartphones“, Mobiquitous 2013
• 10.
  10 Sensing Physical Contextof People – Traffic and Driving Traffic Sensing – City Authority • Congestion Modeling from historical location data crowd sensed from vehicles • Honk Detection from crowd sensed audio data • Road Condition Monitoring from crowd sensed Accelerometer data Driving Behavior - Insurance • Hard Cornering / Breaking / Harsh Acceleration from Accelerometer Analytics Publications o Maiti, Santa, et al. "Historical data based real time prediction of vehicle arrival time." ITSC 2014 o Ghose, Avik et. al., "Road condition monitoring and alert application: Using in-vehicle smartphone as internet-connected sensor.“, Percom Workshops 2012. o Tapas Chakravarthy et. al., “MobiDriveScore — A system for mobile sensor based driving analysis: A risk assessment model for improving one's driving”, ICST 2013
• 11.
  11 Physiological Sensing –Mobile Phone and Wearable
• 12.
  12 Behavioral Sensing usingPhysiology References o Levenson, Robert W. "Blood, sweat, and fears." Annals of the New York Academy of Sciences 1000, no. 1 (2003): o Näätänen, R et.al., "A model for the role of motivational factors in drivers' decision-making." Accident Analysis & Prevention 6, no. 3 (1974) o GW Evans, “Environmental stress”, 1984 o Bechara, Antoine et. al., "Emotion, decision making and the orbitofrontal cortex." Cerebral cortex 10, no. 3 (2000): o Mauss, Iris B et, al., "The tie that binds? Coherence among emotion experience, behavior, and physiology." Emotion 5, no. 2 (2005): 175. • Heart Rate Variability • Blood Pressure • EEG • GSR
• 13.
  13 Physiological Sensing –Heart Rate, BP and HRV PPG Signal Field Trials at TCS Office and Indian Villages Tie-up with Hospitals Wearable variant pilot for Crane Operator Monitoring in Factories
• 14.
  14 rMSSD, DSD, SDNN,nn50, PNN50, nn20, pNN20 Physiological Sensing – Results Publications o Arpan Pal et. al., "A Robust Heart Rate Detection using Smart-phone Video", in MobileHealth workshop of Mobihoc 2013 o Aishwarya Visvanathan et. al., "Smart Phone Based Blood Pressure Indicator", in MobileHealth workshop of Mobihoc 2014. o Anirban Duttachoudhury et.al., "Demo – Estimating Blood Pressure and ECG from Photoplethysmograph using Smart Phones", SenSys 2014 – BEST DEMO o Banerjee, Rohan et al. "Noise Cleaning and Gaussian Modeling of Smart Phone Photoplethysmogram to improve Blood Pressure Estimation“, ICASSP 2015 o Nasim Ahmed et al. “Feasibility Analysis for Estimation of Blood Pressure and Heart Rate using A Smart Eye Wear”, WearSys workshop in Mobisys 2015
• 15.
  15 Psycho-Physiological Sensing –EEG and GSR GSR Mental tasks Cognitive Load Visual Attention(VA) Memory(M) Logic(L) Arithmetic(A) Other(O) Emotion(E) Stress(S) EEGartefact removal API(VA),API(M),… API(S) A p p l i c a t i o n Fusion FeatureExtraction (individualtask)
• 16.
  16 Psycho-Physiological Sensing –Results o “Evaluation of Different onscreen keyboard layouts using EEG signals”, SMC 2013 o “EEG-Based Fuzzy Cognitive Load Classification”, FUZZ IEEE 2013 o “Unsupervised Approach for Measurement of Cognitive Load using EEG Signals”, BIBE 2013
• 17.
  17 Physiological Sensing forBehavior Modeling
• 18.
  18 Behavioral Modeling usingPhysiology Pietro Cipresso et. al., “Psychometric modeling of the pervasive use of Facebook through psychophysiological measures: Stress or optimal experience?”, Computers in Human Behavior , 49 (2015) 576–587, Elsevier
• 19.
  19 Behavioral Modeling usingPhysiology – Early Results Tetris-like game designed for Bored and Flow State Stimuli o Submitted: “Dynamic Assessment of Learners' Mental State for an Improved Learning Experience “, Frontiers of Education 2015 Should be extendable to other use cases
• 20.
  20 Looking Ahead -Challenges Need to take care of Battery Power Issue Need to address Privacy Issue Each sensor may be very accurate on its own – fusion is the key Right feature selection for the given use case would be critical Lack of multi-sensor Dataset needs to be addressd Option • Do controlled experiments on diverse set of sample subjects using physiological sensing and create simplified aggregate models • Use the Model in the field (e.g. - % of people who do not follow the evacuation recommendation can help in creating a probabilistic model) • Would need Individual training or constant wearing of sensors for individual models – Driving / Shopping Behavior TCUP – the TCS IoT platform can be used to collect multi- sensor data in an efficient way
• 21.
  21 More References o KarelA. Brookhuis, Dick de Waard, Monitoring drivers’ mental workload in driving simulators using physiological measures, Accident Analysis & Prevention, Volume 42, Issue 3, May 2010, Pages 898-903, ISSN 0001-4575, http://dx.doi.org/10.1016/j.aap.2009.06.001. o J.A. Healey and R.W. Picard, "Detecting Stress during Real-world Driving Task using Physiological Sensors", Intelligent Transportation System, IEEE Trans, , Vol. 6, No. 2, June (2005) 156-166. o Jordan Smith, Neil Mansfield, Diane Gyi, Mark Pagett, Bob Bateman, Driving performance and driver discomfort in an elevated and standard driving position during a driving simulation, Applied Ergonomics, Volume 49, July 2015, Pages 25-33, ISSN 0003-6870, http://dx.doi.org/10.1016/j.apergo.2015.01.003. o Gianluca Borghini, Laura Astolfi, Giovanni Vecchiato, Donatella Mattia, Fabio Babiloni, Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness, Neuroscience & Biobehavioral Reviews, Volume 44, July 2014, Pages 58-75, ISSN 0149-7634, http://dx.doi.org/10.1016/j.neubiorev.2012.10.003. o David P. Wyon, Inger Wyon, Fredrik Norin, Effects of moderate heat stress on driver vigilance in a moving vehicle, Ergonomics, Vol. 39, Iss. 1, 1996. o Markku Kilpeläinen, Heikki Summala, Effects of weather and weather forecasts on driver behaviour, Transportation Research Part F: Traffic o Psychology and Behaviour, Volume 10, Issue 4, July 2007, Pages 288-299, ISSN 1369-8478, http://dx.doi.org/10.1016/j.trf.2006.11.002. o Mauss, Iris B., Robert W. Levenson, Loren McCarter, Frank H. Wilhelm, and James J. Gross. "The tie that binds? Coherence among emotion experience, behavior, and physiology." Emotion 5, no. 2 (2005): 175. o Levenson, Robert W. "Blood, sweat, and fears." Annals of the New York Academy of Sciences 1000, no. 1 (2003): 348-366. o Bechara, Antoine, Hanna Damasio, and Antonio R. Damasio. "Emotion, decision making and the orbitofrontal cortex." Cerebral cortex 10, no. 3 (2000): 295-307.
• 22.
  Thank You IT Services BusinessSolutions Consulting arpan.pal@tcs.com With Avik Ghose, Aniruddha Sinha, Tanushyam Chattopadhyay, Arindam Pal, Debatri Chatterjee