Towards unravelling the relationship between on-body, environmental and emotion data using sensor information fusion approach

Kanjo, Eiman and Younis, Eman M.G. and Sherkat, Nasser (2017) Towards unravelling the relationship between on-body, environmental and emotion data using sensor information fusion approach. Information Fusion, 40. pp. 18-31. ISSN 1566-2535

[img]
Preview
Text
1-s2.0-S1566253517303433-main.pdf - Published Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (2MB)

Abstract

Over the past few years, there has been a noticeable advancement in environmental models and information fusion systems taking advantage of the recent developments in sensor and mobile technologies. However, little attention has been paid so far to quantifying the relationship between environment changes and their impact on our bodies in real-life settings.

In this paper, we identify a data driven approach based on direct and continuous sensor data to assess the impact of the surrounding environment and physiological changes and emotion.

We aim at investigating the potential of fusing on-body physiological signals, environmental sensory data and on-line self-report emotion measures in order to achieve the following objectives: (1) model the short term impact of the ambient environment on human body, (2) predict emotions based on-body sensors and environmental data.

To achieve this, we have conducted a real-world study ‘in the wild’ with on-body and mobile sensors. Data was collected from participants walking around Nottingham city centre, in order to develop analytical and predictive models.

Multiple regression, after allowing for possible confounders, showed a noticeable correlation between noise exposure and heart rate. Similarly, UV and environmental noise have been shown to have a noticeable effect on changes in ElectroDermal Activity (EDA). Air pressure demonstrated the greatest contribution towards the detected changes in body temperature and motion. Also, significant correlation was found between air pressure and heart rate.

Finally, decision fusion of the classification results from different modalities is performed. To the best of our knowledge this work presents the first attempt at fusing and modelling data from environmental and physiological sources collected from sensors in a real-world setting.

Item Type: Article
Identification Number: https://doi.org/10.1016/j.inffus.2017.05.005
Dates:
DateEvent
28 May 2017Accepted
29 May 2017Published Online
Uncontrolled Keywords: Multi sensor data fusion, Regression analysis sensor data, Multivariable regression, Affective computing, Physiological signals, Machine learning
Subjects: CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
Divisions: Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology
Depositing User: Gemma Tonks
Date Deposited: 07 Mar 2023 16:08
Last Modified: 07 Mar 2023 16:08
URI: https://www.open-access.bcu.ac.uk/id/eprint/14227

Actions (login required)

View Item View Item

Research

In this section...