Hi

My research is the area of gesture and emotion in HCI. Will emotion be important?…it would seem so, we feel that it would be the case but I think that much of it is about our cognitive acceptance of the illusions of intelligence that computing delivers. At the same time the vista for novel interaction is enormous; essentially it is currently almost a vacuum, and clean slate. 

Hi Roman,

I understand your point, people would like to speak to a human-like assistant, that is why companies put so much effort in that area (e.g. siri and now cortana) also in the scope of new interactions i proposed IoT as an enabler for those interactions, that comes to those fuzzy borders of IoT with CPS and HCI. However i think that emotions will play a major role in the HCI and Human-to-Human communication. My point is that, no matter what technological interfacing or services you provide, they will all clash, at same point, by sending the less appropriate message at the wrong timing and that will not be acceptable by a human user. Emotional assessment will provide clues about what people want/need and that i think will be crucial for successful interfacing.

One of my current projects is in the field on behaviour change technology, a field within HCI. The use of emotion to persuade people is more powerful than logic. Targeting peoples emotions brings up many ethical issues in this field. There are new sensors on the market that are able to detect how people are feeling (http://vandrico.com/device/interaxon-muse). Using these sensors, it would not be too difficult to develop apps that respond to how people are feeling. The potential for these devices are enormous. 

Hi Reece, I also feel challenged by that area and those devices. The implications in neuromarketing are disturbing, as you mention, ethical issues are a major concern. There are so many studies in this area that we easily get lost in so many theories, all proposing ways for the companies to take advantage over competitors. However when it comes to human computer interaction there are simpler but challenging questions to address. For instance; if a person feels depressed or sad (supposing that we can determine that mental state) how would the application (or the environment) react? In an IoT context it would be possible to use surrounding devices (color and intensity of lights, ambient sound, etc) to please a person, but what should be presented? happy music? sad music? That is a challenging question as it depends on each person. Don't you think that, even with so many technological 'opportunities' we should bring to discussion more specialists on those social sciences? Psychologists, sociologists or other 'behaviourists'??

The reference below is a design model that I have kept coming back to over and over again, in the development of persuasive technologies. Lets say the sensor could detect a depressed state and we didnt know how to program the system to react. BJ Foogs model suggests that we take an educated guess and then test it out on real participants. Not a formal scientific test but like a short 2 hour test, to see how participants react. The model aims to find answers by learning along the way. I believe solutions can be discovered relatively quickly using this process.

Yes, I would prefer to have the system checked by a psychologist before any formal scientific research began. I reckon at least 80% of the system can be be developed without anyone outside of HCI.

Fogg, B. J. (2009, April). Creating persuasive technologies: an eight-step design process. In Persuasive (p. 44).

I must say that i am not that enthusiastic with persuasive technologies as they walk in the thin edges of ethics and something else. In FP6 we worked in a network of excellence (http://www.biopattern.org/) that addressed cancer and brain disorders, at that time we proposed an extended EHR that could capture lifestyle data the so called bioprofile (http://oa.uninova.pt/1320/). If at that time it seems plausible, now it is much more feasible as a cell phone can seamlessly collect data data about your location, countries and places you visited and other lifestyle habits. The questions of ethics and privacy where already a concern at that time (~10 years ago) and now raised to a new step with the type of data that is collected even without user notice. However the benefits could easily be highlighted; imagine with those ebola outbreaks, if all that population carried smartphones one could easily know the risks of spreading to other places and identify the persons at higher risk. So the question would be; at what point are privacy risks surpassed by the benefits? and who should ensure security of such data to ensure personal privacy and safety while keeping the obvious benefits?

I like both your ideas and think they are doable, despite the stifling requirements imposed on developers by the healthcare industry. Maybe it will be the developing nations, who have less restrictive healthcare systems, who eventually lead the way in innovative healthcare delivery.

I recently had a close look at the privacy laws. From a technical perspective they are broad. The laws use the word 'reasonable' a lot to describe what the end user considers 'reasonable security' and 'reasonable privacy'. Remember, the laws are intended for the healthcare industry, who know a lot about healthcare but not much about computers :) I didn't think the privacy laws were much of a burden. Its about keeping things reasonably private, writing a candid privacy policy and having participants consent before using the system. 

From an ethical perspective, what I have found difficult for the project I'm working on, is the cultural requirements. While I have been in the IT industry for about 20 years, the current project is my first experience working in healthcare. Ive found Western healthcare systems are simply not acceptable to some cultures. The problem is in the healthcare delivery, not with the minority cultures. Respecting the cultural requirements in the methodology and final deliverable has been the main ethical challenge.

Emotions and IoT will be interesting. A question that arises is how much weightage can be assigned to emotions and emotion-based decision making in IoT? Detecting emotions accurately is still a far-fetched goal.

Can an angry emotion after a sudden power-cut  lead to something fruitful? Can an expression of relief after arriving home from work make your dinner better?

Hi Qazi, interesting question those you mention. I think that first we need to determine what data is already available (in wrist bands or other wearables) and what can we extract from them. Then we could determine what is missing and what would be desirable to have in terms of physiological data. An important point is user evaluation. If you get some markers and make correspondence with user's assessment then you have a learning system that, with time, will better determine about your emotional state. Another issue is to determine what to do with that assessment. As stated early in this discussion, once you determine an emotional state how to react? in this matter i think we need more contributions in social and medical sciences (e.g. psychologists, sociologists, etc) also in this matter learning could be a used again. As Reece stated above (from Foogs) what about taking an educated guess and in the future learn if the care put in such dinner resulted better? :)

Every time I hear about using emotions for HCI i wonder, if it is what people actually desire. Is the point of such systems is aid emotion regulation (decrease anxiety or stress)? Or is the purpose for game to react coherently with emotions? 

What I mean is, what if people do not want to follow the emotional state, but they are not sure (usually) what mood do they actually want to be in? How does such emotional HCIs respond to such situations?

That is a good point Krzysztof, one cannot 'impose' a desired mood. Some studies suggest (i don't have any citing at hand but i will find it) that when you are depressed you don't want to listen some happy music, most probably you will prefer sad music. I don't know if that will help on your mood... In the case of gaming we need to look to other aspects; if a game don't activate you then it must be in a boring stage, as a programer you need to foreseen that case and prepare some action, whatever that means in the context of the game. So addressing your question i would say that It Depends ! if you want a player to engage more in the game you need to activate hime. If you want someone to let go a depressive state (in ehealth apps) you need first to get in the mood and then try to cheer up that person. But if you are developing an app for a business man, if you detect high stress rates, better not to ring that high tone ring alarm, right? I don't think there is a yes/no, black/white kind of answer to your question, it depends on context assessment. Do you understand my point and do you agree?

I see the point. My intuition was also that the answer would depend on the context. I was rather asking about what is thought within the field - how is it dealt with at the moment. Do you by any chance have some review or author you could recommend? 

Krzysztof, about what to do varies a lot from gaming to decision support systems to any app design. However if you are looking for the context-awareness, that is prior to any decision in this cases, there are a lot of interesting papers. I would recommend this one as a starting point http://www.sciencedirect.com/science/article/pii/S1574119209000510

as for the applications in the health domain, i co-authored this paper:

https://www.researchgate.net/publication/269631093_Internet_of_Things_for_eHealth_in_a_Physiologic_and_Sensorial_Perspective_Supported_by_the_Cloud

That has a follow up already accepted for publishing.

Conference Paper Internet of Things for eHealth in a Physiologic and Sensoria...

Thank you very much :) I will look into that.

Thanks for your contribution Christophe!

I would ask you if what you mean is somehow big data analysis over tweets? About space and time sure it has space for a lot of correlations; what people do in relation to the space they are, what are the lapses of time between determined events, how places change the way time between events vary according to places, etc it surely has a lot os space for research. Nevertheless there are two constraints; privacy issues (unless people freely tweet much space/time details frequently) and, as I mentioned before; people (also) lie in social networks!