I read this paper written my a few researchers in University of Michigan. The name Newman drew me in, since I have seen the name multiple times in this field. This paper was like many others: a study on how people interact with a smart home system they came up with.
I was quickly skimming through the paper since I was quite well-aware of what most HCI papers talk about, but slowed down a bit when I found an interesting approach to testing how well their system OSCAR works. OSCAR is a form of end-user programming that allows user to flexibly control their devices of networked devices at home. The paper did not mention this explicitly, but from a photo, I could tell that OSCAR in a physical form is a tablet that people can interact with.
Unlike other researchers, who gave a preview of the functionalities of their system beforehand and observed user interaction or did a survey or interview to collect information on user satisfaction, these researchers gave specific tasks for users to perform without any prior help. They recruited nine subjects, most of whom were not professionals in technology. They asked the subjects to do the following tasks:
Their conclusion was that subjects provided positive response to OSCAR and that they were able to complete most tasks without help. When subjects asked for mild help it was either because there were problems with UI and subjects did not know that OSCAR was waiting for user input, or because they were not accustomed to think like a programmer–“designing rules that will react to future states to produce desired outcomes.”
It is notable how in this field, research groups come up with their own system for networking devices and then test it on subjects. It is a bit different from reading biology or chemistry research papers because all the papers seem to conclude with a positive note on their “product” and acknowledging some drawbacks. Especially when I read a paragraph on the specific positive responses subjects mentioned of OSCAR such as “I would find this system very useful in my own home” and “I would like to have this system if it were available,” I felt as if I was reading an advertisement for a product.
All the HCI papers end with similar ideas that there should be a centralized system that controls all connected devices and that it should be easy for people to use.