Think aloud - Kim

Think aloud - Kim

The subject was a 28 year old female from Skåne. The think aloud was performed in front of a computer in a calm environment. The participant tried the prototype by clicking around by herself. I explained the parts that the participant didn't understand and guided her when she got stuck, since some parts of the application were a little bit unclear without any background information. It's important to know that this only shows a prototype of the application and not the full version of it. Therefore some parts aren't fully developed yet, but the core functionality is implemented.

The participant clicks her way to the AR view.
P: "Vadå 850m och 500m? Vad står det för?

The participant clicks on Gröna Lund.
P: "Jaha! Då får jag information om Gröna Lund".

The participant clicks on Skansen.
P: "Det finns ingen information om Skansen. Nu fattar jag vad 850m och 500m står för, det är avståndet till platserna".

I told the participant to click on the other views.
P: "Okej, nu får man fram en karta över området. Vad står båten för?

I told the participant that the boat shows the current location.
P: "Okej, då förstår jag".

The participant clicks on the settings icon.
P: "Bra att det går att kryssa i vilken typ av information som ska visas".

The participants general impression.
P: "Det är bra att man kan hitta det man är ute efter som turist. Snygg layout! Den påminner en hel del om appen Stay, förutom att ni har lagt till AR".

Think aloud - Joakim

Participant: 28 year old woman from Stockholm.

Destination Djurgården, jaha, svenska. Ok om jag trycker där kan jag byta språk.

Nähä, det kunde jag inte, då kom jag direkt till camera view (fundersam).

Ok, 500 m till Gröna lund..mm..

Vad är ditt intryck?

Ja, är det här som en riktig kamera view eller?

Ja det ska vara en riktig kameravy i den riktiga appen

Jaha ok! Jaha, skansen är så nära alltså ok! Hmmm vad ska jag göra nu?

Ja, du kan trycka på någon av dem för att få mer information

Ok, då får jag mer information om Gröna Lund, det skulle jag ju kunna behöva om jag var turist. Nu vill jag trycka på Skansen...nähä det gick inte.

Nej, just nu fungerar endast Gröna Lund. Det är bara en konceptuell prototyp.

Ok, ja vad ska jag göra nu då. Jag antar att jag skulle flyttat telefonen för att se mer?

Ja exakt, vad tycker du om konceptet?

Jag gillar att det är i realtid, det påminner lite om pokemon Go! Det är tilltalande.

Du kan testa de andra vyerna genom knapparna där uppe

Aha, Map view, då kan jag se vart jag är och vart jag är på väg. Det är bra. Och jag kan byta språk igen. Det är bra

Location filter! Det är nog bra, så man kan ställa in "jag vill inte se några restauranger bara historical sites!"

Några fler kommentarer?

Jag gillar det här med att det är real-tid. Att jag kan filma och direkt få information. Det är tilltalande.

Comments

The think aloud was a natural setting type of evaluation employing the method of an in the wild study. Little control over the users activities was underway. The participant was rather given small hints to overcome the fact that she wasn't within the proposed target group to start with.

The problems the user encounters seems more related to the fact that the prototype isn't representing the whole system. The conceptual idea seems to be much appreciated. This person was from Stockholm and not a tourist so it wasn't as accurate as it could have been but still gave some valuable insight. Especially that we have created an interesting application.

Exercise 5: Design critique

We presented our target group, mentioned the user needs and that we assume the users have access to internet. Then we presented the clickable prototype https://www.flinto.com/p/7098746d and also discussed some changes that we were thinking about.

Ideas proposed to the peers

• Keyword: Simplicity
• Skip menu and only have two main views
• Introduce a first-time-open view with language selection

Comments

The reception was good. No design suggestions except the ones proposed by us. There where a few comments:

• Term AR might not be well-known by tourists
  • Might be solved by just having two main views. Then we don't have to spell out the AR view.
• How do tourists know how to download app?
  • Posters on the ferry and by the terminals notifying the users. A good thing to do while waiting for the ferry to arrive.

Exercise 4 - Evaluation

Since no users aren't easily accessible an inspection method for evaluation was considered. An heuristic evaluation was therefore conducted with one expert, another HCI-student. First a presentation of our low-fi prototype was conducted followed by a pluralistic walkthrough involving our design group and the expert. The following topics were then considered by the expert:

• Is it a clearly defined User group?
• “The feeling"
• Interaction and over arching structure
• Primary and secondary functions
• Design/composition
• Help
• Sketches

In summary the overall impression of the app is good. The simplicity was appreciated and provided a good user experience. The evaluation as presented by the expert can be seen here: http://myferrytail.blogspot.se/2016/11/evaluation-for-group-a5-agneev-guin.html and at the end of this post. There where some comments about the settings menu, which might not be necessary. We'll do another review about this when we have constructed a more detailed prototype.

In the following evaluation we would like the evaluators to focus more on Nielsen's usability heuristics. They will also be more meaningful in the context of a high-fi prototype.

Evaluation for Group A5: Agneev Guin

The following is an evaluation of the app prototype of group A5.

User group

The user group consists of Tourists from all backgrounds. The group focuses on everyone carrying mobile devices who are eager to learn about the nearby places of interest.

"The feeling"

The pluralistic walkthrough of the app is quite appealing as it's user centered approach doesn't allow any unnecessary stuff, thus focusing on its usability. The augmented reality section will be quite useful while riding on the ferry, however the use of the map view is constrained while in the ferry as the first time tourists might not be able to relate to the map. The information along with the interest points might help the tourists choose their next destination point. Interaction and over arching structure The one touch selection to display the details of the tourist spots seems simple and easy to operate. There are no unnecessary options to mess around, thus focusing on the easy to grasp capability.

Primary and secondary functions

The main function, to use the Augmented Reality, is the main focus of the design prototype. The AR is the catchy part where information for the important points is easy to capture from your current location. The secondary function, to see the map view, is more valuable to those part of the target group who are on the shore.

Design/composition

An interesting splash screen of AR and Map is a clear segregation. The heuristic approach of having a layout to provide the user an impression of a first time user for a fresh app helps keep the design simple. The design is thus, quite appealing. People might want to explore more based on the simple nature.

Suggestions for improvement

The tag AR or Augmented Reality may not be clear to all. The app seems quite attractive once someone is using it but it might be a bit of a confusion for the first time users. The impression that its a tourist guide can be explained more. The settings for AR and GPS on/off might not be required. It can be like Google maps that if GPS is disabled, it will popup a message to enable. The pop up window should remain open even after moving away the mobile device. Else the user might have to hold it that place and direction to get all the info. In case more info needs to be provided like information about the place or public transport timings to reach there may be provided with easy to grasp icons. Ratings may be provided based on previous tourist travelers. Help No further support is provided or needed for any tricky stuff.

Sketches

The sketches clearly justify their plans. The pop up information shows that they cover a part of the screen. This needs to remain even after removing the device is moved. They have even provided an idea of the settings section to describe the possibilities.

Summary

The usability of using a mobile device to explore the nearby places has been widely explored. The simplicity provides a good user experience focusing primarily on their target group.

Smartphone app prototype

Smartphone app prototype

The prototype of the application was made in Flinto. It is a realization of the sketches that were made by the group during exercise 3. The prototype is in its initial stages and will be evaluated and improved trough an itterative process. The following images show all the screens in the screens in the application prototype. There is also a link were the prototype can be used (https://www.flinto.com/p/7098746d).

The application starts by showing a splash screen with the logo. Then you reach the main menu where you have the option to experience the ferry either through augmented reality or a map view. Pressing the cog in the upper right corner of the main menu, enables you to apply different settings.

Pressing the geotag icon when in the AR/map view, gives you an information pop-up regarding the location in question. As an example the prototype shows Gröna Lund and pressing the geotag gives you information about the location.

Seminar 2 summary

Seminar 2 summary

Evaluation is a method to use in order to make sure that a whole design/product or parts of it is working properly from a user perspective. What part that is being evaluated can differ, depending on what the aim of the design/product is. The different evaluation methods were discussed, settings involving users (both controlled and in the wild) and settings not involving users.

During the seminar the following questions were discussed:
- What setting and method is appropriate for evaluating our application?
- Are the results valid? reliable? What about biases etc.?

In the next excersice we will perform a heuristic evaluation. This evaluation doesn't involve any potential user which could be a seen as a drawback. Furthermore, all experts are from the course. The group is therefore contemplating performing a cognitive walkthrough with a person that isn't taking the course, to avoid bias. We also would like to perform a evaluation in a natural setting to involve actual users. This will be done in the later stages, when we have a hi-tech prototype.

Doing a usability test in a controlled setting such as a laboratory, will not be feasible as the scope of the project does not allow such a evaluation.

Due to the fact that Augemented Reality (AR) is a relatively new technology, performing evaluations without the users could be less reliable and less valid compared to evaluating a more established technology since there is less knowledge about such systems. Furthermore users might not have any previous experiences with AR to base their expressions and opinions on, which can cause problems in regards to the validity of evaluation involving users.

Since only a prototype will be developed in this project, a final summative evaluation will not be conducted. However, starting from next weeks exercises formative evaluations will be performed.

Usability testing through experiments was discussed during the seminar and the group concluded that such a method of evaluation will not be performed as it usually is done in a later stage of the development process. Analytics, where user traffic through a system will not be performed due to the same reasons. We won't have a fully functional system and enough users to test this.

Seminar 2 notes - Sajaval Choudrey

Seminar 2

The book introduced evaluation together with its key concepts. The reason why we need to evaluate, is because users expect a lot from the systems that are available to them, in form of experience, design, usability etc. The system needs to go beyond the basic requirements of the customer, it also needs to provide the user with a pleasing experience and intuitive design. If one can identify all of the important aspects for the customer and also evaluate the product, it can become appealing to a wider audience. Depending on the system and product being created one needs to evaluate different things. Everything stems from the demands of the customer, one system might need to focus on speed during the evaluation, while another might need to focus on design and ease of use. The means of evaluating in terms of location is also dependent upon the system that is being evaluated. In the book it is mentioned that the evaluation of a toy can benefit from being made in a child’s room with the child using the toy. Other evaluations can be made directly in a lab. The nature of the evaluation is taken into consideration when deciding “when” to carry out the evaluation. Does it occur in an earlier stage of the development process? Or Is it left to the end when the product is finished?

How do we interpret data when evaluating? There are several things one should think of when doing so, is the system reliable (not prone to error or inconsistency)? Are the evaluation results valid (do they measure what we wanted)? Are there any biases (are the results of the evaluations affected by any personal traits or agendas)?

Evaluation can as stated in the book be summarized in the following categories:

1.       Controlled settings involving users (examples are laboratories and living labs): users’ activities are controlled in order to test. Usability testing is often performed in such a setting. Usability testing focuses on evaluating how usable a product is, what do users do? And he social of influences on the users behaviour.

2.       Natural settings involving users (examples are online communities and products that are used in public places): there is little or no control of users’ activities in order to determine how the product would be used in the real world. Field studies are often made in natural settings because of the fact that products are increasingly being designed for environments other than the office or the lab.

3.       Any settings not involving users: consultants and researchers critique, predict, and model aspects of the interface in order to identify. Experimental designs can be performed in such environments as well as controlled settings involving users

When users are hard to come by in the evaluating process inspections can be made. This involves getting experts in the field of interaction design, requirements and user behaviour, to evaluate a product. One such inspection method is heuristic evaluation which inspects the usability. Walkthroughs are another form of inspection methods where the product is “walked through” while at the same time taking notes of any issues occurring.

Question: which setting is the most appropriate for evaluating our app?

Seminar 2 notes - Kim

Seminar 2

Chapter 13
This chapter presented the main ideas of evaluation and different methods for evaluating designs or other parts of a design project. Evaluation is an important thing to do because it makes sure that for example a design is accepted among a wider group of users. In the text the author mentions that it’s important to know why an evaluation should be performed, what one should evaluate, where and when the evaluation should be done. It’s often possible to evaluate different aspects in different ways. An evaluation could for example range from a whole system to a smaller part of an application.

In the text the author writes about three different broad categories used to classify evaluations. These categories are controlled settings involving users, natural settings involving users and any settings not involving users. Each category has its advantages and disadvantages. It’s also important to know that using combinations of the different categories is possible to reach a better evaluation that fits to a specific task. Finally, the author mentions things that could influence the way the data is translated. These things are reliability, validity, ecological validity, biases and scope.

Question: What kind of evaluations would be suitable for our design project?

Chapter 14
In this chapter evaluation studies were presented. This involved where the evaluation took place, which in the text focused on usability tests in controlled laboratory settings and field studies in natural settings. In the text different ways of measuring quantitative performances during a test were presented. For example it’s possible to measure the time it takes to complete a task, number and type of errors per task or numbers of users making a specific type of error.  When an evaluation is performed in a controlled setting such as a lab, there are different type of equipment that can be used to gather data to the evaluation.

Hypotheses testing is a way of examining the relationships between variables. A variable can be independent or dependent. An independent variable is possible to manipulate. When the hypothesis is tested it’s common to state a null hypothesis and an alternative hypothesis.

When it comes to experimental designs, it’s important how participants should be involved and which participants that should be used. The author writes about the three different designs different-participant design, same-participant design and matched-pairs design. 

Field studies is a way of gathering data from natural settings. The data is mainly collected using observations and performing interviews. It could also be done by for example filming or capturing audio. The time of the field studies being done can vary from a minute to a several years. In the wild studies is a way of capturing how people are using new technologies in a natural setting.

Chapter 15
This chapter presented methods used to understand users through heuristics, remotely collected data or models that predicts a user’s performance. All the mentioned methods don’t involve any participation of a user. Instead role playing is used to simulate the user and analyzing different relevant aspects and identifying usability problems is done.

Using heuristic evaluations is done checking how different parts of the tested product stands against a list of heuristics. In each iteration any problems are identified and handled.

Walkthroughs is a method used for predicting problems for users without performing any user tests. This method involves going through a list of tasks and marking possible problems with the user features. Some walkthrough methods don’t include the users while other walkthrough methods involves the users as well as usability specialists and developers. In the text the author writes more about cognitive and pluralistic walkthroughs.

Analytics is a method used to evaluate the user traffic through a system. This means collecting data about parts of the system to be able to see if there are parts that are underused.