Notes for seminar 2

As I was reading chapter 11, I wondered which of the four evaluation paradigms (quick and dirty, usability testing, field studies and predictive evaluation) would suit our project best.

Considering our resources, ethos and cumulative experience, I feel that usability testing is a no-go. I don't believe we could maintain the control that the author emphasizes. Setting up and maintaining "laboratory-like conditions" is probably beyond our reach and these "casual visitors" wouldn't respect/trust us, mere first year students at KTH, enough to let us strictly control them.

Our scenarios and personas would come in handy if we would like to use predictive evaluation. The question is if we trust our own analysis and knowledge of the users. Users are generally not involved in the process and if we base our product after false assumptions, the end user might be sorely dissapointed.

Quick and dirty and field studies appear to have alot in common, the major difference seems to be that quick and dirty lives up to its name, it's a much more fast and sloppy evaluation. I feel that we still need to learn more about our potential users. A lot of information can be gained by not only talking to the them, but also by observing how they use products and behave (something we haven't done). I think we should use one (or both?) of these two evaluations, since they wouldn't drain our nonexistant resources while also allowing us to get more interaction and thereby experience with the users. These evaluation paradigms simply seem to fit us and our project more than the other two.

So the questions I want to discuss are:

1. Which evaluation paradigm do you think suits our project the best?
2. Are there any differences between the last two evaluation paradigms that I've missed?

Seminar Notes 2 - Artem

Chapter 11: As many of the chapters we have read, chapter 11 stresses the importance of rapid feedback and iterative design. To our help, there are two main branches: conceptual design (which deals with the actual idea of how something is going to be used) and physical design (which focuses on details as screen layout and menu structure.) When the actual prototype is being designed, it can be grouped into low fidelity and high fidelity. For instance, the first one might be a paper-based prototype while the latter is software-based. The main advantage of low fidelity prototype is that it is quick and easy to produce (and modify, which is good at an early stage). During the construction, it is important to keep in mind metaphors and try to put unfamiliar things close to familiar ones (this is quite similar to when people listen to radio. Usually, you will hear that familiar and new songs are intertwined). The cognitive load at different stages should be considered. The values in different cultures should be kept in mind.

Chapter 12: This chapter introduces the DECIDE framework that has the aim to facilitate the planning of an evaluation. An interesting point brought up in the chapter is whether it is possible to study people without actually changing them (see the example with the tribe on p.490).

Chapter 14: Here we are introduced to different settings that where experiments can be performed. To sum up, a lab is contrasted with the “wild” (natural setting). It mentions the way participants should be selected, and how the data is to be analysed.

Question:

1. The ethical considerations are of great importance. During the museum visit, we observed the way people interacted with the different exhibits, which we later used as a part of our analysis. To what extend are we able to observe participants without telling them that we do so?
2. Once we have a prototype, we can set up an experiment to test it. Would it be better to test our prototype directly in the museum without notifying the users or invite them into a controlled lab setting. Moreover, would it be ethically correct to observe the users in the natural setting without telling them that they are a part of the experiment. 

                                          

Seminar 2 Notes

Chapter 11 discusses prototyping and its uses within the design process. I think it can be useful to invest time in prototyping because it can give a different insight as to how a product's interface functions. The book describes two different types of prototyping, low and high fidelity prototypes.  I think that is important to start off with using low fidelity prototypes out of cardboard and paper then move on to high fidelity prototypes using software. This is because designing the interface is an iterative process and low fidelity prototypes are easier to alter and improve upon. If we were to use a high fidelity prototype first, we would spend more time on coding the interface and debugging it rather than focusing on perfecting the interface's features to suit the user's  requirements.

Chapter 13 defines four evaluation paradigms, quick and dirty, usability testing, field studies and predictive evaluation. This chapter also provides the DECIDE framework. I think that this a useful guideline for people who are unsure where to begin when evaluating user needs. I noticed how the DECIDE framework has appeared during our museum visit. We have asked the museum visitors about practical problems as well as deciding how to deal with the ethics of anonymity. 

Questions to discuss:

Which evaluation paradigm is most suited to our project and is it feasible (in terms of the time it takes to perform the evaluations)?

seminar 2 notes

Chapter 11 discusses prototyping and mentions the two classes of prototypes, low-Fidelity and high-Fidelity prototyping and mentions several advantages and disadvantages with them. Obviously, low-Fidelity prototyping is much easier and faster and can demonstrate conceptual ideas without big problems. High-fidelity prototyping becomes only relevant when you’re in a stage where you want to test that the details are well-formed and integrated in the design.

The chapter also mentions scenarios and storyboards, which really are just detailed scenarios where the user’s interaction with the device is narrated. Although it’s good to imagine how the user may interact with the device in a given context, i feel like this technique suffers from the fact that the designers know everything about their design and may not easily comprehend what difficulties a real user may run into.

Question: What is actually a mood board??

In chapter 13 the authors present us with the evaluation framework DECIDE. Most of it are general instructions like “Determine the goals”, “Explore the questions” etc.

In chapter 14 the authors mention different types of evaluation studies, Usability testing and field studies. The difference between them is that the first one is conducted in a controlled lab environment while the second one is done in the wild.

Which one to choose depends on what the product is. For evaluating a new computer or mobile phone it makes sense to do it in a lab where you could for example extract information through eye tracking. But if you want to evaluate a product that is highly integrated in some outer environment, field studies should be conducted. Although it could be complemented with some usability tests too.