Humanist Evaluation Methods in Locative Media Design
DOI:
https://doi.org/10.5617/jmi.v2i1.893Keywords:
Design, Methodology, Media InnovationsAbstract
Media design can be used for research purposes if it includes a clearly defined research question, and clear evaluation to see whether an answer to the research question has been found. Using a project with locative media for classical music communication as our example, we discuss common evaluation methods from the User Experience field, observing that they all tend to test “interface” and not “content.” Instead we propose three other methods of evaluation, that have a basis in humanist theories, such as textual analysis and genre studies: (1) Qualitative interviews with evaluators after the evaluation, asking them to describe the service in their own words, followed by a semantic analysis to get at how they have understood the service. (2) Within-subject A/B tests with alternative versions that are different in key aspects. (3) Peer review by experienced design researchers, who are likely to have a more fine-tuned vocabulary to express their opinions.
References
Andersson, B.-E., & Nilsson, S.-G. (1964). Studies in the reliability and validity of the critical incident technique. Journal of Applied Psychology, 48(6), 398. doi:10.1037/h0042025
Bolter, J. D. (2003). Theory and Practice in New Media Studies. In G. Liestøl, A. Morrison, & T. Rasmussen (Eds.), Digital Media Revisited (pp. 15-33). Cambridge: MIT Press.
Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4). doi:10.1037/h0061470
Gadamer, H.-G. (2004). Truth and method (W. Glen-Doepel, J. Weisheimer, & D. G. Marshall, Trans. Second, revised ed.). London: Continuum.
Hartson, R., & Pyla, P. S. (2012). The UX book: Process and guidelines for ensuring a quality user experience. Amsterdam: Morgan Kaufmann.
Hassenzahl. (2000). Hedonic and Ergonomic aspects determine a software’s appeal. CHI Letters, 2(1).
Hassenzahl, M. (2001). The effect of perceived hedonic quality on product appealingness. International Journal of Human-Computer Interaction, 13(4), 481-499.
Hevner, A. R., March, S. R., Park, J., & Ram, S. (2004). Design science in information systems work. MIS Quarterly, 26(1), 75-105.
Kohavi, R., Longbotham, R., Sommerfield, D., & Henne, R. M. (2009). Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery, 18(1), 140-181. doi:10.1007/s10618-008-0114-1
Krippendorff, K. (2006). The semantic turn: A new foundation for design. Boca Raton: Taylor & Francis.
Krug, S. (2010). Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems (Kindle ed.). Berkeley, California: New Riders.
Latour, B., & Woolgar, S. (1986). Laboratory work: The construction of scientific facts (Second ed.). Princeton University Press.
Lewis, C. (1982). Using the ‘thinking-aloud’ method in cognitive interface design. Research report RC 9265. IBM TJ Watson Research Center.
Liestøl, G. (1999). Rhetorics of Hypermedia Design. In Essays in Rhetorics of Hypermedia Design (Ph.D. Dissertation ed., p. 265). Oslo: Department of Media and Communication, University of Oslo.
Liestøl, G. (2006). Conducting Genre Convergence For Learning. Cont. Engineering Education and Lifelong Learning, 16(3/4), 255-270.
Løvlie, A. (2009). Textopia: Designing a locative literary reader. Journal of Location Based Services, 3(4), 249-276.
March, S. T., & Smith, G. F. (1995). Design and natural science research on information technology. Decision Support Systems, 15, 251-266.
Moulthrop, S. (2005). After the last generation: Rethinking scholarship in the age of serious play. Proceedings from Digital arts and culture, Copenhagen.
Nielsen, J. (2000). Designing Web Usability: The Practice of Simplicity. Indianapolis: New Rider.
Nielsen, J., & Loranger, H. (2006). Prioritizing Web Usability. Berkeley, California: New Rider.
Nyre, L. (2014). Medium design method: Combining media studeis with design science to make new media. The Journal of Media Innovation, 1(1), 86-109.
Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. University of Illinois Press.
Downloads
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution BY 4.0 License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
