Task motivation: This sub-task focuses on "interactivity" in the use of Europeana. The user experience in dealing with cultural heritage in the real world is about the experience of immersion in that space. It may not be about retrieval and viewing of a specific object. Much like the real world, in virtual culture and heritage situations, the interface in place influences profoundly that experience.
The purpose of this task which will be to gauge user experience by observing user activity with Europeana under controlled and simulated conditions, aiming for as much "real-life" experiences intruding into the experimentation. The output will be a rich data set that includes both user profiles, knowledge and experience, selected individual differences (such as a motivation to explore), a log of user interactivity, and a structured set of questions about the experience. The data will be analyzed by participants to provide a holistic view of user experiences in Europeana, which will feed into IR evaluation tasks in the years ahead.
Task definition: The experimental task that will be assigned to users will follow Borlund's situations. In this case, a common scenario identified from existing research will be used to prime participants. This will not be just about finding specific objects, but about an "experience." The scenario (which will be the same across all observations) will describe a non-intentional interaction (no predetermined information need), in the line of:
"Imagine that you are sitting in a doctor's office, a bookstore, the airport, a pub or coffee shop. You have 20 minutes to spend looking at European art and culture using the Europeana digital library, museum and archive. Insert in your "bag" (an interface object that will work like a shopping cart without the ecommerce checkout) any objects that you found unexpected, surprising or novel and make notes on the stuff you found interesting along the way.
Users will browse and search an experimental interface using the English CHiC Europeana collection. The intent of the open-ended scenario and interface will be to allow a participant to customize somewhat their experience, e.g., selecting a geographic destination, a particular type of art, etc.
The experimental system will use pre- and post-interaction questionnaires and logging of user interactions in order to capture user behavior.
Data Collection & Experimental System Set-up: To conduct the task we will use the web-based experimental system currently being developed under the PROMISE NoE at the University of Sheffield. The experimental system will provide a similar browsing and searching experience as Europeana, only using the English collection.
The following data will be collected:
- user profile (using questionnaires), e.g. age, gender, level of education, native Language, plus all languages fluent, country of residence
- user behaviour/cognition (use questionnaires)
- user actions (from logs), using the following metrics: Queries, mouse clicks, result page, documents viewed, documents saved, etc.
- user experience with browsing (use questionnaires)
- motivation (use questionnaires)
- prior knowledge/experience with C&H (use questionnaires)
- use of Europeana (use questionnaires)
User constraints: Participants in the study must be adults (18 or older) and must be familiar with European culture and heritage.
Participants will receive web-based access to the experimental system, which comes with a standardized protocol for pre-questionnaire, task-based interaction and post-questionnaire. This task is designed for easy entrance by many groups because the experimental system will provide a complete environment for capturing the user experience. One user will approximately need 20 minutes for the entire experiment.
Requirements for participation: Participating research groups agree to:
- use the standard protocol that specifies instructions to the participant, the task the participant will do, and the data that will be collected,
- use the standard interface to Europeana provided by CHiC group,
- process a minimum of "30" users, whereas 10 users should be observed in a lab-based environment and 20 users can remotely access the system,
- conduct all observations in English (since the collection is in English, other languages cannot be served).
- how the communal data will be used (shared data use of all observations),
- specify a research question for data analysis of the shared data points before data is shared (to avoid redundant analyses).
Interactive task timeline:
- December 2012: registration
- January 2013: release of research protocol (questionnaires and task) for review
- 1 February 2013: release of experimental system prototype for testing
- 1-30 March 2013: data gathering period
- 1 May 2013: release of shared data pool to all participants