Cultural heritage (CH) content is often multilingual and multimedia (e.g. text, photographs, images, audio recordings, and videos), usually described with metadata in multiple formats and of different levels of complexity. CH institutions have different approaches to managing information and serve diverse user communities, often with specialized needs. The targeted audience for the proposed lab and its task are those content producers that have to cope with the tension between experts (e.g. museum officials) and non-experts (e.g. the general public).
Evaluation approaches (particularly system-oriented evaluation) in this domain have been fragmentary and often non-standardized. Therefore, the CHiC evaluation lab aims at moving towards a systematic and large-scale evaluation of cultural heritage digital libraries and information access systems. The lab's goal is to increase our understanding on how to integrate examples from the cultural heritage community into a CLEF-style evaluation framework and how results can be fed back into the CH community.
In the 2013 edition of CHiC, the focus of evaluation will focus on truly multilingual retrieval (searching the whole multilingual collection) and how to combine information retrieval with interactive information seeking studies. An interactive observation study will gather data on users, information needs and implicit relevance assessments, which will feedback into an IR evaluation task.