Tasks
We present three tasks, based on the exchange of ideas and proposals among the participants during the last LogCLEF 2010 workshop:
- Language identification task: participants are required to recognize the actual language of the query submitted. Annotated resources manually generated by participants of previous editions to create a basic set of ground-truth data (other manually generated resources will be created during the first months of 2011). This ground truth will be used, for example, to evaluate the automatic language recognition algorithms.
-
Query classification: participants are required to annotate each query with a label which represents a category of interest. A proposal for an initial set of category of interest is:
- Person (i.e. Leonardo Da Vinci)
- Geographic Location (i.e. Mont Saint Michelle)
- Event (i.e. Revolución francesa)
- Work title (i.e. Divina Commedia)
- Domain Specific (i.e. Panthera Pardua)
- Other (i.e. ISBN?)
- Success of a query: participants are required to study the trend of the success of a search. The success can be defined in terms of time spent on a page, number of clicked items, actions performed during the browsing of the result list. A common definition of user session will be given to participants. Participants are also encouraged to carry out two subtasks: a) query re-finding, when a user clicks an item following a search, and then later clicks on the same item via another search; b) query refinement, when a user starts with a query and then the following queries in the same session are a generalization/specification/shift of the original one.