Tasks

We present three tasks, based on the exchange of ideas and proposals among the participants during the last LogCLEF 2010 workshop:

  1. Language identification task: participants are required to recognize the actual language of the query submitted. Annotated resources manually generated by participants of previous editions to create a basic set of ground-truth data (other manually generated resources will be created during the first months of 2011). This ground truth will be used, for example, to evaluate the automatic language recognition algorithms.
  2. Query classification: participants are required to annotate each query with a label which represents a category of interest. A proposal for an initial set of category of interest is:
    • Person (i.e. Leonardo Da Vinci)
    • Geographic Location (i.e. Mont Saint Michelle)
    • Event (i.e. RevoluciĆ³n francesa)
    • Work title (i.e. Divina Commedia)
    • Domain Specific (i.e. Panthera Pardua)
    • Other (i.e. ISBN?)
  3. Success of a query: participants are required to study the trend of the success of a search. The success can be defined in terms of time spent on a page, number of clicked items, actions performed during the browsing of the result list. A common definition of user session will be given to participants. Participants are also encouraged to carry out two subtasks: a) query re-finding, when a user clicks an item following a search, and then later clicks on the same item via another search; b) query refinement, when a user starts with a query and then the following queries in the same session are a generalization/specification/shift of the original one.