Semantic and Multimodal Annotation

Report on the CLARA Summer School on Semantic and Multimodal Annotation, organized by the University of Copenhagen on August 15-26, 2011 in Copenhagen.

Goal and audience

The summer school provided training on semantic corpus annotation (sense ambiguity, semantic roles) and on multimodal (verbal and non-verbal) annotation of video-recorded interactions. The aim of the course has been to train young researchers within different areas of semantic and multimodal annotation as well as evaluation methods of these and application of machine learning techniques. The course was given as a mixture of theoretical lectures and hands-on exercises. Participants were trained in both semantic and multimodal annotation schemes; they also got the opportunity to present their own project and to focus on their preferred area of annotation. Tools: Stamp, Cornerstone, Jubilee, GATE, ANVIL, and others.


  • Susan Brown, University of Colorado at Boulder (DARPA GALE OntoNotes project, VerbNet, SemLink)
  • Bolette Sandford Pedersen, University of Copenhagen (Wordnets and other lexical resources for semantic annotation)
  • Anders Søgaard, University of Copenhagen (machine learning algorithms)
  • Jens Edlund, Department of Speech, Music & Hearing at Kungliga Tekniska Högskolan, Sweden (evaluation via resynthesis and human-computation-style labelling)
  • Przemek Lenkiewicz, Max Planck Institute for Psycholinguistics, Nijmegen (automatic multimodal annotation)
  • Costanza Navarretta, University of Copenhagen (gesture annotation, evaluation and machine learning for multimodal annotations)
  • Patrizia Paggio, University of Copenhagen (MUMIN annotation model, NOMCO corpus, gesture annotation)

Venue: The summer school was held at University of Copenhagen, Faculty of Humanities, room 16.1.08.

Final program

Slides week 1:

Slides week 2 are available to course participants on a password-protected website.


The contact persons for this course were Bolette Sandford Pedersen (, Costanza Navarretta (costanza @, Magdalena Lis (magdalena @, and
Hector Martinez Alonso (alonso @

A free excursion to Roskilde was held on Saturday 20. Course participants visited the Viking Ships Museum and Roskilde Dome and ate dinner at Restaurant Snekken.

The University of Copenhagen provided internet connectivity through Eduroam.
Participants were are asked to bring their own laptops. The list of software to install beforehand and installation manuals was sent out to participants by email.

ECTS Credits

Participation in the course gives and estimated 10 ECTS credits for candidates at PhD level who have presented their work. Note, however, that the final approval of ECTS will be done at your own institutions. PhD students from University of Copenhagen were asked to make a short report of the course, if they want to obtain all 10 credits. All participants will be given a Course Attendance Certificate.

See also the participant list, attendance list and photos from the course.

Leave a Reply

You must be logged in to post a comment.