WG3: Assesment of Empirical and Cognitive Soundness

Primary tabs

This WG approaches DRD's from the perspectives of methodologically-sound cross-linguistic analysis, and of psycholinguistic experimentation. Within the framework of the Action “best practices” will be confronted, including results on inter-annotator agreement, compatibility of the taxonomies with psycholinguistic findings, and performance on automatic sense annotation.

WG3 Members
Co-leaders: 

Prof Ted Sanders, Dr Hannah Rohde

Milestones: 
  • Sharing experimental methodologies for assessing the cognitive processing of DRDs both within and across languages, and for testing the cognitive validity of postulated semantic and pragmatic features used in the annotation scheme
  • Measuring the interrater agreement of the different annotation schemes analysed in WG2, within and between languages
  • Performing automatic sense annotation Challenges
Deliverables: 
  • Experimental design and methods will constitute specific panels of the Training Schools.
  • Administration of open Challenges within one of the established Challenge frameworks such as those run annually by ConNL and SIGSem, where all interested parties can submit their solutions to specific DRD-related tasks, which can then be systematically compared and evaluated.
  • Raising additional funding to submit the proposed taxonomies to empirical testing
Meetings: 
WG3 Meetings