ESWC-16 Open Knowledge Extraction (OKE) Challenge

The OKE challenge, launched as first edition at last year Extended Semantic Web Conference, ESWC2015, has the ambition to provide a reference framework for research on Knowledge Extraction from text for the Semantic Web by re-defining a number of tasks (typically from information and knowledge extraction), taking into account specific SW requirements. The OKE challenge defines three tasks, each one having a separate dataset:

  • Entity Recognition, Linking and Typing for Knowledge Base population
  • Class Induction and entity typing for Vocabulary and Knowledge Base enrichment
  • Web-scale Knowledge Extraction by Exploiting Structured Annotation. 

Task 1 consists of identifying Entities in a sentence and create an OWL individual representing it, link to a reference KB (DBpedia) when possible and assigning a type to such individual. 
Task 2 consists in producing rdf:type statements, given definition texts. The participants will be given a dataset of sentences, each defining an entity (known a priori). 
Task 3 will be based on one of the largest, publicly available collections of triples extracted from HTML pages (provided by the Web Data Commons project). Participants will use annotated Web pages as stimuli for training a Web-scale extraction system which is capable of extracting structured data from non-annotated pages.

Important Dates

  • Challenge papers submission deadline: Friday March 11th, 2016
  • Challenge paper reviews due: Tuesday April 5th, 2016
  • Notifications sent to participants: Friday April 8th, 2016
  • Test data published: Friday April 8th, 2016
  • Camera ready papers due: Sunday April 24th, 2016​

 

Participants must:

  1. Submit a paper describing their system, via [EasyChair], no later then March 11th, 2016. The paper should contain the details of the system, including why the system is innovative, how it uses Semantic Web, which features or functions the system provides, what design choices were made and what lessons were learned. The description should also summarise how participants have addressed the evaluation task(s). Papers must be submitted in PDF format, following the style of the Springer's Lecture Notes in Computer Science (LNCS) series (http://www.springer.com/computer/lncs/lncs+authors), and not exceeding 12 pages in length. 
  2. For task 1 and 2 provide access to the application as webservice (specifications can be found here ), with input/output provided in NIF format. The final evaluation  will be carried out via GERBIL. The implementation of the evaluation for tasks 1-2 is already available on GERBIL (also accessible as open source code) as Web demo. Participants can autonomously test their system using GERBIL (selecting as Experiment Type either OKE Challenge 2015 - Task 1 or OKE Challenge 2015 - Task 2).
    For task 3 participants will have to produce results in the specified format.
  3. The URI for the final system (for task1-2) / results on test set (for task 3)  must be provided by April 8th, 2016 when the organizers will evaluate the systems against the evaluation datasets, which will be publicly released after announcement of results.

Organizing Committee:

  • Andrea Giovanni Nuzzolese, STLab-CNR, Italy
  • Valentina Presutti, STLab-CNR, Italy
  • Anna Lisa Gentile, University of Mannheim, Germany
  • Robert Meusel, University of Mannheim, Germany
  • Heiko Paulheim, University of Mannheim, Germany
  • Aldo Gangemi, Université Paris 13, France

Website: https://github.com/anuzzolese/oke-challenge-2016