Please name your solution files “LANG-task#-solution”, for example “finnish-task1-solution”, etc. 3 = Bonus: tasks are solved using the Standard approach, drawing also possibly on higher task number related training data and/or extra unlabeled data given on the website.2 = Restricted: All tasks are solved using only task-number specific training data: task 1 uses only task 1 train/dev data task 2 uses only task 2 train/dev data task 3 uses only task 3 train/dev data.Anything else is considered “using a bonus resource”. The solution to task 3 may only use task 1, 2, and task 3 training/development data. Likewise, the solution to task 2 may only use task 1 and task 2 training/development data. 1 = Standard: The solution to task 1 may only use task 1 training/development data.The Y should specify either 1, 2, or 3, depending on which data you are using to solve the task. If there are any additional details you would like us to know about your system or resources you used, please write a short description in the body of the email. In the case of multiple institutions, please place a hyphen between each name. If you do submit multiple ordered guesses, please output multiple lines with differing last columns the order in the file will be the order in which we rank them.Įmail the resulting text files to with the subject in the format: INSTITUTION–XX–Y, where you should replace institution with the name of your institution and XX with an integral index (in case of multiple systems from the same institution). Note that you may submit multiple predictions for a given row and we will measure mean reciprocal rank.

Essentially, you will be adding the missing last column of answers to the test files.

The output format should be a text file identical to the train and dev files for the given task. Please run your system for each language and each task for which you wish to submit an entry into the competition. We have released the test data! It is in the same format as the training and dev data with the exception that the last column has been omitted. Please submit the shared-task description papers at by May 15th.

ResultsĪvailable in the shared task overview paper.

Taskpaper columns