MaCro Philosophy
21-Oct-2019

Final Submission Details
Deadline: 1st November 11:59:59pm AOE

As the end of the competition approaches we would like to provide guidance on the final submission process and evaluation.

Prizes that remain are:

Main Prizes WBA Prize from the Whole Brain Architecture Initiative Category Prizes Thanks again to our wonderful sponsors and collaborators!.

Submission and final tests

You will be able to submit two agents for the final testing. The first will be your best agent from the current track (this will be submitted automatically and there is nothing for you to do), and the second is an agent of your choice that you can submit to the track ‘Submission 2 (optional)’. We will run both your submissions on the final test set and your final result will be the best overall score out of these agents.

If you want to be considered for the WBA prize then you should submit a short write-up explaining how your entry is biologically inspired following the submission guidelines. Full details here.

The deadline for submissions (to either track) and to the WBA prize is 23:59:59 1st November (Anywhere on Earth time). Please note that there may be heavier than usual traffic close to the deadline so we encourage you to submit early if possible.

In order to submit to the second phase please refer to the submission page on evalAI. Note that there is no evaluation on this track, you only upload a docker image that we will automatically retrieve at the time of final evaluation. The status on the EvalAI website will show as “Running”, you can ignore this. You can make up to 20 submissions but only the latest one will be used.

As was previously announced, the final evaluation test set is very similar to the one used throughout the competition, but contains extra tests, all minor variations of those in the current test set. This is to prevent overfitting to the exact tests on the current leaderboard.

To reiterate, the final evaluation process is the following:

We will then take the best of the two in terms of “total” score, and use this as your final submission for both global and categories ranking.

Results and prizes will be announced once we have been able to run the final tests.

Good Luck!