MaCro Philosophy
1-July-2019
steampunkFOUR

Animal-AI Olympics now released into the wild!

The competition is now live, with v1.0 released, official rules and further information about the syllabus available here and from the competition website and GitHub page. We are performing the last few tests on the evaluation platform and the competition and submission instructions will be pushed to EvalAI by July 8th, at which point it will be possible to submit your entry and see how well it performs.

The Animal-AI Olympics is an AI competition with tests inspired by animal cognition. Participants are given a small environment with just seven different classes of objects that can be placed inside. In each test, the agent needs to retrieve the food in the environment, but to do so there are obstacles to overcome, ramps to climb, boxes to push, and areas that must be avoided. The real challenge is that we don't provide the tests in advance. It's up to you to play with the environment and build interesting setups that can help create an agent that understands how the environment's physics work and the affordances that it has. The final submission should be an agent capable of robust food retrieval behaviour similar to that of many kinds of animals. We know the animals can pass these tests, it's time to see if AI can too.

Prizes: $32,000 (equivalent value)

Overall Prizes

WBA Prize from the Whole Brain Architecture Initiative

Category Prizes

Research credits

The top 20 entries (overall score) on September 1st will be awarded $500 of AWS credits for use in the second half of the competition. Participants will have to opt-in for these prizes to be considered eligible and conform with the AWS terms of service.

Final Categories

The 300 tests have been finalised and we can now reveal the full ten categories. Note that all tests are pass/fail based on achieving a score above a certain threshold. In most cases, this corresponds to retrieving the only piece of food in the environment. The tests in each category range from relatively easy - solvable by most to all animals - to very hard, solvable by only a few animals and expected to be left unsolved at the end of the competition. As always, further information is available from the competition website and GitHub page.

  1. Food: Most animals are motivated by food and this is exploited in animal cognition tests. The same is true here. Food items provide the only positive reward in the environment and the goal of each test is to get as much food as possible before the time runs out (usually this means just getting 1 piece of food). This introductory category tests the agent's ability to reliably retrieve food and does not contain any obstacles.
  2. Preferences: This category tests an agent's ability to choose the most rewarding course of action. Almost all animals will display preferences for more food or easier to obtain food, although the exact details differ between species. Some animals possess the ability to make complex decisions about the most rewarding long-term course of action.
  3. Obstacles: This category contains immovable barriers that might impede the agent's navigation. To succeed in this category, the agent may have to explore its environment. Exploration is a key component of animal behaviour.
  4. Avoidance: This category introduces the hot zones and death zones, areas which give a negative reward if they are touched by the agent. A critical capacity possessed by biological organisms is the ability to avoid negative stimuli. The red zones are our versions of these, creating no-go areas that reset the tests if the agent moves over them. This category of tests identifies an agent’s ability to detect and avoid such negative stimuli.
  5. Spatial Reasoning: This category tests an agent's ability to understand the spatial affordances of its environment. It tests for more complex navigational abilities and also knowledge of some of the simple physics by which the environment operates.
  6. Generalization: This category includes variations of the environment that may look superficially different to the agent even though the properties and solutions to problems remain the same. These are still all specified by the standard configuration files.
  7. Internal Models: This category tests the agent's ability to store internal models of the environment. In these tests, the lights may turn off after a while and the agent must remember the layout of the environment to navigate it in the dark. Many animals are capable of this behaviour, but have access to more sensory input than our agents. Hence, the tests here are fairly simple in nature, designed for agents that must rely on visual input alone.
  8. Object Permanence: Many animals seem to understand that when an object goes out of sight it still exists. This is a property of our world, and of our environment, but is not necessarily respected by many AI systems. There are many simple interactions that aren't possible without understanding object permanence and it will be interesting to see how this can be encoded into AI systems.
  9. Advanced Preferences: This category tests the agent's ability to make more complex decisions to ensure it gets the highest possible reward. Expect tests with choices that lead to different achievable rewards.
  10. Causal Reasoning: Finally we test causal reasoning, which includes the ability to plan ahead so that the consequences of actions are considered before they are undertaken. All the tests in this category have been passed by some non-human animals, and these include some of the more striking examples of intelligence from across the animal kingdom.

We have also simplified the environment and used some conventions to make the tests as achievable as possible. Unnecessary objects have been removed from the environment. In the tests, walls that are used as walls are mostly grey, walls used as platforms (that can't be climbed) are blue, and ramps are pink. More details in the docs on the GitHub. You can also find example training files under examples/configs for the first seven categories. These are just guidelines to get started and do not contain enough detail to solve the full category. For the last three you're on your own!

Sponsors and Partners

This competition would not have been possible without our sponsors.

Sponsor/Partner Thanks
Thanks to Amazon for sponsoring the research credit prizes and also the infrastructure for running all the tests required over the course of the competition.
Thanks to Unity Technologies for sponsoring prizes and providing the environment, both the Unity Platform and ML-Agents which we have used to build the competition environment.
Thanks to the Whole Brain Architecture Initiative for sponsoring a prize for the most biologically inspired entry.
Artificial Intelligence Journal Thanks to AIJ for funding to be put towards travel prizes to bring top competition entrants together to present their work.
Thanks to EvalAI for hosting the competition.
Thanks to GoodAI for supporting the competition from its initial planning stages. Without them it would not be possible to grow into the project that it has become.
Thanks to the Leverhulme Trust and the Leverhulme Centre for the Future of Intelligence for funding the research that went into designing and building this competition.

Further prize and entry information

Full details about the rules and regulations of the competition and the prizes we offer can be found on the main competition website.

*Travel allowance for NeurIPs can be used to cover conference registration, economy travel and accommodation up to a total of $1,000

*To be applicable for the WBA prize applicants must write a 2-page description of their approach (submission details TBC). They will be judged primarily on biological plausibility and this will be weighed against performance. An independent panel of experts will be convened for this purpose and their decision will be final.

*Category cash prizes are awarded in descending order of categories. The prize goes to the first place entry for each individual category that has not already won a prize of equal or greater value. Any entrant that has already received a larger or equal prize will also receive a certificate.

Back to top