The Geometry Friends competition

After refining our agents, it was time to participate in the annual Geometry Friends competition.

Although our Final Undergraduate Project was graded 10/10 and was described as one of the best of the year, my classmate and I knew our job was not over. The public levels of the 2023 Geometry Friends competition had been revealed, and we wanted to make sure our agents passed them fluently. Our rectangle agent was shining, having no trouble at tackling its levels, but our circle and cooperative agents were having some trouble.

After several improvements that we made to both, we managed to submit our agents with a perfect public performance. This can be seen here (an archive of the state of the competition page the last day before closing). We led on the three competitions, being the circle track the only one in which we did not have a significant margin over the next participant. We were confident on our chances in the rectangle and collaborative tracks, but the circle tracks was anyone's game. We had to make sure everything was as optimal as possible, and there were no loose ends. Our slight advantage came from the lower elapsed times we had over the rest of the participants, but it was not nearly enough to secure first place.

Finally, after a few days, all agents were evaluated on the private levels, which had remain unknown. While this was happening, we examined which could be the most troublesome for our agents.

From the rectangle track, every level seemed easy. Very rarely (around 1 every 20 runs), our agent got stuck in level 9. Unfortunately, this happened during the competition on one ocassion, but it did not have any impact on the final result.

From the cooperative levels, most were similar to previous years' levels, and, at first glance, it looked like we were not going to have any problem. However, after some tests, level 6 resulted problematic. After picking the first two diamonds, they always got stuck in this position:

This happens because of a discretization error, in which our agents think they are able to slide under the black block when they actually cannot. This is something we knew could happen, but it was a design decision in which the pros significantly outweighted the cons. Every other level was beaten almost every time without issues.

Finally, from the circle levels, the only one that scared us was level 6. This level had appeared before in other competition, and looks like this:

The strategy is not obvious at first, and all human players with whom we tested the game struggled solving it. The most problematic feature of this level is that you cannot recover if you fall from the upper platforms, which is easy due to how physics work when hitting the ceiling. The key is going after the left before going to the right, which our agent knew. However, the landing on the small platform and a small miss of the necessary speed makes it hard to complete the level even if you know what you are supposed to do. Our agent manages to beat it around half the times, getting the other half 2 out of the 3 available diamonds.

This level turned out to be the deciding factor. The Thunder agent, our main competitor, managed to complete it 6 out of the 10 times, one more than our UCMAgent. With perfect results in the other levels, we obtained a close and well deserved 2nd place. It is worth noting the bronze medal went to other agent which completed the level 4 times, so the podium scores were extremely close one to another.

And, with this, the competition was over. Our final results can be seen either on the official webpage or here (if you are seeing this many years from the future). We absolutely dominated the rectangle and collaborative tracks, and almost won the circle track as well. You can access and use our submissions and reports from the respective scoreboards, and also through this Github repository.

As for our next steps, we are working on a level corpus, that may help future neural solutions. We believe having a big amount of solvable levels will make this easier to obtain proficient results that overcome human performance at the game. Once this is finished, we intend to publish an article that summarizes and explains our achievements. I will write another post if this ever happens.