Last week, I joined machine learning enthusiasts from around the world in Prague for Europe's biggest conference on ML and AI applications, ML Prague 2023.
The event featured a broad lineup of speakers, covering a wide range of topics - from the latest breakthroughs in natural language processing to cutting-edge computer vision applications. In this blog post, I want to write about some of the most interesting talks from the conference and highlight the key takeaways. So, without further ado, let's dive in and start with two topics from the area of image processing.
Piotr Skalski from Roboflow showed us how he was able to reproduce parts of the Video Assistant Referee (VAR) system shown in the following clip.
He used YOLOv7 and two cameras that record from different perspectives to track a skeleton of his body.
When using this technique, there are some key points to consider:
Take a look at his GitHub repository to see the results.
Aimira Baitieva from Valeo offered insights into anomaly detection in images using neural networks.
One interesting idea was to train a model that outputs embeddings for images, which represent their information well enough. You could then create a "good" embedding for normal parts (without anomalies) to compare against. If an embedded image has a dissimilarity above a certain threshold when compared with the good embedding, one can mark it as an anomaly.
Their final multi-model approach, however, consisted of three steps:
This turned out to work quite well and is currently in use.
Marek Rosa from GoodAI shared their progress in creating LLM-driven game characters.
In their AI Game, characters use a large language model to generate their thoughts, actions and speech. The player can interact with them via a chat interface and with normal in-game actions like exchanging objects.
Every character has a long-term memory which has been implemented via a vector database. New thoughts and interactions are preprocessed and then saved into the database. When needed, the database will be queried with respect to
The short-term memory is controlled with the current context of the language model.
This combination of short and long-term memory lets agents learn continually by using it to plan actions based on past observations, experience, its current environment and thoughts.
Personally, I think that LLMs will completely redefine the role of NPCs (Non-Player Characters) in video games. Today, the player-controlled character is often the central figure in the game's story while other characters are less important. In the near future, the player will take on a more supporting role and the game does not have a clearly defined protagonist. The story will be open-ended and allows the players and NPCs to make choices that affect the outcome and direction of it. More games will adopt a sandbox-style approach, similar to Minecraft, where there is no apparent goal or ending of the game.
Other interesting projects that try to use LLMs to create autonomous agents are
In conclusion, the ML Prague Conference 2023 was a fantastic opportunity for ML professionals to come together, learn about the latest trends and innovations in the field, and connect with others. I had a great time and am glad that I was able to be there. I hope you enjoyed this short recap of the conference. If you've been there too, let's connect on LinkedIn and have a chat about your favorite talks.