TLDR: Created an AI to play Team fight tactics. It is starting to learn but could some help. Hope to bring it to the research world one day.

Hey! I am releasing a new trainable AI to learn how to play TFT at https://github.com/silverlight6/TFTMuZeroAgent. This is the first pure reinforcement learning algorithm (no human rules, game knowledge, or legal action set given) to learn how to play TFT to my knowledge and may be the first of any kind of AI.

Feel free to clone the repository and run it yourself. It requires python3, numpy, tensorflow, collections, jit and cuda. There are a number of built in python libraries like time and math that are required but I think the 3 libraries above should be all that is needed to install. There is a requirement script for this purpose.

This AI is built upon a battle simulation of TFT set 4 built by Avadaa. I extended the simulator to include all player actions including turns, shops, pools, minions and so on.

This AI does not take any human input and learns purely off playing against itself. It is implemented in tensorflow using Google’s newish algorithm, MuZero.

There is no GUI because the AI doesn’t need one. All output is logged to a text file log.txt. It takes as input information related to the player and board encoded in a ~10000 unit vector. The current game state is a 1390 unit vector and the other 8.7k is the observation from the 8 frames to give an idea of how the game is moving forward. The 1390 vector’s encoding was inspired by OpenAI’s Dota AI. The 8 frames part was inspired by MuZero’s Atari implementation that also used 8 frames. A multi-time input was used in games such as chess and tictactoe as well.

This is the output for the comps of one of the teams. I train it using 2 players but this method supports any number of players. You can change the number of players in the config file. This picture shows how the comps are displayed. This was at the end of one of the episodes.

Random early game TFT comp

This project is in open development but has gotten to an MVP (minimum viable product) which is ability to train, save checkpoints, and evaluate against prior models. The environment is not bug free. This implementation does not currently support exporting or multiple GPU training at this time but all of those are extensions I hope to add in the future.

For all of those code purists, this is meant as a base idea or MVP, not a perfected product. There are plenty of places where the code could be simplified or lines are commented out for one reason or another. Spare me a bit of patience.

RESULTS

After one day of training on one GPU, 50 episodes, the AI is already learning to react to it’s health bar by taking more actions when it is low on health compared to when it is higher on health. It is learning that buying multiple copies of the same champion is good and playing higher tier champions is also beneficial. In episode 50, the AI bought 3 kindreds (3 cost unit) and moved it to the board. If one was using a random pick algorithm, that is a near impossibility.

I implemented an A2C algorithm a few months ago. That is not a planning based algorithm but a more traditional TD trained RL algorithm. After episode 2000 from that algorithm, it was not tripling units like kindred.

Unfortunately, I lack very powerful hardware due to my set up being 7 years old but I look forward what this algorithm can accomplish if I split the work across all 4 GPUs I have or on a stronger set up than mine.

This project is currently a training ground for people who want to learn more about RL and get some hands on experience. Everything in this project is build from scratch on top of tensorflow. If you are interested in taking part, join the discord below.

https://discord.gg/cPKwGU7dbU --> Link to the community discord used for the development of this project.