Deep Down

Action RPG game where your allies are controlled with deep learning.
This is an entry for the AI and Games Jam 2021

(Mouse control is quite interesting, so give it it a try ! :D)

'''

Theme:
AI: For the AI theme we used Reinforcement learning for the main characters. The enemies have different hardcoded AI, some will follow a given path, others have aggro values depending on the distance with a main character.

Break the rules: Here we tried to add some comportement for the main character that are against their normal rule/behaviour  expected from them. The DPS would prefer to go drink a nice glass of wine instead of fighting enemies, the tank is scared of snake and "should avoid them", and the mage love trees and "should" go berserk on them.

'''

The game has a save functionality. So if you are stuck or bug, you can reload it and get back to the lobby.

'''
This game is our first experimentation with Deep Learning in video games. The main idea is to be able to use reinforcement learning as a party, with each character having its own unique abilities. We used ML Agents from Unity to train the characters. ML agents uses Proximal Policy Optimization (PPO) algorithm by OpenAI (https://openai.com/blog/openai-baselines-ppo/)

It was quite harsh to learn all about ML Agents, and apply reinforcement learning in such a short time. 

As the PNJ says, they might lack hidden layers, but you can still enjoy the game !

'''

Pre-Update behaviour:

The AI is pretty broken when playing with DPS. It might be better to select any of the 3 other possibilities to have the best experience !

For some backstory about it, we trained all three character to be able to focus on their role. The DPS follow the lead and try to hit close monsters, the tank try to take the damage, and the mage heal the others. But during training, the tank focused on shielding so much, that it is lost when there are no monsters ! Moreover, during the training, the mage and him grew pretty close, and became good friends. So she focus more on healing him. The end result is that these two refuse to help the poor DPS...

Post-Update behaviour:

Way better ! They now follow quite well the eader, with the mage staying a little bit behind to heal. We focused the training on FOLLOWING THE LEADER !!! We still tried ot give them role specific reward, but I believe that we would need way more training time to be able to get a working collaborative behaviour between the party. So the emphasis was put mostly on moving to the target. Doing so, we were able to keep the party together, and we believe that this is an achievement by itself (please don't shatter our dreams...). They also can behave perfectly well in their respective role, but let say it is more random from one play to another.

"

AI training details:

DPS: The main goal of the DPS was to be able to follow the leader, while going slightly out of the way to slain some monsters. He's also an alcoholic (a whole life of butchery take a toll on him...) and so we wanted him to go catch the wine bottle on the map. 

During the training, he became berserk at some point and stopped following the leader to focus exclusively on attacking monsters. So we had to teach him some manners, and now he is following the leader quite well !

Tank: He was trained to go closer to the monster when the party is approaching one. He also have a defense raise ability that we want him to use when close to monsters. Also, he is scared of snake, and will avoid them at all cost.

The training is not what we can call a success, with him. Maybe all the hit taken during years of adventures cost him his brain... I believe that we focused to much on close combat during its training, and he is kinda lost when no enemies are nearby. But as the first zone don't have monsters, it is difficult to lead him outside. It is an overfit of combat situation.

Mage: The mage is the main healer of the party. In consequence, we wanted her to be able to heal/revive the ally. To do so, we focused the training on abilities and mana save. Also, she loves nature, and wants to hug all trees she can find !

As the mage have more abilities, and have to focus more on her ally situation (position/HP/death), it was really hard to get a correct behaviour for her. We have to extend the training and test more diverse situations.

The healing can be quite random sometimes, but overall, if you take the time, she can save you. However, the mana saving part was too hard to achieve, as to many unknown situation arise when playing compared to trainning, she can easily become a spell machine gun.

Overall, I believe that the main mistake we made during the training was not focusing enough on non-monsters situations. In consequence, the healer spam spells at the beginning, and the tank look frenetiquely for enemies that are not there. The DPS not so bad though !

'''

Design and music are originals !

Adrien Dorise: dev, design, music

Ronan Pons: design

https://soundcloud.com/user-564089708/sets/deep-down-ost

'''

Build description:

V6: The tank is a moron, he skipped to many training classes (surely to go drink with the mage...). In short, the tank does nothing, the mage tries to heal him but it is useless. The DPS follow the leader fine, be behave like a mosquito...

V10: Things start to get interesting ! The DPS and the tank follow just fine ! The training camp is finally paying ! The mage can be confused sometimes and get lost in the maze FOR ETERNITY !

V11: Tried to increase the behaviour of tank and DPS. The DPS can be a killing machine, which is nice ! The tank does not fullfil its obligations as expected. but hey, who can blame him for not wanting to get hit... The mage is more on track, but rarely heal, which is problematic...

V12: Extensive training on the mage. We made her do some push ups, and read some incantations, and now she follow the lead just fine ! Well, she do spam heal, as a total counterpart of before, but still, count as improvement !

Anyway, the game can be played just fine with any character (even better with mouse) starting at build 10 !

If you want to see mosquito behaviour with overcomplicated algorithms and maths behind it, go for build6 !