Hello there.
I started a thread about the same topic months ago, but it seems that it has been closed due to its inactivity.
Here's the link to the old thread:

http://www.hcrealms.com/forum/showthread.php?t=585154

Just for you to know, this project is still alive and I got some progresses too, but I had no time to share unfortunately (mostly because I was writing my master thesis in the past months!).

Stay tuned.

i'm all ears!

Quick recap of the project’s goal: develop a program that can play against a human player in a standard heroclix game. The program should be able to make decisions during the gameplay AFTER the game set up. This means that team building and team positioning phases are still left to humans. Maybe in the future one can think of letting the program manage these preliminary phases too, but that's another story. From now on I am supposing two teams, which were previously built and positioned in the map by someone else, opposing to each other. One of those teams is the AI team and it is played by the program.


First of all, I need to clarify the meaning of the word "decision" in the heroclix sense. When I say “decision” I mean the collection of all the moves made by the player during his turn.
Example. In the map there are two characters A and B belonging to the AI team and an enemy character adjacent to both of them. Assuming each character can be given just one action and total number of actions is 2, AI has several choices:
Decision 1: character A attacks the enemy, then character B attacks the enemy.
Decision 2: character B attacks the enemy, then character A attacks the enemy.
Decision 3: character A attacks the enemy, character B rests.
Decision 4: character B attacks the enemy, character A rests.
Decision 5: character A rests, character B rests.
Decision 6: character A moves to square (x,y), character B rests.
Decision 7: character A moves to square (x’,y’), character B rests.
Decision 8: character B moves to square (x’,y’), character A rests.
Decision 9: character A moves to square (x,y), character B moves to square (x’’,y’’).
…
Until all the possible combinations (or permutations, I should say) are finished. For simplicity, forget about free actions at this stage and suppose that every single character can only be given a non-free action per turn.

The main idea is to simulate EVERY SINGLE POSSIBLE decision a player can make, then the one that seems the most promising will be chosen.
Imagine this as if there were MULTIPLE parallel dimensions or multiple parallel players and each one choosing a different decision for the same turn.
Everyone plays and at the end of the turn they get a score depending on how well they played.
This is actually the tricky part because now we need to agree which decision is the best one that is we need to decide an appropriate way of setting scores for all those players.
Supposing we are able to do that, the player with the highest score is charged to play in the real life and the game goes on until the next round comes over and a new turn must be played again and the reality splits in multiple dimensions again and so on...
This means that the simulation must happen at two levels:
1) at the lowest level all possible actions a single character can do must be generated. This number could be very high if we think about all the very specific situations that occur so often during a game. Let's take movement as an example. When a character activates for movement it is necessary to specify which square he will end up to. The number of possible destinations depends on the number of squares that character can actually reach (does he fly? how much is his speed value? …). Some constraints which come from the final goal of that movement will decrease the number of possible destinations. If the goal is to get into position to make an attack the player usually needs to fulfill certain requirements (adjacency for CC or range and line of fire for RC) but a lot of options are still left open, like when there are two or more destinations suitable for the future attack to be made and you would like to choose the one that allows the character to receive support from his teammates while at the same time ensures protection from the enemy.
2) at a higher level, after that a list of all possible actions for each character belonging to the AI team has been generated, all the actions must be combined together in different ways to build all possible decisions the program can make during his current turn. If the team only consists of a single character, this step is useless and each decision corresponds to each possible action that the only character can be given. But in the most general situation a team consists of two or more characters and several combinations are possible. Let’s consider the very simple situation where the AI team consists of three characters A, B and C. The list of possible actions has been generated for each of the three characters and list of A has 10 elements, list of B has 14 elements and list of C has 20 elements. Total number of possible decisions in this turn is equal to 10 x 14 x 20 = 2800. But usually the order in which the characters are activated does matter, so we need to take into account additional combinations (B activates before C but after A, B activates after A and C, B activates before A and C, and so on…), so you have to multiply that number per the number of permutations of 3 characters: 3! = 6.
Finally, we end up with 2800 x 6 = 16800 possible DIFFERENT decisions in total. Please, note that I chose three random numbers (10 actions or 14 or 20 are just a few options for a full powered character, especially if he has a lot of movement) and I made the calculation for a team consisting of only three characters. Furthermore, I deliberately decided to leave free actions out of the equation (for now).
Okay, that's already a lot. I pulled out those numbers because I wanted to show how complex a decision process may look like if we consider ALL THE POSSIBLE OPTIONS a player has during its turn. Of course, a human player doesn't even care about all these possibilities and he just goes for the decision that seems the most promising (based on his experience and on his knowledge of the game) in that particular situation. Now, let's think for a moment about how many missed opportunities are there when we make a decision without considering all the possibilities. Maybe the best decision one could make just hides between those opportunities. Imagine if one could always find the best decision among them and go for it. The main challenge of my project is to find a way to let the computer do all the work. What do you think? Is there any reason why you think this won't ever happen? Have you ever thought about it (or anything similar)? Was the explanation clear enough? Let me know! Here, I wanted to talk only about my ideas for the AI but soon I will share the current status of the project before moving to the next level.

Its very well written. However, when it comes down to the correct possibilities you may run into some serious issues due to the traits and special powers of characters and how they are written. There is a lot of confusion between people on how things are worded as is. That's means you would have to constantly update the clix themselves as to how they act, on top of that update with the new clix every month. Your project is possible, however it will be a huge time consumption, and may not be worth it in the end. However, if you still want to proceed with it. I welcome it, and would love a system to challenge me.

Really very well written! I also broke my mind trying to find a way to play alone or just cooperatively with my son. Due to this absurd amount of possibilities the simplest way I found was by using the system of zombie hordes (which are not at all clever) drastically reducing this amount of possible actions:
Rules link: https://drive.google.com/open?id=1ch...1pcNPUUEGUPv6m
Tokens link: https://drive.google.com/open?id=1Sw...MDh8b0hB00pqsi

At the beginning I'm only considering standard powers to keep things simple. But you're right, I am aware that if at some point I want to extend the whole thing to modern age or even carded figures Era, then I have to face all those issues related to the interpretation of the wording of the special powers etc. I just don't want to care about this now lol

I think your link is broken... It doesn't work for me

Try this:
https://drive.google.com/file/d/1cha...ew?usp=sharing
https://drive.google.com/file/d/1Sw7...ew?usp=sharing

Today I will tell you about the generation of all the possible actions that a single character can take during AI player's turn. I'm referring to the lowest level of the two that I described last time.

First, let me explain some preliminary aspects.
How does a player usually give actions during its turn? Well, once he has chosen the character to be activated and the type of action to give, he needs to set some other options depending on the type of action chosen. For example, if the player wants to move one of his characters on the map, he also has to declare: who is moving; where he's going to; which squares the character is moving through (i.e. the path).
Now the move action is completely defined and it can finally take place.
Second example. The player wants to make an attack: who is attacking? Is it a close or a ranged attack? Who is (are) the target(s)? Only after that all these questions are answered the action can take place.
The AI plays in such a way that it has to make statements in order to give actions every turn. The syntax of these statements is specified by a set of answers to the questions that I listed above.
So there is a kind of paradigm that every statement about an action must respect:

NAME: character who is given the action;
TYPE: type of the action;
PATH: list of squares involved in the action (if required);
TARGET: list of other characters involved in the action along with any useful information about them (if required);
SQUARE: position of a specific square on the map (if required).

These so called visible features of an action are everything you need to state and whoever is sitting at the table with you will immediately know what you want to do (even if you are a non-sentient machine).
Trust me when I say that every action that can be thought of in the game (including power actions granted by STANDARD powers but still not all the ones that come from SPECIAL powers) can be modeled on the basis of this paradigm.
Some examples will clear your mind.

Example 1. Wolverine (which is represented on the map by the black circle with a red 0 inside) is given a move action to run away from Spider-Man (the black circle with a blue 0). He moves 3 squares straight to the north side of the map (if the breakaway is successful, of course).

Translation of the action in terms of its visible features:
NAME: Wolverine
TYPE: movement
PATH: (H8) -> (H7) -> (H6) -> (H5)
TARGET: - -
SQUARE: (H5) (i.e. the destination square, okay it's a bit redundant since the movement is already fully defined by the path but it can be specified as well)

The AI player had a lot of other choices, (Wolverine could have been moved to somewhere else or he could have been given a close action to attack Spider-Man instead) but as soon as we look at those visible features it's pretty straightforward to understand what the AI meant to do.


Example 2. Spider-Man (red 0 here) is given a ranged combat action to attack Sabretooth (blue 0).

Translation of the action in terms of its visible features:
NAME: Spider-Man
TYPE: ranged combat
PATH: - -
TARGET: Sabretooth
SQUARE: - -

Here again, at the beginning there were multiple possibilities (Spider-Man could have been given either a ranged action to attack Wolverine (blue 1) or a move action instead) but as soon as the AI makes that particular statement there is no doubt anymore.



Example 3. Quicksilver (red 0) is given a power action to use Hypersonic speed. He first moves adjacent to Sabretooth (blue 0), he makes a close attack and then he runs away (if the breakaway is successful, of course).

Translation of the action in terms of its visible features:
NAME: Quicksilver
TYPE: hypersonic
PATH: (E8) -> (F9) -> and so on… Just look at that pink pattern in the image.
TARGET: Sabretooth
SQUARE: (J13) (now it's important to specify the square where Quicksilver is located during the attack which is in general different from the destination square!)

This time I won't even try to list all the alternative possible actions because, as you can imagine, there are so many! The number is around:
(Number of reachable targets) x (number of squares adjacent to the target where Quicksilver can legally move) x (number of possible destinations still reachable by Quicksilver with the rest of his movement) .


Example 4. A is given a power action to use Barrier. 4 blocking terrain markers are placed 2 squares away from him in a straight line (path).
Translation of the action in terms of its visible features:
NAME: A
TYPE: barrier
PATH: (x,y) -> (x,y+1) -> (x,y+2) -> (x,y+3)
TARGET: - -
SQUARE: - -

Example 5. A is given a move action. Since A has the Flight ability he can carry any adjacent friendly character fulfilling the right requirements. Let's suppose that B fulfills all the needed requirements and so A will be carrying B during this action.
Translation of the action in terms of its visible features:
NAME: A
TYPE: movement + carry
PATH: whatever it is, you got it
TARGET: [B (x, y)] (the square where B will be placed must also be specified!)
SQUARE: destination

Okay, that's enough for now. This time I've only showed two things of my program: the definition of the actions in terms of their visible features; the generation of the list of all possible actions for a given character. I didn't say anything about the way the AI chooses the best action among all the ones in the list for each character every turn, that is how the AI makes decisions.
I mean, at least I didn't show any of my plans for that part. This will have to do with the so called hidden features of an action and the way we should set scores for every action. Soon I will write more about, but in the meantime I’ll leave you become acquainted with the concepts that I explained in this post. What do you think about? Do you agree with me when I say that every action in the game can be modeled on the basis of a common paradigm? Can you think about any action that can't be described in terms of those 5 visible features (see above)? Actually there is at least one standard power (guess which one!) that can not, though with a slight change in the paradigm I think it would still work.

In the previous post I showed you that before making decisions, the AI has to produce a collection of actions that is as complete as possible in order to choose from it.
In this post I will try to answer the question: how can we say that a given action is better than another one?
So far I've only said that we should assign some kind of score to each action so that we can compare them and then choose the best one.
How do we set scores? Well, these scores should be proportional to the benefits that an action brings to the table for the active player. I would say that the closer an action brings you to victory, the higher its score. This is still kind of obvious, I think. But to find the game variables that are somehow correlated to the score it is not. One of these is surely the damage that you can deal to the opponent within a given action: if we have to choose between two very similar actions, we always choose the one that deals the highest amount of damage. But this is just one of the many, it is not the ultimate answer to the question. How silly would it be to always choose the action that deals the highest amount of damage? Sometimes it would not be useful at all. That's why I think one has to combine several informations of the same action to obtain a realistic score for it.
Here, I identified a set of variables that in my opinion are the most correlated to the score:

1) probability to perform the action (if any dice roll is required)
2) maximum amount of damage that can be dealt among all the enemy targets
3) total number of action tokens that can be given among all the enemy targets
4) unavoidable damage that has to be dealt to the character after the action (e.g. pushing damage, etc.)
5) clicks of life that can be healed among the AI team
6) danger to end up in square (x,y)

These are what I call hidden features of the action. The word “hidden” means that there is no need to let the opposing player know them since they are there just for the decision to be made.
The first 5 are quite straightforward and in every situation it's possible to read them or at least to compute them starting from characters’ values.
Instead the number 6 is still a bit subtle to me, in the sense that I want to have a variable that takes into account the risk of placing a character in a particular square, but I don't know yet how to quantify it. There are a few possibilities, which I will discuss next time because now it may bring us off-topic.

Now, a few examples to better explain their meaning:



Example 1. Sabretooth is adjacent to both opposing Spider-Man and Wolverine. Among all the possible actions he can be given this turn, there is the possibility of attacking either one of the two opponents. Let's look at the hidden features of these two actions:

Close combat action against Spider-Man
1) probability to hit the target (roll at least a 5 with two dices = 83.33%) AND to fail Super Senses (roll less than a 5 with one die = 66%) = 55.58%
2) maximum damage is 2 (4.9998 if using Blades/Claws/Fangs)
3) 0
4) 0
5) 0
6) - -

Close combat action against Wolverine
1) probability to hit the target (roll at least a 5 with two dices = 83.33%)
2) maximum damage is 1 (4.1665 if using Blades/Claws/Fangs)
3) 0
4) 0
5) 0
6) - -



Here it comes the evergreen dilemma: better to hit harder in spite of a lower probability of success or to hit softer with more chances? I guess it mostly depends on one's playing style, though this is something that must also be brought to the AI at some point. And again another common dilemma is whether to use or not BCF in a close combat action. I mean, in this particular situation there is no doubt since most of the time you are going to deal more damage with BCF than how much you would do without. But in general you don't want to use it every time just because your character has got it, especially if he already has an high damage on its own. That's why I designed a quick and easy way to quantify its usefulness. When you fill the hidden feature slot no.2 for an action with BCF, instead of filling it with the actual maximum damage (which would always be 6 minus any damage reductions) you use the result of the following formula:
(damage that would be dealt if the BCF roll was 6) x (probability to obtain an equal or better damage value than the printed one with BCF)/100.
This way if you have damage value = 1 it's 100% sure that you will get a better or equal result with BCF, then the use of BCF is strongly recommended since its maximum damage will be 6 (minus any damage reductions). On the other hand, the higher the printed damage the less likely will be to get an equal or better result and the lower the final result of the formula, being always less than 6 with a minimum of 1.02 if the damage that would be dealt without BCF was 6, which encourages to find other solutions.




Example 2. Captain America is adjacent to Spider-Man but he can also be given a power action to use Charge and attack Sabretooth. There are a few possibilities depending on the destination square of his Charge, the corresponding hidden features look like:
Power action to use Charge against Sabretooth
1) probability to successfully breakaway (roll at least a 4 with one die = 50%) AND to hit the target (roll at least a 4 with two dices = 91.66%) = 45.83%
2) maximum damage is 1
3) 0
4) 0
5) 0
6) ? it should definitely depend on the destination square...



Same considerations here for the first two hidden features. If you don't care about no.6 that would be enough. Each of the five possible ways to charge would be identical, either Captain America attacks Sabretooth from square (I8) or from (I9) or from (H9) or from (I7) or from (H7), that wouldn't matter.
Though in the real life, we know that it makes a great difference to place Captain America in (I9) rather than (I7) because the former is out of Spider-Man's line of fire while the latter can still be the target of a ranged attack during the opponent's turn. Therefore,
Danger to be in (I8) or (I9) or (H9) < danger to be in (I7) or (H7)
even if at the moment I haven't figured out yet how to represent the “danger” feature.

That's all for the moment. As you can see, we are slowly moving to the very heart of the AI program. The decision algorithm is what truly makes it worth and it's the most difficult thing to build. However I wanted to insist on the importance of these two concepts, hidden features and visible features, because these are the INPUT and the OUTPUT of the algorithm itself. So even if I manage to get a super optimal and very sophisticated algorithm structure it would be almost useless if the set of input variables is not enriched with useful information.

Here I thought this was going to be a thread like the memes you see on Facebook that say "I forced a bot to watch 400 hours of SLVRSR4's HeroClix matches, and now it won't play anything but Alpha Flight theme teams and it's dice rolls SUCK" threads.

The only intelligence this game needs is at their corporate office for their play testing group and marketing teams.

This is where things start to fall apart for Heroclix AI unless you do full-on machine learning.

Assuming all clicks were equal you would always go for the highest expected damage. i.e. do which ever action leads to more clicks of damage being done on average. But all clicks are not equal, so there might be cases where you want to try and get the opposing character on to or off of certain clicks. There is also the more obvious case where you only need to do enough damage to KO a character.

I did plan to use Machine learning in the future and I already have some experience with it because I use it at work. My idea is to complete a not-so-bad version of the program in order to be able to test it soon. There are a few things that I'm neglecting right now which should definitely be included in the next version, but I need to keep things simple at the beginning.

This looks intriguing and impressive. Going off what Zandig was talking about...

Would the AI "know" all the dials? For example, determining the actions when controlling 2 characters, as explained in an initial example above, character A can attack and then B, or vice versa. When an enemy gets hit and it's defense goes UP, or gets hits and it's damage reducer improves from Toughness to Impervious, sometimes it's makes sense to attack with the lower AV/Dmg character first.