Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts

Monday, September 5, 2011

Measuring fluke and Command and Control

This article describes how to quantify the amount of bad luck (or fluke) that affects the unfolding of a game. It also allow to quantify how much the stochasticity of dice roll dominate the game play (Command and Control).

Command and Control
If a player minimizes the effect of randomness by following a sound strategy, the log likelihood of his actions will be maximized.

Let's consider the following table of values:


Target Number Log-Odd Ratio Approx LOR
3-7.75-8
4-5.76-6
5-4.37-4
6-3.28-3
7-2.39-2
8-1.51-1
9-0.70-0.5
100 0
110.760.5
121.511
132.392
143.333
154.244
165.616
177.638


It is convenient to use the approximate values as they are easier to add mentally. 

The interpretation of the CC score is that the larger the CC score is, the more likely that on average the player maintain the control of the narrative. This doesn't mean that the strategy is sound, however. A player making many trivial initiative will get a large CC without necessarily achieving its objectives. The CC score thus makes sense only if a player achieved the objectives set for its actor or faction. CC are most comparable among opposing players for a given game. Discrepancies in CC to complete all faction objectives is diagnostic of a game imbalance. Note that simulations are not expected to be balanced, however. A better than average CC for a successful strategy indicate superior gameplay.

Fluke
Almost everyone who played a game at some point and attributed defeat to bad luck (it seems to happen to me all the time). The Fluke score is an objective way to determine how surprising were the negative outcomes for a player or group of players. The procedure is analogous to the CC score, but sums only the LOR for initiatives that failed.

The expectation is that, on average, initiative will fail when their target number is at 10 or lower. In all of these cases, the fluke score is expected to decrease as the game progresses. The fluke score increases only when an initiative with a target number above 10 fails. In the possible case of a sound strategy undermined by "bad luck", the fluke score will be positive by the end of a game.

Over a long enough game, a sound strategy that maximizes CC will maintain a fluke score around 0. The more risks are taken, the more negative the fluke score is expected to grow. If the fluke score is positive, then a player may claim that the same strategy should work if it was to be repeated a large number of time. Let's all keep in mind that there is no such thing as bad luck: and blaming the dice as unfair is an hypothesis with a very low prior probability.

Comparing CC and Fluke



CC<<0
Poor Control
CC is small
Weak Strategy
CC >> 0
Sound Strategy
F << 0

No sound strategy was found
OR
the scenario is unbalanced.
Set your daring moves better.Too many wild gambles
F is small
The few sane moves you made
should have worked... they didn't.
10,000 monkeys would play like this on average.Expected outcomes for
 a sound strategy on a hard scenario.
F >> 0Almost impossible to get here.The dices were not kind, nor was your strategy sound.You breezed through it,
OR
the scenario is too easy. 

These outcomes were determine with a simulation approach comparing consistent and inconsistent  poor, weak and strong strategies. 10,000 games of 20 initiatives were simulated. Weak strategies had a range of target number centered around 10, poor had the distribution centered slightly below 10 and sound strategies was centered above 10. Consistent play had 5 possible targets values while inconsistent play drew from 7 possible target values. This combination of 6 gameplay style was used to generate the table above.

An unbalanced scenario will cause CC to be negative and large and fluke to be similar to CC. A player consistently relying on long shots will get the same results. The better the strategy, the better is the control of the narrative and this the higher is CC. However, the fluke score should not rise significantly above 0 unless a players proposes exclusively strong initiatives. At this point, the umpire should increase the difficulty for this faction, somehow.

Wednesday, August 17, 2011

Multiple outcomes

I'll try to rationalize the rule on multiple outcomes.

The rule
Multiple outcomes and a success.

The number of possible successful outcomes equals the positive difference between the target number and the sum of the dice roll used to determine the success of the initiative. Outcomes becomes true in the order in which they were proposed by the conch holder.
Example: An initiative is made with four outcomes. The target number is 11 and the sum of 3D6 is 9. The first three (11-9) outcomes are said to be true, leaving the last one false. 


The rationale
Two neutral initiative (target=10) have a combined probability of consecutive success of 0.25. If one combines the two outcomes into 1 initiative, the success of both outcomes will have a base target number of 9. It is thus slightly advantageous to tack in two outcomes to one initiative for as long as the base target of the argument is 10. If it is any different because the base target is set by training/complexity (see table in the rules), the initial advantage dissapears. This is so because the difference in probability in the 9-11 range is flatter than for the rest of the spectrum.

The downside is, however, that the logical link between the outcomes can be attacked with CONs arguments/facts. At -2 per cons, any advantage is lost unless the link between both outcomes is rock solid.

More than 2 outcomes makes less sense, unless the conch holder sees the additional outcomes as nice to have bonuses.

Sunday, August 14, 2011

Statistical Basis for TOEM

The target number of 10 is making the following assumptions:
  1. There is two possible outcomes (Success or Failure).
  2. In absence of evidence, both outcomes are equally likely.
This thus means, that the most reasonable probability for both outcomes should be equal, or with a probability of 50% each. This is very much in line with the Bayesian framework , where the target number reflects the posterior probability and the absence of supporting (or not) evidence indicate flat prior probabilities.

The role of additional information
Any relevant fact either support or counter the logical connection between and event and its outcome. Unless some validated modeling can be done, the effect of facts on probabilities is somewhat arbitrary and the course of most complexity in games. TOEM look at things from a narrative perspective. If fact X is such that it could either explain why an outcome occurs (or not), then it it considered. All arguments or facts are thus weighing the same on shaping the probability of an outcome. However, as more facts are adding up, the effect of additional arguments becomes smaller such that the probabilities never reach 0% or 100%. This effectively imply that all arguments have an influence on the outcome that is distributed roughly according to a geometric distribution. The facts/arguments don't need to be ordered, but it is assumed that some are more influential than others: the way the probability change as facts accumulate abstractly ranks them. In TOEM, the ultimate objective is to generate a reasonable narrative, not to model event the way a numerical calculation would.

Here is an example. In the Harpoon 4.5 ruleset, the probability to damage a ship with 2nd generation counter measures using a 3rd generation, sea skimming missile has a net probability of X%. This probability is compounded by the ship's point defense system and the generation of this system's sensor specs. Finally, in the eventuality of a hit, the chance that a vital system is knocked out depends on the missile DP and the ship point value. Finding the exact probability of this happening can be done with the Harpoon 4.5 appendices and rules, but figuring out whether this happens requires modeling the engagement on a 30 sec impulse basis until the missile has reached the target. With TOEM, the following initiative would be made:
Event: I fire at the opponent using my state-of-the-art SSM missile and disable its ship. Outcome: The ship is now dead in the water, performing firefighting activities and treating injured personel. This is going to happen because my missiles are more sophisticated that the ship's defense (+1), using a sea-skimming approach which makes it harder to detect on time (+1). My missile is designed to disable or sink ship of this size (+1). 
The target number is thus 10+3 = 13, or a probability of  84% of success. I roll 10: a clear success.
It is important to note that although there is a ruleset to find precise probabilities for modern naval combat, this is not the case if the situation calls for determining the reaction of a crowd to a certain event. TOEM provides thus a systematic way to assess the likelihood of "fuzzy" situations.

Using prior information to set the base number
There is no reason why a base of 10 should be selected if there exist information helping in setting it to another base value. Here is a copy of the table:


 TGT number
Prob (%)
Expected consecutive successes
Narrative levels 
Task
Simple
 Task
Technical
Task
Complex
 3
 0.46
 1/217
Desperate



 4
 1.8
 1/55
Desperate



 5
 4.6
 1/22




 6
 9.3
 1/10
very unlikely



 7
 16
 1/5




 8
 26
 1/3
unlikely



 9
 38
 1/2.6




 10
 50
 1
possibly
 Naive


 11
 63
 1.7
likely



 12
 74
 2.8
very likely
Trained


 13
 84
 5




 14
 91
 10
 As a rule, ...
Experienced


 15
 95
 18




 16
 98
 29

Elite


 17
 99.5
Indefinite
 Certainly



 18
 100
Indefinite 






Method A - Probabilities
The second column provides the % probability for all target number. If the probability of an event is known, it is best to select the smallest target number which includes the probability. For example, if  there is a chance in 4 to draw a Spade from a deck of cards, then use the target number 8 which is valid for events of probability over 16% and up to 26%.

Method B - Narrative Levels
Certain words have a specific meaning in a TOEM game. The 4th column lists them with the base target number that they correspond to. This method is simple and works for events that are otherwise impossible to quantify.

Method C - Expected numbers
This is more esoteric, but it can do the trick in some cases. The 3rd column shows the number of expected consecutive success with this target number in average. This information is redundant for target number under 10, but is helpful to set a target number of likely events. For example, if on average one lake out of six contain a shellfish toxin, one would expect to find 5 lakes on average before finding one with the toxin. The initiative: "I eat the mussel from this lake and they are free of toxin." should have a base target value of 13.

Method D - Training and task difficulty
This is a variant of the narrative level which prove to work well during playtesting. The last three columns show the target number for three task complexity against 4 skill level: naive/untrained, trained, experiences and elite. Complex tasks are difficult to achieve, but this simply implies that these task are likely to surrender the initiative while an actor/faction is attempting to complete it.