Scoring Tenets
In this section we use RFC2119 keywords to specify scoring formula requirements.
Tenets
Total score MUST increase with more flags captured
The scoring formula must reward the effort and skill required to exploit more services, and thus to capture more flags.
Total score MUST decrease with more flags lost
The scoring formula must reward the effort and skill required to defend against attacks.
Flag value MUST diminish with more successful attacks
The scoring formula must reward the effort and skill required to exploit a vulnerability in proportion to its difficulty, inferred from the number of successful exploits.
Perfect SLA MUST be worth more than any attacker's relative gain
The scoring formula must reward participation in the CTF and therefore disincentivize intentionally shutting off services to prevent other teams from overtaking you.
The cost of downtime MUST NOT outweigh the benefits of patching
To accurately measure player effort and skill in exploiting and defending, the scoring formula must not put such a disproportionate emphasis on defense or SLA, e.g. by scaling attack points with either, that it disincentivizes patching.
SLA / DEF / ATK points SHOULD be normalized against teams, services and total flagstores
The scale of points awarded by the scoring formula should be predictable by organizers and playing teams ahead of time, and thus be normalized against the number of teams, services and total flagstores.
SLA SHOULD decrease fairly with every missing flag in the recovery period
We define the recovery period as the number of rounds N (>= 0) that a service can become unavailable for and still receive a fraction of its SLA points. The scoring formula should award partial SLA points for each round a service was available for within the past (N+1) rounds and make this information available to the players, since it more fairly rewards service uptime.
Flag value SHOULD be calculated independent of its flagstore
Vulnerabilities in services with many flagstores should not be worth less to exploit. The dynamic flag value calculation already ensures that more difficult exploits are rewarded.
Evaluation
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale linearly with the amount of flags lost.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
In the worst-case, when every team exploits a service, more defense points are lost than gained from SLA.
The cost of downtime MUST NOT outweigh the benefits of patching
The cost of downtime is similar to the cost of defense per round. Patches prevent loss of points over multiple rounds and are thus favorable to not patching.
SLA / DEF / ATK points SHOULD be normalized against teams, services and total flagstores
Attack, defense and SLA points are not normalized to the number of teams, services or total flagstores.
SLA SHOULD decrease fairly with every missing flag in the recovery period
SLA points awarded from recovering services do not scale with amount of uptime in the recovery period.
Flag value SHOULD be calculated independent of its flagstore
Flag value is not scaled to the number of flagstores and thus independent.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale linearly with the amount of flags lost.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
Flag value is scaled by the total amount of flagstores.
The cost of downtime MUST NOT outweigh the benefits of patching
In the worst-case, SLA and defense points cancel out, but the attacker still gains points relative to the victim.
SLA / DEF / ATK points SHOULD be normalized against teams, services and total flagstores
Attack, defense and SLA contribute independently to the total score, and defense and SLA points are roughly similar in scale.
SLA SHOULD decrease fairly with every missing flag in the recovery period
SLA points awarded from recovering services do not scale with amount of uptime in the recovery period.
Flag value SHOULD be calculated independent of its flagstore
SLA points are typically not significantly larger than defense points.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale non-linearly with the amount of flags lost. Beyond the first capture of a flag, the points lost due to defense do not increase.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
Based on the default constants for
ATTACK,SLAandDEF, teams receive more SLA points than they lose through defense, but attackers may gain significantly more points than awarded through SLA.The cost of downtime MUST NOT outweigh the benefits of patching
Attack, defense and SLA contribute independently to the total score, and defense and SLA points are roughly similar in scale.
SLA / DEF / ATK points SHOULD be normalized against teams, services and total flagstores
Attack and defense points are not scaled to the amount of teams.
SLA SHOULD decrease fairly with every missing flag in the recovery period
SLA points awarded from recovering services do not scale with amount of uptime in the recovery period.
Flag value SHOULD be calculated independent of its flagstore
Flag value is scaled to the amount of flagstore per service, not to the total amount of flagstores.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale linearly with the amount of flags lost
Flag value MUST diminish with more successful attacks
Flag values scales with the difference in score between attacker and victim, but not the difficult of exploiting the specific vulnerability.
Perfect SLA MUST be worth more than any attacker's relative gain
Perfect SLA is worth more than an attacker's gain, since turning off a service would mean a loss of competitiveness, which undermines the purpose of tactically disabling it.
The cost of downtime MUST NOT outweigh the benefits of patching
Attack points are scaled with SLA points, disincentivizing patching when gains from attacking are high.
SLA / DEF / ATK points SHOULD be normalized against teams, services and total flagstores
SLA, defense and attack points are not normalized against the number of teams, services and flagstores.
SLA SHOULD decrease fairly with every missing flag in the recovery period
The formula does not feature a grace period, and such the points are divided fairly.
Flag value SHOULD be calculated independent of its flagstore
Flag value is not scaled to the amount of flagstores, and thus independent of flagstore.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale non-linearly with the amount of flags lost. Beyond the first capture of a flag, total score does not decrease with more flags lost.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
In the worst-case, SLA and defense points cancel out, but the attacker still gains points relative to the victim.
The cost of downtime MUST NOT outweigh the benefits of patching
Attack, defense and SLA contribute independently to the total score, and defense and SLA points are roughly similar in scale.
SLA / DEF / ATK points SHOULD be normalized against teams, services and total flagstores
Attack, defense and SLA are not normalized against the number of teams, services and flagstores.
SLA SHOULD decrease fairly with every missing flag in the recovery period
The formula does not feature a grace period, and such the points are divided fairly.
Flag value SHOULD be calculated independent of its flagstore
Flag value is not scaled to the amount of flagstores, and thus independent of flagstore.
Total score MUST increase with more flags captured
TBD
Total score MUST decrease with more flags lost
TBD
Flag value MUST diminish with more successful attacks
TBD
Perfect SLA MUST be worth more than any attacker's relative gain
TBD
The cost of downtime MUST NOT outweigh the benefits of patching
TBD
SLA / DEF / ATK points SHOULD be normalized against teams, services and total flagstores
TBD
SLA SHOULD decrease fairly with every missing flag in the recovery period
TBD
Flag value SHOULD be calculated independent of its flagstore
TBD