Scoring Tenets
In this section we use RFC2119 keywords to specify scoring formula requirements.
Tenets
Total score MUST increase with more flags captured
The scoring formula must reward the effort and skill required to exploit more services, and thus to capture more flags.
Total score MUST decrease with more flags lost
The scoring formula must reward the effort and skill required to defend against attacks.
Flag value MUST diminish with more successful attacks
The scoring formula must reward the effort and skill required to exploit a vulnerability in proportion to its difficulty, inferred from the number of successful exploits.
Perfect SLA MUST be worth more than any attacker's relative gain
The scoring formula must reward participation in the CTF and therefore disincentivize intentionally shutting off services to prevent other teams from overtaking you.
The cost of downtime MUST NOT outweigh the benefits of patching
To accurately measure player effort and skill in exploiting and defending, the scoring formula must not put such a disproportionate emphasis on defense or SLA, e.g. by scaling attack points with either, that it disincentivizes patching.
SLA SHOULD decrease fairly with every missing flag in the retention period
We define the retention period as the number of rounds N (>= 0) that a service must retain deployed flags for to receive full SLA. The scoring formula should award partial SLA points according to the fraction of flags retrieved and make this information available to the players.
Flag value SHOULD be calculated independent of its flagstore
Vulnerabilities in services with many flagstores should not be worth less to exploit. The dynamic flag value calculation already ensures that more difficult exploits are rewarded.
Evaluation
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale linearly with the amount of flags lost.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
In the worst-case, when every team exploits a service, more defense points are lost than gained from SLA.
The cost of downtime MUST NOT outweigh the benefits of patching
The cost of downtime is similar to the cost of defense per round. Patches prevent loss of points over multiple rounds and are thus favorable to not patching.
SLA SHOULD decrease fairly with every missing flag in the retention period
SLA points awarded from recovering services do not scale with amount of uptime in the retention period.
Flag value SHOULD be calculated independent of its flagstore
Flag value is not scaled to the number of flagstores and thus independent.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale linearly with the amount of flags lost.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
Depending on the number of flagstores, more points can be lost from defense than are gained from SLA.
The cost of downtime MUST NOT outweigh the benefits of patching
The cost of downtime is similar to the cost of defense per round. Patches prevent loss of points over multiple rounds and are thus favorable to not patching.
SLA SHOULD decrease fairly with every missing flag in the retention period
SLA points awarded from recovering services do not scale with amount of uptime in the retention period.
Flag value SHOULD be calculated independent of its flagstore
SLA points are typically not significantly larger than defense points.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale non-linearly with the amount of flags lost. Beyond the first capture of a flag, the points lost due to defense do not increase.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
Based on the default constants for
ATTACK,SLAandDEF, teams receive more SLA points than they lose through defense, but attackers may gain significantly more points than awarded through SLA.The cost of downtime MUST NOT outweigh the benefits of patching
The cost of downtime is similar to the cost of defense per round. Patches prevent loss of points over multiple rounds and are thus favorable to not patching.
SLA SHOULD decrease fairly with every missing flag in the retention period
Independent of the amount of flags missing from the retention period, the service is awarded the same amount of SLA.
Flag value SHOULD be calculated independent of its flagstore
Flag value is scaled to the amount of flagstore per service, not to the total amount of flagstores.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale non-linearly with the amount of attackers.
Flag value MUST diminish with more successful attacks
Flag values scales inversely with the amount of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
For the given constants, the attacker's relative gain will always be less than the points awarded from SLA and
BASE_DEF.The cost of downtime MUST NOT outweigh the benefits of patching
For the given constants, it would take significantly more rounds than the amount spent unavaiable to recover the losses of SLA, which disincentivizes patching.
SLA SHOULD decrease fairly with every missing flag in the retention period
SLA does not decrease fairly with the amount of missing flags in the retention period.
Flag value SHOULD be calculated independent of its flagstore
Flag value is not scaled to the amount of flagstores, and thus independent of flagstore.
Total score MUST increase with more flags captured
Attack points scale linearly with the amount of flags captured.
Total score MUST decrease with more flags lost
Defense points scale linearly with the amount of flags lost
Flag value MUST diminish with more successful attacks
Flag values scales with the difference in score between attacker and victim, but not the difficulty of exploiting that specific vulnerability.
Perfect SLA MUST be worth more than any attacker's relative gain
Perfect SLA is worth more than an attacker's gain, since turning off a service would mean a loss of competitiveness, which undermines the purpose of tactically disabling it.
The cost of downtime MUST NOT outweigh the benefits of patching
Attack points are scaled with SLA points, disincentivizing patching when gains from attacking are high.
SLA SHOULD decrease fairly with every missing flag in the retention period
The formula does not feature a retention period, and such the points are divided fairly.
Flag value SHOULD be calculated independent of its flagstore
Flag value is not scaled to the amount of flagstores, and thus independent of flagstore.
Total score MUST increase with more flags captured
Score increases with attack, which scales with flags captured.
Total score MUST decrease with more flags lost
Score decreases with defense, which scales with flags lost.
Flag value MUST diminish with more successful attacks
A flag's value scales inversely with the number of captures.
Perfect SLA MUST be worth more than any attacker's relative gain
The maximum points gained by any attack (
flagstores * 2) is less than the minimum cost of downtime (sla_max = flagstores * 2 + 1).The cost of downtime MUST NOT outweigh the benefits of patching
The cost of downtime due to patching can be recovered in few subsequent rounds of prevented exploitation.
SLA SHOULD decrease fairly with every missing flag in the retention period
sla_ratiodecreases fairly with every missing flag in the retention period.Flag value SHOULD be calculated independent of its flagstore
Flag value does not depend on the amount of flagstores in the service.