ECSC 2024 Scoring Formula

Scoring formula for the European Cyber Security Challenge 2024 A/D CTF.

Summary

The total score of each team is calculated using a (pseudo) zero-sum scheme, where each team begins the CTF with the same, non-zero amount of points, and exploited teams redistribute a portion of their points to the attackers.

The following python pseudo-code captures the team scores calculation:

SCALE = 15 * sqrt(5)
NORM = log(log(5)) / 12
BASE = 5000

def score(teams: list[str]):
    # Initialize (partial) scores
    score = {team: {} for team in teams}
    sla = {team: {} for team in teams}
    attack = {team: {} for team in teams}
    defense = {team: {} for team in teams}
    for team in teams:
        score[team] = {service: BASE for service in services}
        attack[team] = {service: 0 for service in services}
        defense[team] = {service: 0 for service in services}
        sla[team] = {service: 0 for service in services}

    # Redistribute points based on attacks
    for rnd in rounds:
        for flag, attacker, victim in flags_stolen_in_round(rnd):
            for service in services:
                for _ in flags_stolen_from(team, service, rnd):
                    value = (1 + exp(NORM * (sqrt(score[attacker][service]) \
                          - sqrt(score[victim][service])))) * SCALE
                    attack[attacker][service] += value
                    defense[victim][service] += min(score[team][service], value)
        for team in teams:
            for service in services:
                score[team][service] = BASE + attack[team][service] \
                            - defense[team][service]

    # Accumulate score, weighted by SLA
    total = {team: 0 for team in teams}
    for team in teams:
        for service in services:
            sla_weight = sla[team][service] / len(rounds)
            total[team] += max(0, sla_weight * score[team][service])

    return total

Note how score[team][service] is recalculated each round and may turn negative although the total score total[team] can not.

Review

• Difficult to reason about and does not deduce constants / formula logically
• SLA is a multiplicative factor to the attack / defense score, this violates Tenet 5 since a large amount of attack points may be lost through downtime which are not outweighed by the saved defense points.
• Flags do not lose value based on the amount of successful attacks, violating Tenet 3
• Since the score gained from an attack scales exponentially with the difference of squares in the position, the formula may incentivize reaching a low scoreboard position while hoarding exploits in the early game before deploying all of them at once in a sort of 'rubber banding' strategy

Tenets

1. Total score MUST increase with more flags captured

   Attack points scale linearly with the amount of flags captured.

2. Total score MUST decrease with more flags lost

   Defense points scale linearly with the amount of flags lost

3. Flag value MUST diminish with more successful attacks

   Flag values scales with the difference in score between attacker and victim, but not the difficulty of exploiting that specific vulnerability.

4. Perfect SLA MUST be worth more than any attacker's relative gain

   Perfect SLA is worth more than an attacker's gain, since turning off a service would mean a loss of competitiveness, which undermines the purpose of tactically disabling it.

5. The cost of downtime MUST NOT outweigh the benefits of patching

   Attack points are scaled with SLA points, disincentivizing patching when gains from attacking are high.

6. SLA SHOULD decrease fairly with every missing flag in the retention period

   The formula does not feature a retention period, and such the points are divided fairly.

7. Flag value SHOULD be calculated independent of its flagstore

   Flag value is not scaled to the amount of flagstores, and thus independent of flagstore.