FaustCTF 2024 Scoring Formula

Scoring formula for FaustCTF 2024 organized by FAUST.

Summary

The total score of each team is calculated from offense, defense and sla components of every team for each of their services and rounds played.

The checker returns one of three results for each service: up, recovering and down. The result is up if all SLA checks pass, and down if some SLA checks do not pass. A service is considered recovering if flags for one round in the retention period could not be recovered, but the latest round passed SLA checks.

The following python pseudo-code captures the team score calculation:

type CheckerResult = Literal["up"] | Literal["recovering"] | Literal["down"]

@dataclass
class RoundStateFlagstore:
    lost: str | None # flag of the current round if stolen by any team
    captures: list[str] # flags of this flagstore captured from other teams

@dataclass
class RoundStateService:
    flagstores: list[RoundStateFlagstore]
    checker_result: CheckerResult
    team_results: list[CheckerResult]

@dataclass
class RoundState:
    services: list[RoundStateService]

def score(rounds: list[RoundState], owner: dict[str, str],
          captures: dict[str, int]):
    attack = defense = sla = 0
    for rnd in range(len(rounds)):
        for service in rnd.services:
            for flagstore in service.flagstores:
                for flag in flagstore.captures:
                    if owner[flag] != "NOP":
                        attack += 1 + 1 / captures[flag]

                if (flag := flagstore.lost) is not None:
                    defense -= captures[flag] ** 0.75

            if service.checker_result == "up":
                teams_up = rnd.team_results.count("up")
                teams_rec = rnd.team_results.count("recovering")
                sla += (len(teams_up) + 0.5 * len(teams_rec)) \
                    * len(rnd.team_results) ** 0.5
    return (attack, defense, sla)

Review

• Scoring is easy to understand and reason about
• Since total_captures_of(flag) is independent of round time, it may change after a previous attack score has been calculated, meaning efficient recalculation of team score each round is somewhat non-trivial. Additionally, score recalculation may cause total attack points to decrease from one round to the next, which tends to confuse new players.
• In the worst-case, when all teams capture flags of a service, the defense points \((N_\texttt{Teams} - 1)^{0.75}\) lost outweigh the SLA points \((N_\texttt{Teams})^{0.5}\) gained for that service, thus violating Tenet 4

Tenets

1. Total score MUST increase with more flags captured

   Attack points scale linearly with the amount of flags captured.

2. Total score MUST decrease with more flags lost

   Defense points scale linearly with the amount of flags lost.

3. Flag value MUST diminish with more successful attacks

   Flag values scales inversely with the amount of captures.

4. Perfect SLA MUST be worth more than any attacker's relative gain

   In the worst-case, when every team exploits a service, more defense points are lost than gained from SLA.

5. The cost of downtime MUST NOT outweigh the benefits of patching

   The cost of downtime is similar to the cost of defense per round. Patches prevent loss of points over multiple rounds and are thus favorable to not patching.

6. SLA SHOULD decrease fairly with every missing flag in the retention period

   SLA points awarded from recovering services do not scale with amount of uptime in the retention period.

7. Flag value SHOULD be calculated independent of its flagstore

   Flag value is not scaled to the number of flagstores and thus independent.