Skip to content

All Flags are not Created Equal

Scoring formula from a paper by students of the Norwegian University of Science and Technology (NTNU) on scoring in Jeopardy and Attack-Defense competitions.

Summary

The paper analyzes many different scoring formulas including those used for FaustCTF 2024 and SaarCTF 2024 to collect requirements for a good scoring formula.

The checker returns one of three results for each service: up, recovering and down. A service is considered recovering if SLA checks both suceeded and failed in atleast one round of the recovery period.

The following python pseudo-code captures the team score calculation:

WEIGHT_DEF = 1
WEIGHT_RANK = 1

def score(rnd, team):
    attack = defense = sla = 0
    for service in services:
        for flag in service_flags(rnd, team, service):
            num_defended = len(teams) - len(captures(flag))
            if num_defended > 0:
                defense += 2 + 1 / num_defended
            if checker_status(rnd, service) == "up":
                sla += 1 + WEIGHT_DEF + WEIGHT_RANK
            elif checker_status(rnd, service) == "recovering":
                sla += (1 + WEIGHT_DEF + WEIGHT_RANK) / 2
    for victim in teams:
        for service in services:
            for flag in service_flags(rnd, victim, service):
                if team in captures(flag):
                    attack += 1 + WEIGHT_DEF + 1 / len(captures(flag))
                    rank_v = scoreboard_pos(rnd, victim)
                    rank_a = scoreboard_pos(rnd, team)
                    if rank_v < rank_a:
                        attack -= 4 / 5 * ((rank_v - rank_a) / len(teams)) ** 2
    return attack + sla + defense

def scores():
    return {rnd: {team: score(rnd, team} for team in teams} for rnd in rounds}

Review

  • Difficult to reason about and does not deduce constants / formula logically
  • When all teams are exploited in a service, no team loses defense points for that service (weird but improbable edge condition).
  • Flag value and thereby attack points are not normalized over the number of flagstores, violating Tenet 6

  1. Scoring formula was derived from paper and its implementation in ECSC 2023. ↩