Service Tenets
In this section we use RFC2119 keywords to specify service / checker requirements.
Service
A service MUST be able to keep track of flags for a specified number of rounds
The service must keep track of flags for atleast one round to allow extraction thorugh exploitation. Requiring flags to be stored for more than one round e.g. allows players to investigate the artifacts related to network traffic before they are cleaned up by the service.
A service MUST NOT lose existing flags due to being restarted
The service must not hold important data in volatile memory, as this make the result of patching on SLA unnecessarily unpredictable, and violates an assumption many players will make about the default service behavior.
A service MUST be able to handle game traffic with reasonable compute resources
SLA must be a measure of the effort and skill of the players, not of the compute at their disposal.
A service MUST have at least one, but MAY have multiple flag stores
For each service to be exploitable, it must have atleast one vulnerability and therefore flag store. Multiple flag stores allows for more depth in service design and exploitation than a single flagstore with multiple vulns, since players will need to understand how data is stored in each of the flagstores, which may have a completely individual context.
It SHOULD NOT be practical to reimplement a service within the timeframe of the contest
For the CTF results to rank team skill and effort, all teams must participate and solve the same problems. When patching can be circumvented by replacing the service with a dummy implementing just enough features to pass the SLA check and avoid exploitation, this is not the case.
Vulnerabilities
A service MUST have one or more vulnerabilities, and SHOULD have at least one complex one
Each service must be exploitable, such is the nature of the competition, and thus have atleast one intended vulnerability. Atleast one complex vulnerability ensures the players' effort of analyzing each service in detail is rewarded.
An unpatched vulnerability MUST be exploitable and result in a correct flag
This is the definition of a service vulnerability.
An unpatched vulnerability SHOULD be exploitable within the round time.
Since the attack-defense format is designed around on a round-schedule, each action should complete within a single round for ease of reasoning about game state. Note, this tenet concerns vulnerability which can't be exploited within a single round, but not due to computational bounds.
An unpatched vulnerability SHOULD be exploitable over the course of the complete game
Each vulnerability should be available throughout the course of the game to be able to accuartely reason about how team performance changes over time without taking vulnerability changes into effect.
Every vulnerability MUST be patchable during contest and without breaking the checker
Patching a vulnerability must not require more time than is alloted in the contest while expending reasonable effort. Changes to the service necessary to fix a vulnerability must not impact the performance of the checker negatively.
A vulnerability MUST be exploitable with reasonable computing resources
The contest must reflect the teams effort and skill, not the extent of their computing resources.
A vulnerability MUST be exploitable with reasonable amounts of network traffic
The contest must reflect the teams effort and skill, not the extent of their network bandwidth.
A service SHOULD NOT have unintended vulnerabilities
For the organizers to prevent manipulation of vulnboxes by other teams via unintended remote-code execution, and accurately gauge the difficulty of each service to balance it against the scoring weight, they must be aware of every vulnerability in every service.
A service MUST NOT have vulnerabilities that allow only one attacker to extract a flag
For the contest to measure team effort and skill, granted points must not depend on infrastructure factors which allow one team to exploit a race-condition more consistently others. This includes flag-deletion.
A service SHOULD have atleast one vulnerability which is not easily replayable
To prevent other teams from reversing exploits soley from the game traffic, atleast one vulnerability of each service should not produce traffic which can easily be correlated, to the extent possible.
All vulnerabilities SHOULD NOT be exploited by a single team by the end of the game
To ensure players stay entertained throuhgout the CTF, and accurately measure the range of teams' effort and skill, no team should reach the maximum amount of points possible by exploiting every vulnerability before the end of the contest.
Checkers
A checker MUST check whether a flag is retrievable, and fail iff the flag is not retrievable
This is the definition of the Service-Level Agreement.
A checker MUST NOT crash or return unexpected results under any circumstances
For the competition to reflect the effort and skill of teams, their ability to patch vulnerabilities must be measured accurately, which is not possible if the checker is unable to do so.
A checker MUST NOT rely on information stored in the service in rounds before the flag was inserted
Relying on previously store information makes resoning about checker behavior unnecessarily complex, thus must be avoided.
A checker MUST log sufficiently detailed information that operators can handle complaints from participants
For organizers to properly attribute issues in the checker behavior and process user requests, which they must, the checker must provide logs that aid them in that task.
A checker MUST check the functionality of a service sufficiently to prevent reimplementation
The checker must check the service surface area sufficiently to prevent teams from reimplementing the required checks within the timeframe of the contest. This is the compliment requirement to Service Tenet 5.
A checker SHOULD not be easily identified by the examination of network traffic
To accurately detect whether services are available to all players, which is a prerequisite for the fair evaluation of teams' effort and skill, the checker must be indistinguishable from other traffic, such that it can not be fingerprinted and treated separately.
A checker SHOULD use unusual, incorrect or pseudomalicious input to detect network filters
Since patching in the contest is meant emulate the real-life equivalent, patches should not be based on specific user(-data) patterns, but on the interaction involved with the vulnerability.