Skip to content

DevRetreat21RuleDocTemplate

Jump to bottom
Christian Folini edited this page Jan 24, 2022 · 12 revisions

Planning how to document CRS rules in the future

Date: 2021-10-26
Participants: 10 Dev Retreat participants

This was a detailed 90 minute discussion covering two topics:

  • How rules should be documented in the future
  • The technical implementation of rule documentation

How Rules Should be Documented

Key
🟧 Will need to be added by hand
🟦 Will need to be added semi-by hand
  • Rule
    • 🟧 Title
    • 🟧 Description
      • Formal rule description (character or word limited)
      • Extended remarks (no length limit)
      • Operator, pattern, and rule chains: just include a copy of the actual rule?
      • Exploit
    • Per version
    • 🟧 References
      • General
      • CVE (?)
    • 🟦 Rule type (see below for detailed definition)
      • Blocking
      • Detection
      • Admin/scaffold rule
      • Config
    • 🟧 Instructions to build the rule plus source files
    • Link to rule tests
    • 🟦 Example payloads
      • Positive example
      • Negative example (optional)
    • Characteristics
      • Rule ID
      • Paranoia level
      • Severity (critical, warning, etc.)
      • Rule group and rule file
      • Tags
        • CAPEC
      • 🟦 Rule siblings/family
      • Targets
      • Phase

Rule types

Type Description
Blocking Disruptive action deny
Detection Rule ID from 910,000 - to 948999 and from 950,000 to 958,000, not including 9xx0xx block
Admin/scaffold rule 901xxx - 909xxx block, 9xx0xx rule blocks, 980xxx rule block
Config 900xxx rule block

Technical Implementation

  • Repository: The rule documentation will live in the same repository as the rules, but as separate files, e.g. /docs/rules/{v3.3.2,dev,latest}/912345.html.
  • Format: YAML will be used for rule documentation files.
  • There should be an option to enrich the rule files with the associated rule comments, for users who want/need rule documentation to exist in the same files as the rules (as is currently the case). It should be possible to add this functionality to the rulectl script.
  • Information should be automatically extracted from the rules, where possible.
  • The website should show rule documentation on a per-version/release basis, e.g. v3.2.1, latest, v3.4/dev.
  • It should be possible to download a PDF/HTML copy of the rule documentation for offline viewing (e.g. when deploying CRS on a secure/dark site.)

YAML Rule Documentation Draft Proposal (AH, Dec 2021)

Rule Documentation Template

---
title: Rule title

description:
  short: Character or word limited formal rule description
  extended: |
    Extended remarks with no length limit.
    Can be multi-line.
  exploit: |
    Description of the exploit the rule detects.
    Can be multi-line.
  rule: |
    Text copy of the rule itself.
    Can be multi-line.

crs_version: Relevant CRS version, as rules are documented per-version

references:
  - https://www.example.com/reference.html

rule_type: blocking / detection / admin-scaffolding / config (pick one)

build_instructions: |
  Instructions on how to build the rule plus any necessary source files.
  Can be multi-line.

link_to_rule_tests: ???

example_payloads:
  positive:
    - Positive payload which should cause the rule to match
  negative:
    - Negative payload which should not cause the rule to match

characteristics:
  id: Rule ID number
  phase: Rule phase: 1 / 2 / 3 / 4 / 5
  paranoia_level: Rule paranoia level
  severity: Rule severity: critical / error / warning / notice
  file: Which file/group the rule is in
  tags:
    -
  siblings:
    -
  targets:
    -
...

Rule Documentation Example

---
title: Invalid HTTP request line

description:
  short: Validate request line against the format specified in the HTTP RFC
  extended: |
    Uses rule negation against the regex for positive security. The regex specifies the proper construction of URI request lines such as:
      "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
    It also outlines proper construction for CONNECT, OPTIONS and GET requests.
  exploit: |
  rule: |
    SecRule REQUEST_LINE "!@rx (?i)^(?:(?:[a-z]{3,10}\s+(?:\w{3,7}?://[\w\-\./]*(?::\d+)?)?\/[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?|connect (?:(?:\d{1,3}\.){3}\d{1,3}\.?(?::\d+)?|[\w\-\./]+:\d+)|options \*)\s+[\w\./]+|get \/[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?)$" \
        "id:920100,\
        phase:1,\
        block,\
        t:none,\
        msg:'Invalid HTTP Request Line',\
        logdata:'%{request_line}',\
        tag:'application-multi',\
        tag:'language-multi',\
        tag:'platform-multi',\
        tag:'attack-protocol',\
        tag:'paranoia-level/1',\
        tag:'OWASP_CRS',\
        tag:'capec/1000/210/272',\
        ver:'OWASP_CRS/3.4.0-dev',\
        severity:'WARNING',\
        setvar:'tx.anomaly_score_pl1=+%{tx.warning_anomaly_score}'"

crs_version: v3.4/dev

references:
  - https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.2.1
  - http://capec.mitre.org/data/definitions/272.html

rule_type: detection

build_instructions: |
  Regexp generated from util/regexp-assemble/920100.data using Regexp::Assemble.
  To rebuild the regexp:
    cd util/regexp-assemble
    ./regexp-assemble.py 920100

link_to_rule_tests: ???

example_payloads:
  positive:
    - \tGET / HTTP/1.1
    - LONGIMAGINARYMETHOD / HTTP/1.1
  negative:
    - OPTIONS * HTTP/1.1
    - CONNECT www.example.com:80 HTTP/1.1

characteristics:
  id: 920100
  phase: 1
  paranoia_level: 1
  severity: warning
  file: REQUEST-920-PROTOCOL-ENFORCEMENT
  tags:
    - application-multi
    - language-multi
    - platform-multi
    - attack-protocol
    - paranoia-level/1
    - OWASP_CRS
    - capec/1000/210/2
  siblings:
    -
  targets:
    - REQUEST_LINE
...

Markdown Output Proposal (AH, Jan 2022)

Rationale

Basic Markdown serves as a lingua franca between documentation systems. It can be used to generate:

  • Plain HTML, for integration into HTML-based documentation systems:

    pandoc 920100.md -o 920100.html

  • reStructuredText (e.g. for Sphinx):

    pandoc 920100.md -o 920100.rst

  • AsciiDoc (e.g. for AsciiDoctor):

    pandoc 920100.md -o 920100.adoc

as well as PDF, ODT, DOCX output, and more

It should be possible anyone to integrate this Markdown output into their own documentation pipeline.

Markdown Rule Documentation Example

Note: The escaped backticks surrounding the rule code block, \```, are only to stop GitHub from interpreting the three backticks as a special mark. The backslash should not be present in real output: it should look like so: ```

# Rule 920100: Invalid HTTP request line

## Description

Validate request line against the format specified in the HTTP RFC

## Explanation

Uses rule negation against the regex for positive security. The regex specifies the proper construction of URI request lines such as:

"http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]

It also outlines proper construction for CONNECT, OPTIONS and GET requests.

## Exploit Details

## Build Instructions

Regexp generated from util/regexp-assemble/920100.data using Regexp::Assemble.

To rebuild the regexp:

cd util/regexp-assemble

./regexp-assemble.py 920100

## Example Payloads

### Positive

- ``\tGET / HTTP/1.1``
- ``LONGIMAGINARYMETHOD / HTTP/1.1``

### Negative

- ``OPTIONS * HTTP/1.1``
- ``CONNECT www.example.com:80 HTTP/1.1``

## References

- <https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.2.1>
- <http://capec.mitre.org/data/definitions/272.html>

## Rule Code

\```
SecRule REQUEST_LINE "!@rx (?i)^(?:(?:[a-z]{3,10}\s+(?:\w{3,7}?://[\w\-\./]*(?::\d+)?)?\/[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?|connect (?:(?:\d{1,3}\.){3}\d{1,3}\.?(?::\d+)?|[\w\-\./]+:\d+)|options \*)\s+[\w\./]+|get \/[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?)$" \
    "id:920100,\
    phase:1,\
    block,\
    t:none,\
    msg:'Invalid HTTP Request Line',\
    logdata:'%{request_line}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'platform-multi',\
    tag:'attack-protocol',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/210/272',\
    ver:'OWASP_CRS/3.4.0-dev',\
    severity:'WARNING',\
    setvar:'tx.anomaly_score_pl1=+%{tx.warning_anomaly_score}'"
\```

## Rule Characteristics

**CRS Version:** v3.4/dev

**Rule Type:** Detection

**Rule ID:** 920100

**Phase:** 1

**Paranoia Level:** 1

**Severity:** Warning

**File:** REQUEST-920-PROTOCOL-ENFORCEMENT

**Tags:**

- application-multi
- language-multi
- platform-multi
- attack-protocol
- paranoia-level/1
- OWASP_CRS
- capec/1000/210/2

**Siblings:**

**Targets:**

- REQUEST_LINE
Clone this wiki locally