Rewrite Rules

Rewrite rules

Rewrite rules provide a fast, declarative way to normalize and enrich incoming events. Rules match on event attributes and perform straightforward updates such as setting fields, adding tags, replacing text, or dropping noise before storage.

When to use rewrite rules

• Simple match-and-modify changes that do not require branching logic.
• Standardizing program, host, or message text.
• Adding user_tags for dashboards, filters, or alerts.
• Quieting low-value events (for example, frequent keepalives).
• Parsing simple key-value pairs from messages.

YAML is recommended for authoring rewrite rules due to readability and change review. JSON is also supported if preferred by organizational standards.

How rewrite rules work (conceptual)

• Rules are evaluated in a deterministic order.
• Within a rule group, rules are processed top to bottom.
• Some groups can stop on first match to reduce processing.
• A matching rule can update core fields, add tags, perform controlled replacements in text fields, or drop the event.

Common actions include:

• Set or normalize program, host, or message.
• Add user_tags (for example, site, role, device_type).
• Replace text using safe patterns.
• Parse simple key-value pairs embedded in the message.
• Drop events that match well-defined noise patterns.

Authoring guidelines

• Keep rules simple and focused; prefer many small rules over one broad rule.
• Use consistent tag names across teams to support shared dashboards and alerts.
• Avoid high-cardinality tags unless necessary for operations.
• Document intent in the rule comments to aid reviews.
• Prefer adding structured user_tags rather than rewriting message text unless normalization is required.

Managing rules via CLI

Refer to the CLI section for full syntax and options: Command Line Tools --- Data Commands.

Typical operations include:

bash
Wrap: OnCopyFullscreen
# List rules and review status
logzilla rules list

# Enable, disable, or reload rules after changes
logzilla rules enable "My Rewrite"
logzilla rules disable "My Rewrite"
logzilla rules reload

# Check for recent rule errors
logzilla rules errors

Rule order

The parser runs the two rule types in a fixed sequence that no name or prefix changes: first every Lua rule (the rules the apps install), in three passes over all Lua rules (preprocess, process, postprocess), then every rewrite rule. A rewrite rule therefore always sees the fields and tags the apps have set, and nothing a rewrite rule does can feed an app.

Within each type, rules run in name order, which is why app rules carry a numeric prefix:

The two ends of the range differ in intent: 000 to 099 prepares data that later rules match on, while 991 to 999 tidies values after all apps have matched on the raw ones (the shipped program cleanup rule replaces junk program names with Unknown and strips paths, which must not happen before a vendor app has seen the original name).

Give a rewrite rule a prefix as well, for example --name "100-noise-reduction", to place it among the other rewrite rules deliberately; a name without a prefix sorts after every numbered rule. logzilla rules list shows both types sorted by name; read it as two sequences, Lua rules first, then rewrite rules:

bash
Wrap: OnCopyFullscreen
logzilla rules list

text
Wrap: OnCopyFullscreen
Name                     Source           Type    Status    Errors
-----------------------  ---------------  ------  --------  --------
100-noise-reduction      user             parser  enabled   -
200-cisco                cisco            lua     enabled   -
500-checkpoint           checkpoint       lua     enabled   -
950-secops               secops           lua     enabled   -
999-program-cleanup      program_cleanup  lua     enabled   -

Here the lua rules run first, 200-cisco through 999-program-cleanup, and 100-noise-reduction runs after them despite its lower prefix.

Verification workflow

• Generate representative events. For custom JSON input, see the HTTP Event Receiver.
• Reload rules and confirm the expected changes in search results.
• Inspect tags and fields with targeted queries and widgets.
• Monitor parser health using:

bash
Wrap: OnCopyFullscreen
# Parser statistics overview
logzilla events parser-stats

# Inspect values observed for key fields and tags
logzilla events values --scope fields --limit 50
logzilla events values --scope tags --limit 50

Best practices

• Start with a small set of rewrite rules and expand iteratively.
• Use rewrite rules for simple, deterministic edits; escalate to Lua for multi-step extraction, external lookups, or complex formatting.
  • Keep rule names and comments clear to simplify audits and reviews.

Capability quick reference

• Normalize core fields
  • Standardize program, host, or message text for consistency.
• Add or update tags
  • Enrich events with stable, searchable user_tags (for example, site, device_role).
• Search-and-replace in selected fields
  • Perform controlled text substitutions to clarify messages.
• Parse simple key=value pairs
  • Extract basic attributes embedded in message text.
• Drop matched noise
  • Suppress well-defined low-value events (for example, periodic keepalives).
• Stop further processing
  • End rule evaluation after a successful match when configured.

When more advanced logic is required:

• Use Lua rules for multi-step extraction, conditional branches, message reformatting, or external mappings/lookups.
• Package logic as a LogZilla App when distributing parsing together with dashboards and triggers for consistent rollout.

Match conditions (shared with forwarders)

Many LogZilla components, including rewrite rules and forwarders, use a common match structure to select events for processing. A match element contains:

• field: The event field to inspect (for example, program, host, message, or a user tag).
• op: The comparison operator.
• value: One or more values to compare against.

Multiple entries in a match list act as a logical AND: all conditions must be true for the event to match.

Commonly used fields in match conditions include:

• program: Syslog program/tag field.
• host: Source hostname.
• message: Event message text.
• facility: Syslog facility.
• severity: Syslog severity.
• MSWin EventID: Windows Event ID (requires the ms_windows app).
• User tag fields: Any user tag created by parsers or apps.

The following operators are supported in match conditions and are shared across rewrite rules and forwarders:

Several value entries are alternatives: the condition is true when any one of them matches. Wildcards belong to eq and ne: "*keepalive*" under op: eq matches any message containing "keepalive" in any letter case. The =* and !* operators test for a literal substring and treat * as an ordinary character. eq, ne, =~, and !~ ignore case by default; add ignore_case: false to a match entry to make them case-sensitive. A match entry without op uses eq. wm and nwm are the operators behind message search; in a rewrite rule their value is a plain list of words, the search bar's boolean syntax (|, !, parentheses) is not interpreted.

Format examples (YAML and JSON)

YAML is recommended for readability. JSON is also supported.

yaml
Wrap: OnCopyFullscreen
rewrite_rules:
  - match:
      - field: message
        op: eq
        value:
          - "*keepalive*"
    drop: true

  - match:
      - field: program
        op: eq
        value:
          - "netd"
    tag:
      site: west-dc
      device_role: edge

JSON example (equivalent):

json
Wrap: OnCopyFullscreen
{
  "rewrite_rules": [
    {
      "match": [
        { "field": "message", "op": "eq", "value": ["*keepalive*"] }
      ],
      "drop": true
    },
    {
      "match": [
        { "field": "program", "op": "eq", "value": ["netd"] }
      ],
      "tag": { "site": "west-dc", "device_role": "edge" }
    }
  ]
}

Related reading

• Overview and decision guide
• Lua rules
• Rule test fixtures
• LogZilla Apps
• Command Line Tools --- Data Commands