Introducing Sigma Filters

This post is all about Sigma – the generic detection format for security professionals. If you're curious or are unfamiliar with what Sigma is, you can check out that here.

As any Security Operations Centre (SOC) analyst or engineer will tell you, there are two big challenges faced when hunting for threats within an environment.

The first is making sure you're detecting the right thing. This covers everything from making sure your logsource is onboarded properly, to all fields and values are named and mapped correctly, to testing and validating true-positive scenarios to ensure your detection rules are working as intended.

This also entails designing your rule in such a way that it can't be easily evaded by an attacker – so ensuring that the rule is wide enough to catch them given a wide range of circumstances and situations.*

The second – and arguably more frustrating – is minimising time triaging the wrong thing. This essentially entails filtering our "false-positives" detections, or alerts that have fired that are found to be benign.

Striking this balance is an art form.

Build rules too specific, and attackers might evade your detections. Build rules too generic, you and the rest of your SOC are going to grow linearly in accordance with the amount of triage needing to be done.

And casting a wide enough net of detection rules in order to ensure that attackers are promptly found vs ensuring your SOC isn't ballooning with staff due to the number of incoming alerts requiring triage – is a challenge very often & easily under-estimated.**

Exclusions in Sigma

Within the Sigma Detection ecosystem, these exclusions or "filters" are often represented as a not conditions applied to the initial detection.

title: Network Connection Initiated Via Notepad.EXE
author: EagleEye Team
logsource:
    category: network_connection
    product: windows
detection:
    selection:
        Image|endswith: '\notepad.exe'
    filter:
        DestinationPort: 9100
    condition: selection and not filter
falsepositives:
    - Printing documents via notepad might cause communication with the printer via port 9100 or similar.

In this example, we want to filter out port 9100 as it's a commonly used print protocol called AppSocket.

But what if you don't use 9100 at your organisation, and instead use the more secure IPP (631) ? Will this mean this alert will continue to fire every time someone prints from notepad? What if they open a document on a network drive?

The real problem with this approach is that filters that apply to everyone won't capture the reality of your unique environment.

If you're like most, you'll download the Sigma rule into your SIEM, intertwine your own environments' concerns with the concern of the detection itself – making some hybrid creature of detection and exclusion.

# ...
detection:
    selection:
        Image|endswith: '\notepad.exe'
    filter_printing:
        DestinationPort: 631
        DestinationIp|cidr: 
            - 192.168.24.0/24   # Printer Network
    filter_file_share:
        DestinationPort: 
            - 445
            - 139
        DestinationIp|cidr: 
            - 192.168.32.0/24   # Fileshare Network
    condition: selection and not (filter_printing or filter_file_share)
# ...

Now you've "tuned" this rule for your organisation, you're now unable to share this Sigma rule with anyone else, as they may use a different technologies, network address allocations etc.

The other issue is that now you've written this explicitly for a single rule, you might have to update 10s of other rules with the same kind of filter.

We wanted to fix this.

Introducing Sigma Filters

This is where Sigma Filters is designed to help with this. Sigma Filter rules are built as an extension on the Sigma rule format – that allow you to build filters that sit independently from your Sigma rules.

Made possible from the development around pySigma – a full re-write of the Sigma conversion strategy – we designed Filters to re-think how SOCs write, develop and integrate exclusions into their Sigma detection-as-code strategy.

Let's start with a simplified detection of the Sigma rule above.

title: Network Connection Initiated Via Notepad.EXE
name: network_connection_initiated_via_notepad_exe
author: EagleEye Team
logsource:
    category: network_connection
    product: windows
detection:
    selection:
        Image|endswith: '\notepad.exe'
    condition: selection
falsepositives:
    - Printing documents via notepad might cause communication with the printer via port 9100, 631.
    - Loading files from the internet, or an SMB share.

We can combine this Sigma rule with a "Sigma Filter" rule to ensure that these concerns remain separate. The goal of this is to ensure that the initial Sigma rule remains as generically applicable as possible.

Let's store this file in a new folder called filters at ./filters/windows/filter_out_file_sharing_and_printer_networks.yml

title: Filter out File Sharing and Printer Networks
description: >
    Filters out Company XXXXXXXXX file sharing servers and printer servers. These are often visted by our MS Office (Word) and notepad.txt users.
logsource:
    product: windows
filter:
    rules:
      - network_connection_initiated_via_notepad_exe
    filter_printing:
        DestinationPort: 631
        DestinationIp|cidr:
            - 192.168.24.0/24   # Printer Network
    filter_file_share:
        DestinationPort:
            - 445
            - 139
        DestinationIp|cidr:
            - 192.168.32.0/24   # Fileshare Network
    condition: not filter_printing and not filter_file_share

Note that we now supply the filter keyword instead of the detection keyword, and we supply a list of rule references in a list, either by the rule's name , or by the rule's id .

When we convert this using the sigma-cli command, we end up with the following query:

$ sigma convert -t lucene -p ecs_windows \
    --filter ./filters \
    ./rules/windows/network_connection/net_connection_win_notepad.yml

process.executable:*\\notepad.exe AND ((NOT (destination.port:631 AND destination.ip:192.168.24.0\/24)) AND (NOT ((destination.port:(445 OR 139)) AND destination.ip:192.168.32.0\/24)))

🎉 We've just applied our very first Filter to our detection rule.

The advantages of this at first don't seem to be very obvious, but as SOCs mature, over time, each detection rule becomes fairly unwieldily, eventually becoming an unmanageable mess in some situations.**

We're now also free to update the rules folder within our Sigma detection-as-code repo with any of the available rule-pack update releases, without having to worry about our exclusion lists being removed as well.

With this addition, Sigma Filters can also be applied to more than one of our existing Sigma rules. This happens to be extremely useful for excluding or only-including certain hosts, users, or software from an entire list of detection rules – so no more copying the filter into each Sigma rule.

We hope you are as excited as we are about this release, and as of v0.11.7, it's ready to use in pySigma, and soon to be merged in sigma-cli too.

If you're curious about some of the details, you can head over to the docs where we outline more about filters and how to use them.

Future Steps

Whilst this is an amazing first step for Sigma Meta Rules (with the announcement of Sigma Correlations not too long ago ) – there's a lot that Sigma HQ and the wider the community is going to have to do to make this feature really wonderful to use.

The first by-far – in my opinion – is to start work on a sigma list rules command, in which Sigma Filters are listed in tree structure alongside their rule references. This would be to help debug some resultant queries – as it's no longer obvious or transparent how a query comes together at conversion time.

$ sigma list rules ./rules

┬ Network Connection Initiated Via Notepad.EXE
├── Filter out File Sharing and Printer Networks
└── Filter out Service Accounts

Footnotes:

* For further context → It's important to note that most of these rules will return no results – as the behaviour the SOC is looking for only triggers when these malicious behaviours are enacted.

** Not only this – so often have I seen SOCs update a rule to exclude some behaviour – either in-SIEM or defined within a Github Pull Request – only to later find out that that same exclusion broke the initial detection, either by a typo or a badly escaped character