Can I filter which log files/entries are ingested via syslog?

Modified on Mon, 19 Aug, 2024 at 3:49 PM

Yes, this is either configurable at the log source (software writing to this log), within the Gravwell Simple Relay Ingester or a combination of both.


TABLE OF CONTENTS


Log Source


Typically, *nix services/software configurations include the ability to limit the syslog logging level (severity) to one of the following:



For example, Corelight@Home has the ability to limit the logging level to a specified severity as documented below:



Simply change the value of the "Corelight::syslog_severity" option found under the "# JSON over TCP syslog export" block in the corelight-softsensor.conf file:



In this case, the Corelight@Home services require a restart to complete this log level change, but now logs up to the newly configured severity will be flowing to Gravwell.


Gravwell Simple Relay


NOTE: Since Priority is a calculated value based upon Facility and Severity, this will change if multiple Facilities are forwarding logs to this listener.


Occasionally, it's not as easily tuned at the source as detailed above so to accomplish similar filtering on the Gravwell side, we'll need to use the regexrouter preprocessor.


First, we need to understand how the Priority is calculated: Priority = (Facility * 8) + Severity


Meaning the 2 bits of information we need are numerical values for facility and severity, but currently we only have their names:



This helpful chart allows us to convert the facility to a numerical value:



and then we can convert "info" to a value using the chart near the top of this page:



Using the Priority equation we should get <134> = (16 * 8) + (6): 



So if WARN is the most verbose severity we're after we need to keep events with the following Priority values:


128,129,130,131,132


Now in the gravwell_simple_relay.conf, we need to create a preprocessor to filter for these values:


[Preprocessor "limitSeverity"]
    Type=regexextract
    Regex="^(?P<log>\<1(2[89]|3[0-2])\>.*)$"
    Template="${log}"
    Drop-Misses=true


This regex string will only match incoming logs with one of the above Priority values while dropping those that do not match thus filtering by 













Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article