Using Timestamp and Eval to filter search results

Modified on Wed, 22 Oct at 3:13 PM

When searching Gravwell, the first and most powerful filter you can apply is time. Your Query Window, either in the GUI, or when specified within the query, allows you to easily filter your search to only show results which occurred during a single time window; however, what if you are only interested in events that happened within a subset of your larger time window?

i.e. events that occurred over the last 24 hours

Using The Time Module to create a filterable EV

The TIME module in your search pipeline can easily be used to extract a specific value, or section, of the event's timestamp and assign it to its own Enumerated Value which can then be referenced later in the search pipeline.

i.e.

time -oformat 15 TIMESTAMP hour

In this example, we are extracting just the hour field of the entries TIMESTAMP value in a 24hr format and assigning it to the new Enumerated Value "hour". By doing so we have removed any additional time information that is not needed for our new time sub-filter, resulting in a single EV containing a value of 0–23. This simplified integer can now be referenced by a module later in the query.

The Time module's output format references the "Magic Date" of Mon Jan 2 15:04:05 MST 2006
 in determining which value, and how you would like it displayed.  This translates to "Monday 01/02 03:04:05 '06 -07:00".  With this information you can easily extract other parts of the timestamp for different or additional values which can be used in your later filtering logic.

The Day of the Week does not have a numerical representation,  and will either be the 3 letter abbreviation, or the full day spelled out.

i.e.
"Mon" or "Monday"

Creating a custom Filter using the EVAL Module

The EVAL module can do many things, but in its simplest form can be used in advanced and custom filtering operations. In this use case, we only want entries from a small subset of time from the bigger search window our query is processing. To accomplish this, we will be referencing our newly created Enumerated Value from the Time module to determine if the entry fits within our desired window of time. This will require creating the logic to compare against our EV so that anything that results in a TRUE will get passed along, and anything evaluated as FALSE in our comparison will be filtered out of our final results.

i.e.

 eval ( hour >= 20 || hour <= 7)

In this example, we are looking for events that happened between the hours of 8pm-7am. If the "hour" EV is greater than or equal to 20, or less than or equal to 7, then it will be evaluated as TRUE.

If you are looking for events that happened on a particular day or days of the week, then your Enumerated Value will be of the string type, and not an integer. This will require a slightly different approach in your eval statement as you will be doing string comparisons. There are 2 ways to approach this comparison. The first would be with a standard equal comparison.

i.e.

eval ( day == "Sat" || day == "Sun" )

The second approach would be to use eval's built-in in function.

i.e.

eval in(day,"Sat","Sun","Fri")

The use of the Exclamation point (!) before any operator or function within eval will reverse the logic to a "NOT" type.

i.e.
"!="  means "Not Equal"
"!>=" means "Not Greater or equal"
"!in" means "NOT in"