How do I filter on multiple values?

Modified on Wed, 28 May at 2:30 PM

Gravwell's search offers a ton of flexibility and power when exploring your data. Most extraction modules include the ability to filter during the extraction process, which is generally one of the most effective ways to optimize your query by reducing the amount of data that needs to be read and processed further down to the search pipeline. ex.

tag=palo_traffic ax src dst dport==80 action proto | table

But how can you provide multiple values in an AND/OR scenario to filter? It depends on the Module in question. There are a few filter modules which natively will allow multiple values. For example:

Words

The words module by default will apply AND logic when given multiple values. For example, If you are looking in your Apache logs and wanting to find results with both Mozilla and Firefox, your search may look like this:

tag=apache words Mozilla Firefox

But what if you wanted to look for either Firefox or Edge based browsers? the -or flag comes to the rescue, allowing you to invert the standard AND logic into OR logic, allowing any matching term to match

tag=apache words -or Firefox Edg

The ip module is another which will default to an AND logic when provided multiple values. For example, if you wanted to filter out the 10.x.x.x or 192.168.x.x networks from a list of IP addresses, you could provide both filters when running the module

tag=json json ipaddr | ip ipaddr !~ 10.0.0.0/8 ipaddr !~ 192.168.0.0/16

If, however, you wanted the OR logic to match any potential match, the -or flag could be used. For example, if you only wanted to match 2 /24 subnets within the 10.0.0.0/8 block, you could use a query like:

tag=json json ipaddr | ip ipaddr ~ 10.10.20.0/24 ipaddr ~ 10.0.2.0/24

Grep

The grep module will treat multiple value with an OR logic, allowing any match to pass. For example, if you are looking for "foo" or "bar", you only need to provide both values to grep

tag=syslog grep "foo" "bar"

If you want to match both values and have grep treat the multiple values with AND Logic, you can use the -s flag to force a strict match of all values

tag=syslog grep -s "foo" "bar"

Advanced Filtering

There are many Modules, including most extraction modules, which do not have the ability to accept multiple values when doing the filtering. There are also use cases where you may want to have some more complicated combinations of potential filters. In these cases, the use of the eval module will likely be the best solution for you.

Basic filtering using the eval module will utilize the && and || operators to represent AND or OR respectively. You can also use Parentheses () to group your filter criteria. This will give you the ability to create simple filters with multiple possible values, to more complex filtering logic that can handle multiple potential combinations of values.

The results of the filter logic ultimately will come down to a final TRUE/FALSE evaluation on if there is a match between your filters and the data.   As a result,  be careful when combining AND and OR operators within your search to ensure the TRUE/FALSE result is what is expected.

tag=palo_traffic ax src dst dport action proto 
| eval (dport == 80 || dport == 443)
| table

tag=palo_traffic ax src dst dport action proto 
| eval (dst == 172.16.0.33 && action == "allow" && ( dport == 22 || dport == 3389 ))
| table

eval also has several built in helper functions that can simplify the process of filtering off several potential values within an EV. These functions can be added to your filter logic in line with the other criteria.

in will likely be the most useful of these helper functions when doing filtering. It will allow you to provide a comma separated list of values to check for within an EV. The use of a ! before the helper name will reverse the standard behavior of the helper and exclude values that match.

tag=palo_traffic ax src dst dport action proto 
| eval in(dport,"80","443","22","21")
| table

tag=palo_traffic ax src dst dport==80 action proto 
| eval !in(dport,"3389","22","21")
| table

tag=palo_traffic ax src dst dport action proto 
| eval ( action == "deny" && in(dport, "22","3389") && !in(dst,"10.0.0.1", "10.10.10.1","10.20.20.1") )
| table