How to inject values into the search pipeline

Modified on Wed, 3 Sep at 9:53 AM

There will be times when you would like to inject a value or Enumerated Value (EV) into your search pipeline that is not already within your data. This may be because you want to add some arbitrary value or label to your data for sorting or ease of human understanding purposes. There may also be times when you wish to replace an empty or null value within your pipeline with a default value to avoid sorting issues or to provide clarity on the meaning of the search results.

In Gravwell, a null value is not the same as an empty value. This is especially relevant when looking at Enumerated Values. If something doesn't exist, then it has no value to be referenced when doing comparisons.

Manually Creating a New Enumerated Value

The eval module has the ability to create and inject a new Enumerated Value into the search pipeline, which can then be referenced and displayed by later modules. The syntax uses a single Equal ( = ) symbol to assign a value to a variable, instead of the double Equal ( == ), which is used to compare values.

e.g.

someVariable = someValue

This syntax can be used anywhere within an eval invocation, including inside or outside of loops. The value assigned using this syntax will be added to the entry as it continues to be processed by subsequent modules and the final render module, but will not be persistent between the different entries within the pipeline.

Persistent variables are also supported within eval, but are outside the scope of this KB article.

Checking if an Enumerated Value is null or Empty

As mentioned in the above note, Gravwell handles Empty values differently from null values. This becomes particularly evident when attempting to check for an Enumerated Value that doesn't exist within the entry. To understand the logic, consider this scenario:

eval (foo == ""  || foo != "" )

If "foo" does not exist as an Enumerated Value, should eval drop the entries or pass them along?

To that end, we decided to mark any expressions with EVs that don't exist as "undefined". The canonical way to check if an EV exists within the data is the has() function, which will return true if the specified Enumerated Value exists.

So to check if the Enumerated Value is null or empty, you would use an eval statement like:

eval (has(EVname) && EVname !="")

The use of has() is also a great way to add a validation that an Enumerated Value actually exists before running additional logic within your eval statements. This can help you avoid unexpected behavior in your logic.

Injecting a default EV value to empty or null Enumerated Values

When working with your data, an empty or null value can cause issues with your results, making the information unclear or easy to misinterpret. An example would be in cases where sorting data with an empty value could result in undesired results, or when your audience could easily misinterpret an empty field. In those situations, having a default value, such as a zero ( 0 ) or "No Data" text, can be preferable to help provide clarity in your results. Combining the steps above, where you can check for the empty/null value and then inject a value, can provide the solution.

eval if (!has(name) || name == "") {
     name = "No Data";
}

The addition of this eval line will add and/or assign the "No Data" value to the "name" EV if either the "name" EV doesn't exist, or it exists with no value currently assigned. It will not, however, touch or modify any entry that already has a value assigned to the "name" EV.