Use Golang template to customize analaysis result in eKuiper

    The Golang template applies a piece of logic to the data, and then formats and outputs the data according to the logic specified by the user. The common usage scenario of the Golang template is in web development. For example, after converting and controlling a data structure in Golang, it is converted to HTML tags and output to the browser. eKuiper uses Golang template to implement “secondary processing” of the analysis results. Please refer to the following introduction from Golang.

    The Golang template provides some which allows users to write various control statements to extract content. For example,

    • Output different contents according to judgment conditions
    • Iterate through the data and process it

    Readers can see that actions are delimited by {{}}. During the use of eKuiper’s data templates, the output is generally in JSON format, and the JSON format is delimited by {}. Therefore, if the readers are not familiar with it, they will find it difficult to understand the functions of eKuiper’s data templates. For example, in the following example,

    1. {{if pipeline}} {"field1": true} {{else}}  {"field1": false} {{end}}

    The meaning of the above expression is as follows (please note the delimiter of action and the delimiter of JSON):

    • If the condition pipeline is met, the JSON string {"field1": true} is output
    • Otherwise, the JSON string {"field1": false} is output

    The Golang template can be applied to various data structures, such as maps, slices, channels, etc., and the data type obtained by the data template in eKuiper’s sink is fixed, which is a data type that contains Golang map slices. It is shown as follows.

    1. []map[string]interface{}

    The data flowing into the sink is a data structure of map[string]interface{} slice. However, when the user sends data to the target sink, it may need a single piece of data instead of all the data. For example, in this article of Integration of eKuiper and AWS IoT Hub , the sample data generated by the rule is shown below.

    1.  ...
    2.  "sendSingle": true,
    3.  "dataTemplate": "{{toJson .}}"
    • After setting sendSingle to true, eKuiper traverses the []map[string]interface{} data type that has been passed to the sink. For each data in the traversal process, the user-specified data template will be applied.

    Golang also provides some built-in functions. Users can refer to for more function information.

    Still for the above example, you need to do some conversion on the returned t_av (average temperature). The basic requirement of the conversion is to add different description text according to different average temperatures for processing in the target sink. The rules are as follows,

    • When the temperature is less than 30, the description field is “Current temperature is$t_av, it’s normal.”
    • When the temperature is greater than 30, the description field is “Current temperature is$t_av, it’s high.”

    Assuming that the target sink still needs JSON data, the content of the data template is as follows:

    1. ...
    2. "dataTemplate": "{\"device_id\": {{.device_id}}, \"description\": \"{{if lt .t_av 30.0}}Current temperature is {{.t_av}}, it's normal.\"{{else if ge .t_av 30.0}}Current temperature is {{.t_av}}, it's high.\"{{end}}}"
    3. "sendSingle": true,

    ::: v-pre In the above data template, the built-in actions of {{if pipeline}} T1 {{else if pipeline}} T0 {{end}} are used, which looks more complicated. We can do a little adjustment, remove the escape and add abbreviation. The typesetting afterwards is as follows (note: when generating eKuiper rules, the following optimized typesetting rules cannot be passed in). :::

    1. {"device_id": {{.device_id}}, "description": "
    2.   {{if lt .t_av 30.0}}
    3.     Current temperature is {{.t_av}}, it's normal."
    5.     Current temperature is {{.t_av}}, it's high."
    6.   {{end}}
    7. }

    Use Golang’s built-in binary comparison function:

    • lt：less than
    • ge：greater or equal to

    It is worth noting that in the lt and ge functions, the type of the second parameter value should be consistent with the actual data type of the data in the map, otherwise an error will occur. As in the above example, when the temperature is greater than 30, because the type of actual average number in map is float, the value of the second parameter needs to be passed into 30.0, not 30.

    In addition, the template is still applied to each record in the slice, so you still need to set the sendSingle attribute to true. Finally, the content generated by the data template for the above data is as follows,

    By setting the sendSingle property of the sink to true, the slice data passed to the sink can be traversed. Here, we will introduce some more complex examples. For example, for the result of the sink which contains data of the nested array type, how can it realize the traversal by the traversal function provided in the data template.

    1. {"device_id":"1", 
    2.  "values": [
    3.   {"temperature": 10.5},
    4.   {"temperature": 20.3},
    5.   {"temperature": 30.3}
    6. }

    The requirement is:

    • When a value of temperature in the “values” array is found to be less than or equal to 25, add an attribute named description and set its value to fine.
    • When a value of temperature in the “values” array is found to be greater than 25, add an attribute named description and set its value to .
    1. "sendSingle": true,
    2. "dataTemplate": "{{$len := len .values}} {{$loopsize := add $len -1}} {\"device_id\": \"{{.device_id}}\", \"description\": [{{range $index, $ele := .values}} {{if le .temperature 25.0}}\"fine\"{{else if gt .temperature 25.0}}\"high\"{{end}} {{if eq $loopsize $index}}]{{else}},{{end}}{{end}}}"

    The data template is relatively complicated, which is explained below:

    ::: v-pre

    • {{$len := len .values}} {{$loopsize := add $len -1}}, this section executes two expressions. For the first one, len function gets the length of values in the data. For the second one, add decrements its value by 1 and assigns it to the variable loopsize. At present, since the operation of directly decrementing the value by 1 is not supported by the Golang expression, add is a function extended by eKuiper to achieve this function. :::

    ::: v-pre

    • {\"device_id\": \"{{.device_id}}\", \"description\": } This piece of template is applied to the sample data, and a JSON string {"device_id": "1", "description": } is generated. :::

    ::: v-pre

    • {{range $index, $ele := .values}} {{if le .temperature 25.0}}\"fine\"{{else if gt .temperature 25.0}}\"high\"{{end}} {{if eq $loopsize $index}}]{{else}},{{end}}{{end}} ,this section of the template looks relatively complicated. However, if we adjust it, remove the escape and add indentation, the typesetting is as follows which may be clearer (note: when generating the eKuiper rules, the following optimized typesetting rules cannot be passed in). :::

      1. {{range $index, $ele := .values}} 
      2.   {{if le .temperature 25.0}}
      3.     "fine"
      4.   {{else if gt .temperature 25.0}}
      5.     "high"
      6.   {{end}} 
      7.   {{if eq $loopsize $index}}
      8.     ]
      9.   {{else}}
      10.     ,
      11.   {{end}}

      The first condition judges whether to generate fine or high; the second condition judges whether to generate , to separate the array or ] at the end of the array.

    In addition, the template is still applied to each record in the slice. Therefore, we still need to set the sendSingle attribute to . Finally, the content generated by the data template for the above data is as follows:

    In addition, the eKuiper team plans to support custom extended template functions in sinks in the future, so that some more complex logic can be implemented within the function. For the users, they only need a call of a simple template function.