Stream specs

    Below is the list of data types supported.

    Language definitions

    The supported property names.

    Property name Optional Description
    DATASOURCE false The value is determined by source type. The topic names list if it’s a MQTT data source. Please refer to related document for other sources.
    FORMAT true The data format, currently the value can be “JSON” and “BINARY”. The default is “JSON”. Check for more detail.
    KEY true Reserved key, currently the field is not used. It will be used for GROUP BY statements.
    TYPE true The source type, if not specified, the value is “mqtt”.
    StrictValidation true To control validation behavior of message field against stream schema. See Strict Validation for more info.
    CONF_KEY true If additional configuration items are requied to be configured, then specify the config key here. See for more info.
    SHARED true Whether the source instance will be shared across all rules using this stream
    TIMESTAMP true The field to represent the event’s timestamp. If specified, the rule will run with event time. Otherwise, it will run with processing time. Please refer to timestamp management for details.
    TIMESTAMP_FORMAT true The default format to be used when converting string to or from datetime type.

    Example 1,

    2.   (id bigint, name string, score float)
    3. WITH ( datasource = "topic/temperature", FORMAT = "json", KEY = "id");

    The stream will subscribe to MQTT topic topic/temperature, the server connection uses server key of default section in configuration file $ekuiper/etc/mqtt_source.yaml.

    • See for more info.

    The stream will subscribe to MQTT topic , the server connection uses settings of demo section in configuration file $ekuiper/etc/mqtt_source.yaml.

    By default, each rule will instantiate its own source instance. In some scenarios, users may need to manipulate the exact same data stream with different rules. For example, for the data of temperature from a sensor. They may want to trigger an alert when the average for a period of time is higher than 30 degree and trigger another alert when it is lower than 0. With default configuration, each rule creates a source instance and may receive data in different order due to network delay or other factors so that the average calculation may happen with different context. By sharing the instance, we can assure both rules are processing the same data. Additionally, it will have better performance by eliminating the overhead of instantiation.

    1. demo (
    2.         ...
    3.     ) WITH (DATASOURCE="test", FORMAT="JSON", KEY="USERID", SHARED="true");

    If the data type of the stream is unknown or varying, we can define it without the fields. This is called schema-less. It is defined by leaving the fields empty.

    2.   ()

    Schema-less stream field data type will be determined at runtime. If the field is used in an incompatible clause, a runtime error will be thrown and send to the sink. For example, where temperature > 30. Once a temperature is not a number, an error will be sent to the sink.

    See for more inforamtion of SQL language.

    Specify “BINARY” format for streams of binary data such as image or video streams. The payload of such streams is a block of binary data without fields. So it is required to define the stream as only one field of bytea. In the below example, the payload will be parsed into image field of demoBin stream.