Spark IoTDB Connecter

mvn clean scala:compile compile install

1. maven dependency

2. spark-shell user guide

spark-shell --jars spark-iotdb-connector-0.10.0.jar,iotdb-jdbc-0.10.0-jar-with-dependencies.jar
import org.apache.iotdb.spark.db._
val df = spark.read.format("org.apache.iotdb.spark.db").option("url","jdbc:iotdb://127.0.0.1:6667/").option("sql","select * from root").load
df.printSchema()
df.show()

3. Schema Inference

The existing data in the TsFile is as follows:

You can also use narrow table form which as follows: (You can see part 4 about how to use narrow form)

4. Transform between wide and narrow table

import org.apache.iotdb.spark.db._
val wide_df = spark.read.format("org.apache.iotdb.spark.db").option("url", "jdbc:iotdb://127.0.0.1:6667/").option("sql", "select * from root where time < 1100 and time > 1000").load
val narrow_df = Transformer.toNarrowForm(spark, wide_df)

5. Java user guide

import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.SparkSession;
import org.apache.iotdb.spark.db.*
public class Example {
  public static void main(String[] args) {
        .builder()
        .appName("Build a DataFrame from Scratch")
        .getOrCreate();
    Dataset<Row> df = spark.read().format("org.apache.iotdb.spark.db")
        .option("url","jdbc:iotdb://127.0.0.1:6667/")
        .option("sql","select * from root").load();
    df.printSchema();
    df.show();
    Dataset<Row> narrowTable = Transformer.toNarrowForm(spark, df)
    narrowTable.show()