Variables

How do I use a variable?

In the Hop user interface all places where you can enter a variable have a ‘$’ symbol to the right of the input field:

You can specify a variable by referencing it like this:

TIP You can see a list of defined variables by using CTRL-SPACE (CMD-SPACE on OSX) in the input field. This helper will insert a selected variable into the input field.

Please note that you don’t have to specify a variable in these places and that you can combine the variable with other information like:

In rare cases you might have a need to enter non-character values as separators in ‘binary’ text files with for example a zero byte as a separator. In those scenarios you can use a special ‘variable’ format:

A few examples:

How can I define variables?

Variables can be defined and set in all the places where it makes sense:

• in hop-config.json when it applies to the installation

• in an environment configuration file when it concerns a specific lifecycle environment

• In a project

• As a default parameter value in a pipeline or workflow

• Using the transform in a pipeline

• Using the Set Variables action in a workflow

• When executing with

Variables are local to the place where they are defined. This means that setting them in a certain place means that it’s going to be inherited from that place on. This also means that’s important to know where variables can be set and used and the hierarchy behind it.

Hierarchy

All system properties defined in the Hop configuration file ‘hop-config.json’ are available as variables as well as all Java system properties. You can define new system properties in the Hop GUI using the system properties editor:

You can also edit the ‘hop-config.json’ file manually:

{
  "systemProperties" : {
    "MY_SYSTEM_PROPERTY" : "SomeValue"
  }

You can also use the hop-config command line tool to define system properties:

System properties get set in Java Virtual Machine that Hop is running. This means that you should limit yourself to only those variables which are really system specific.

As you can read in the documentation regarding environments you can set variables there as well. This helps you configure folders and other things which are environment specific.

You can set those in the Environment settings dialog or using the command line:

sh hop-config.sh -e MyEnvironment -em -ev VARIABLE1=value1

You can specify variables here to make a pipeline or workflow run in an engine agnostic way. As an example you can have the same pipeline run on Hadoop with Spark and specify an input directory using hdfs:// and on Google DataFlow using gs://

You can define variables in a workflow either with the “Set Variables”, “JavaScript” actions or by defining parameters. Parameters are variables which have a description and can have a default value.

You can define variables in a pipeline either with the “Set Variables”, “JavaScript” transforms or by defining parameters. Parameters are variables which have a description and can have a default value.

IMPORTANT Since in pipelines all transforms run in parallel you should never try to set and use the same variable in the same pipeline.

The following variables are available in Hop through Tools → Edit config variables

Environment variables

Set the environment variables listed below in your operating system to configure Hop’s startup behavior:

HOP_AUDIT_FOLDER

Set this variable to a valid path on your machine to store Hop’s audit information. This information includes last opened files per project, zoom size and lots more.

HOP_CONFIG_FOLDER

Hop stores your configuration in the config folder by default. Set this environment variable to point to a folder outside of your Hop installation to keep your configuration, projects and environment list etc, no matter which Hop version or installation you use.

HOP_PLUGIN_BASE_FOLDERS

Set this variable to point Hop to a comma separated list of folders where you want Hop to look for additional plugins.

The variable which points to a shared folder with JDBC drivers in them.