MongoDB Output

For additional information about MongoDB, see the MongoDB documentation.

Options

Transform name : Specify the unique name of the MongoDB Output transform in the pipeline.

The Configure connection tab enables you to specify the database and collection for your output.

Enter the following information in the transform fields:

The Output options tab provides additional controls for inserting data into a MongoDB collection. If the specified collection does not exist, it is created before a document is inserted.

Enter the following information in the fields on this tab:

Option	Description
Database	Specify the target database for the output. When a valid hostname and port has been set, you can click Get DBs to retrieve the names of existing databases within a selected database.
Collection	Specify the target collection for the output. When a valid hostname and port has been set, you can click Get Collections to retrieve the names of existing collections within a selected database. If the specified collection does not exist, it will be created before data is inserted.
Batch insert size	Specify the batch size for bulk insert operations. The default value is 100 rows.
Truncate collection	Select to delete existing data in the target collection before inserting new data.
Update	Sets the update write method for the specified database and collection. The Upsert and Modifier update options are not available unless the Update field is selected.
Upsert	Select to change the write method from insert to upsert. The upsert method replaces a matched record with an entire new record based on all the incoming fields specified in the Mongo document fields tab. A new record is created if match conditions fail for an update.
Multi-update	Select to update all matching documents for each update or upsert operation.
Modifier update	Select to enable modifiers ($ operators) to be used to modify individual fields within matching documents. All matching documents are updated when the Multi-update option is selected. To update more than one matching document, select Modifier update and Upsert. Selecting Modifier update, Upsert, and Multi-update applies updates to all matching documents, instead of just the first.
	Specify the minimum number of servers that must succeed for a write operation. The values are: -1 : Disables all acknowledgement of write operation errors 0 (Zero) : Disables basic acknowledgment of write operations, but returns information about socket excepts and networking errors 1 : Acknowledges write operations on the primary node >1 : Wait for successful write operations to the specified number of slaves, including the primary. Click Get custom write concerns to retrieve custom write concerns that you have stored in the repository.
w Timeout	Specify time (in milliseconds) to wait for a response to write operations before terminating the operation. Leave blank to never terminate.
Journaled writes	Select to set write operations to wait until the mongod (the primary daemon process for the MongoDB system) acknowledges the write operation and commits the data to the journal.
Read preference	Specify which node to read first: - - `Primary preferred` - `Secondary` - `Secondary preferred` - `Nearest` The default is . The Read preference is available when Modifier update is selected.
Number of retries for write operations	Specify the number of times that a write operation is attempted.
Delay, in seconds, between retry attempts	Specify the number of seconds to wait before the next retry.

Use the Mongo document fields tab to define how field values coming into the transform are written to a Mongo document. The Modifier policy column controls when the execution of a modifier operation affects a particular field. You can use modifier policies when the data for one Mongo document is split over several incoming Hop rows or when it is not possible to execute different modifier operations that affect the same field simultaneously.

There are 2 helper buttons you can use: * Get fields :Populates the Name column of the table with the names of the incoming fields. * Preview document structure : Opens a dialog showing the structure that will be written to MongoDB in JSON format.

Enter the following information in the fields on this tab:

Example

Here is an example of how you can define a document structure with an arbitrary hierarchy. Use the following input data and document field definitions to create the example document structure in MongoDB:

Input data

Document field definitions

Name	Mongo document path	Use field name	JSON	Match field for update	Modifier operation	Modifier policy
first	top1	Y	N	N	N/A	Insert&Update
last	array[O]	Y	N	N	N/A	Insert&Update
address	array[O]	Y	N	N	N/A	Insert&Update
age	array[O]	Y	N	N	N/A	Insert&Update

Document structure

{
  "top1" : {
    "first" : "<string val>"
   },
  "array" : [ { "last" : "<string val>" , "address" : "<string val>"}],
}

Use the Create/drop indexes tab to create and drop indexes on one or more fields. Unless unique indexes are being used, MongoDB allows duplicate records to be inserted. Indexing is performed after all rows have been processed by the transform.

You can use the Show indexes button to display a list of existing indexes.

Enter the following information in the fields in this tab:

Create/drop indexes example

The following options defines the creation of a compound index of the “first” and “age” fields in ascending order:

Index fields	Index opp	Unique	Sparse
top1.first,age	Create	N	N