You want to create a custom ambience ETL Step quickly, but the Ambience API seems to complex to learn
Use the pre-fabricated template on giter8 to quickly create a skeleton module.
Although Ambience 2020 provides a huge wealth of ETL Steps, sometimes there is need to support custom features for a project, for instance to support connecting to an external device. In this instance, the development team for Ambience has kindly provided a simple to use giter8 template for your use.
The giter8 template is available here:
Assuming that you have SBT installed, simply open your command prompt and change to your desired directory, then initialize the template:
cd <my-dir> sbt new email@example.com/incubator/custom-etl-step.g8
This will bootstrap up a series of Scala source files and properties files that offer a basis for starting off your module.
Your next step will be to edit the main file which will reside somewhere in the folder hierarchy of the
server subdirectory and will be named XXXStep.scala.
The main meat of an ETL Step is the
run() method, which takes in an
monix.reactive.Observable of type
Record - denoting the source records - and returns another
monix.reactive.Observable containing the processed records.
Record is the Ambience alias for
BsonDocument in the MongoDB library.
The signature of the
run method is:
Context is not necessary for most ETL Steps, so it will not be covered in this guide.
Info represents the parameters for the step extracted from the UI and set up in the
From is the input observable of records discussed earlier.
buildPanel method defines the UI that you will see on the client side UI of the ETL Designer Module. Different UI components returning different types of values are created through factory methods on the
PanelItem object in the ambience-crust module.
The key part of this file to override is the variable
all. This is meant to contain all the ETL steps in the category if you are creating more than one step.
This file contains the il8n strings for the text used in your UI.
This is the configuration file that you will add all your custom settings to.
This file contains the il8n strings used by your ETL Step.