Problem:
You want to create a custom ambience ETL Step quickly, but the Ambience API seems to complex to learn
Solution:
Use the pre-fabricated template on giter8 to quickly create a skeleton module.
Discussion:
Although Ambience 2020 provides a huge wealth of ETL Steps, sometimes there is need to support custom features for a project, for instance to support connecting to an external device. In this instance, the development team for Ambience has kindly provided a simple to use giter8 template for your use.
The giter8 template is available here:
git.elixirtech.com/incubator/custom-etl-step.g8
Assuming that you have SBT installed, simply open your command prompt and change to your desired directory, then initialize the template:
cd <my-dir>
sbt new git@git.elixirtech.com/incubator/custom-etl-step.g8
This will bootstrap up a series of Scala source files and properties files that offer a basis for starting off your module.
Your next step will be to edit the main file which will reside somewhere in the folder hierarchy of the server subdirectory and will be named XXXStep.scala.
CustomStep.scala
The main meat of an ETL Step is the run() method, which takes in an monix.reactive.Observable of type Record - denoting the source records - and returns another monix.reactive.Observable containing the processed records. Record is the Ambience alias for BsonDocument in the MongoDB library.
The signature of the run method is:
run(Context)(Info)(From)
Context is not necessary for most ETL Steps, so it will not be covered in this guide.
Info represents the parameters for the step extracted from the UI and set up in the buildPanel method.
From is the input observable of records discussed earlier.
The buildPanel method defines the UI that you will see on the client side UI of the ETL Designer Module. Different UI components returning different types of values are created through factory methods on the PanelItem object in the ambience-crust module.
CustomCategory.scala
The key part of this file to override is the variable all. This is meant to contain all the ETL steps in the category if you are creating more than one step.
server/src/main/resources/
il8n/customcategory.properties
This file contains the il8n strings for the text used in your UI.
reference.conf
This is the configuration file that you will add all your custom settings to.
This file contains the il8n strings used by your ETL Step.