Array Paginate

This recipe is an example of how to design a simple ETL chainset for the step, “Array Paginate”.

Background Information:

  • This step converts each record in holding an array into a series of records out, each holding a slice from the array, where the length of the slice is limited to a fixed page size.
  • More information can be found in the ETL dictionary.

Prerequisite:

  • A dataset which consists of an Array field.

Notes:

  • This example uses a JSON data structure

ETL Array Paginate

  1. Create a Sample JSON data which includes an Array Field locally & save it as a .json file
    For example, sample Data Used: [sampledata2.json]
    {"color":"red", "shades":["cherry", "rose", "mahogany", "crimson", "garnet", "wine", "currant", "ruby", "burgundy"]}

  2. Create a new ETL Chainset.
    For example,

  3. Add a new Blank Chain.
    For example,

  4. Add the Step, “URL Reader” & input in your .json file path over in the “URL” field.
    For example,

    Expand the icon on the top right corner for more details on this step.

Notes:

  • Do ensure that the path given in the “URL” field is declared as readable in your Ambience Server “application.conf” file.
  • This setting can be found under “elixir.data.url”
  1. Add the Step, “JSON Reader”

  2. Add the Step, “Array Paginate” & configure the necessary parameters.
    For example,

    Expand the icon on the top right corner for more details on this step.

  3. Run the ETL Steps.


Output:

// Initial Data

// After running the ETL chain

[As configured in the “Page Size” for Array Paginate ETL step, each record only holds a maximum of 2 Array Slices.]