Beginner Recipe: Use a Subset of an Array Field

Ambience 2020+ ETL Cookbook
,
1

Level: Beginner

Prerequisites: -

Steps Used:

  • Array Slice
  • Array Slice (Field)

Scenario:
How to modify a field to only retrieve a subset of the value from the initial array field values

ETL Array Slice:

  1. Create a Sample JSON data which includes an Array Field locally & save it as a .json file
    For example, sample Data Used: [sampledata3.json]
    [{"color": "red","value": "#f00","shades": ["cherry", "rose", "mahogany", "crimson"]}]

  2. Create a new ETL Chainset.
    For example,

    1070Ă—238 26.9 KB

  3. Add a new Blank Chain.
    For example,

  4. Add the Step, “URL Reader” & input in your .json file path over in the “URL” field.
    For example,

    Expand the icon on the top right corner for more details on this step.

Notes:

  • Do ensure that the path given in the “URL” field is declared as readable in your Ambience Server “application.conf” file.
  • This setting can be found under “elixir.data.url”

  1. Add the Step, “JSON Reader”

  2. Add the step, “Array Slice” & configure the necessary fields
    For example,

    Expand the icon on the top right corner for more details on this step.

  3. Run the ETL Steps

Output:
// Initial Data
|414.27981044571715x258

// After running the chain

[As configured in the Array Slice ETL step, Array index 0 & 1 are rendered in a new column called “result”]

ETL Array Slice (Field)

Sample Data Used:

[{"from": "0", "to": "2", "color": "red", "value": "#f00", "shades": ["cherry", "rose", "mahogany", "crimson"]}]

  1. After importing the data, head over to the ETL module.
    More information on how to Import a data can be found here:

  2. Create a new ETL Chainset.
    For example,

    1070Ă—238 26.9 KB

  3. Add a new Blank Chain.
    For example,

  4. Add the step, “MongoDB Definition” & “MongoDB Reader” to view the records.
    For example,


    [Configure the necessary fields]

  5. Add the step, “Array Slice(Field)” & configure the necessary fields
    For example,

    [Expand the icon on the top right corner for more details on this step.]

  6. Run the ETL Steps

Output:

// Initial Data

// After running the chain

691Ă—249 7.03 KB

[As configured in the Array Slice(Field) ETL step, Array index 0 & 1 are rendered in a new column called “result2” which are based on the column “from” & “to”]

Notes:

  • Array Slice example uses a JSON data structure.
  • Array Slice (Field) example uses a MongoDB database:collection
1 Like