How To Tokenize Column

This recipe shows how to tokenize columns using JavaScript in a composite datasource.

Steps taken:

  1. Create a composite datasource (cmps-Concat.ds). Drag and drop the tabular datasource (Concat.ds) as the beginning of the flow.
  2. Add a “Derivative” element following the tabular datasource. In “Derivative > Derived” tab, add the following columns:(Column1) Name=dev1, Type=String, Value=Col1 + tokens[1];
    (Column2) Name=dev2, Type=String, Value=Col1 + tokens[2];
    (Column3) Name=dev3, Type=String, Value=Col1 + tokens[3];
  3. In “Derivative > JavaScript” tab, enter the following code:importScript("/Elixir/ER_Cookbook/lib/Tokenizer.js")
    var tokens = Col2.tokenize(",", “”, false);

HowToTokenizeColumn

To download the necessary files for this recipe, refer to the attached ZIP file.
HowToTokenizeColumn.zip