Java Utility to Add <ds:parameter-cache> Block to Repertoire 7/8.x Datasources

This sample utility (17.8 KB) sweeps a repository, finds every JDBC and Composite datasource (.ds) file that predates the Ambience 4.x / Repertoire 202x schema, and inserts the <ds:parameter-cache> block those engines expect, populated with exactly the ${...} parameters each datasource actually references, and without disturbing a single other byte of the file.

For JDBC datasources, the utility scans the file text for ${...} placeholders and caches
every distinct parameter name. For Composite datasources, it analyses the <graph> section to classify parameters as external (caller-supplied) or internal (resolved at runtime by a
ParametersGO Lookup node), and caches only the external ones. Other datasource types (e.g. Text) are skipped.

Sample files attached to this post:

  • ParameterCacheFixer.java: the utility. Single source file, no dependencies, runs directly
    on JDK 11+.
  • README.md: full reference: every option, the log format, the preservation guarantees, and a code map for anyone extending it.

Repertoire 8.x wrote datasources without a parameter cache. The newer engines declare the same parameters up front, as the first child of <ds:datasource>, and that block is what they read when a subreport datasource in an .rml / .docx template, or a composite datasource processor, needs to discover a datasource’s parameters without executing it. A datasource carried over from 8.x with no cache will not expose its ${...} parameters to those lookups, the subreport simply finds nothing to bind.

Opening and re-saving each datasource in Repertoire 202x Designer does regenerate the block, but it is manual, one file at a time, and it rewrites the whole document on the way out. A generic XSLT or DOM transform has the same problem from the other direction: re-serialising the document reflows whitespace, reorders attributes, normalises self-closing tags and discards comments across the entire file, so the diff on a production report datasource becomes unreviewable. This utility reads with a DOM parser but edits as text, so the only lines that change are the ones it adds.

1. Prerequisites

JDK 11 or newer (developed and tested on Temurin 17) and read/write access to the repository root. The root of the Ambience / Repertoire file system, the folder whose subdirectories hold your .ds files. Work against a checked-in or backed-up repository, or pass --backup (step 5).

2. Place the utility

Download ParameterCacheFixer.java from this post and put it anywhere convenient, it does not need to live inside the repository. Java 11+ runs a single source file directly, so there is no
compile step. To compile once and run repeatedly instead:

javac ParameterCacheFixer.java
java ParameterCacheFixer ROOT_PATH [LOG_FILE_OR_DIR] [options]

3. Dry-run the repository

Always do this first on a tree you have not swept before. The first argument is the repository root; the second, optional, is the log destination.

java ParameterCacheFixer.java "C:\reports\repository" --dry-run

Nothing is written to any .ds file. The log still is, so you get the complete picture of what the
real run would do.

Configuration Note: omit the second argument and the log lands at
ROOT_PATH/parameter-cache-fixer.log. It is a .log, so it is never picked up as a datasource, but if your deployment process copies the whole tree, pass an explicit path outside it instead, for
example "D:\logs\ds-migration.log".

4. Read the log

The log is UTF-8, echoed to stdout, and flushed after every line, so a killed run still leaves
something usable. Each file’s first line carries the path relative to the root; the full path line
beneath it is absolute. An update block is verbose by design:

[UPDATE] subdir\Example_Report.ds
         full path : C:\reports\repository\subdir\Example_Report.ds
         ds type   : JDBC
         action    : created <ds:parameter-cache>
         ${...}    : 2 distinct in 3 occurrence(s)
                     ${REPORT_ID}  x2  first seen line 37
                     ${BATCH_CODE}  x1  first seen line 41
         inserted  : 2 parameter element(s)
                     <ds:parameter name="REPORT_ID" type="" default-value="" />
                     <ds:parameter name="BATCH_CODE" type="" default-value="" />
         written   : 1947 bytes (UTF-8)

For a Composite datasource with a Lookup node, the log shows which parameters are external
(cached) and which are internal (excluded):

[UPDATE] subdir\Example_Composite.ds
         full path : C:\reports\repository\subdir\Example_Composite.ds
         ds type   : Composite
         action    : inserted <ds:parameter-cache> with 2 external parameter(s)
         lookup    : ParametersGO detected; graph analysis used to exclude internal parameters
         external  : ${REPORT_ID}
         external  : ${BATCH_CODE}
         internal  : ${LOOKUP_KEY} (resolved by Composite Lookup Processor, excluded from cache)
         written   : 4821 bytes (UTF-8)

Five status tags appear:

Tag Meaning
[UPDATE] File was rewritten.
[WOULD ] File would be rewritten (--dry-run).
[OK ] Already consistent; nothing to do.
[SKIP ] Not applicable — root element is not a datasource, or datasource type is not JDBC or Composite.
[ERROR ] Problem at the datasource level; file left untouched, sweep continues.

Read the [ERROR ] entries before applying. A file that is not well-formed XML is reported and
skipped rather than guessed at, so a single broken datasource never stops the sweep or corrupts anything. The run closes with a tally of scanned / updated / already-consistent / skipped / errors, and exits 1 if any datasource errored (2 for a bad invocation, 0 for a clean run) — useful if you drive this from a build step.

5. Apply the change

Drop --dry-run. Add --backup to keep a timestamped <name>.ds.<yyyyMMdd-HHmmss>.bak beside each rewritten file; one timestamp per run, so repeated runs do not clobber each other’s backups, and .bak is not .ds, so backups are never re-scanned.

java ParameterCacheFixer.java "C:\reports\repository" --backup

Four options are available:

Option Effect
--dry-run Report what would change; write no .ds file.
--prune Also delete <ds:parameter> entries in an existing cache that no ${...} references.
--backup Keep a timestamped .bak beside every rewritten file.
--empty-cache Emit an empty <ds:parameter-cache> for datasources that reference no ${...} at all.

Configuration Note: --prune is off by default, so a hand-configured parameter is never
silently lost. Without it, a cached entry that nothing references is reported as a WARNING and
kept, review those before deciding to prune.

6. Verify a rewritten datasource

The change is an insertion and nothing else. For a JDBC datasource, before:

<ds:datasource xmlns:ds="http://www.elixirtech.com/DataSource" name="Example_Report" type="JDBC" ...>
  <ds:schema case-sensitive="Yes">
  ...
      report.ID = ${REPORT_ID}

After:

<ds:datasource xmlns:ds="http://www.elixirtech.com/DataSource" name="Example_Report" type="JDBC" ...>
  <ds:parameter-cache>
    <ds:parameter name="REPORT_ID" type="" default-value="" />
  </ds:parameter-cache>
  <ds:schema case-sensitive="Yes">
  ...

For a Composite datasource, only the external parameters appear in the cache, parameters
resolved internally by a Lookup node are excluded:

<ds:datasource xmlns:ds="http://www.elixirtech.com/DataSource" name="Example_Composite" type="Composite" ...>
  <ds:parameter-cache>
    <ds:parameter name="REPORT_ID" type="" default-value="" />
    <ds:parameter name="BATCH_CODE" type="" default-value="" />
  </ds:parameter-cache>
  <ds:schema case-sensitive="Yes">
  ...

Two checks make this concrete and are worth running on your own tree. First, diff a rewritten file against its .bak, the diff must contain only the inserted parameter-cache lines. SQL
formatting, tabs, blank lines, -- comments inside <ds:sql>, attribute order and quoting style are all untouched; character encoding is read from the XML declaration and reused, a UTF-8 BOM is preserved, line endings are detected per file (CRLF if the file contains any, else LF), and
indentation is taken from the file’s own first child element, so a 4-space document gets 4-space
insertions. Every edit is re-parsed before being written, and if the result would not be well-formed
the file is left alone and an [ERROR ] logged.

Second, run the same command again. The utility is idempotent: the second pass must report
0 updated, with every previously updated file now [OK ]. On a mixed test tree of 11
datasources, covering no cache, a correct cache, a partially populated cache, a self-closed
<ds:parameter-cache/>, a stale entry, a default (unprefixed) namespace, malformed XML and a non-datasource root; the first pass reported 7 updated, 2 already consistent, 1 skipped and 1 error; the second reported 0 updated and 9 already consistent, with the skip and the error unchanged.

7. Reload the repository

Ambience / Repertoire caches datasource definitions in memory. Reload or restart the server, or
refresh the repository, so the new parameter caches are picked up. Then open a report whose subreport datasource was failing to bind and confirm its parameters now resolve.

Note: type and default-value are deliberately written empty, matching what Repertoire 202x emits for a freshly discovered parameter. If a particular lookup needs a declared type or a default, set it in the Designer afterwards, a later run of the utility preserves what you configured and only appends what is missing.

Note: placeholders with default values (${MY_PARAM##some_default}) are handled, the
## suffix is stripped and only the parameter name is cached.

Note: placeholders inside XML comments are ignored, as is anything already inside the
parameter-cache block (so a default-value="${X}" never counts itself). SQL line comments are not masked, however they are SQL, not XML, and the engine still substitutes a ${...} sitting on a commented-out SQL line, so those are cached too. If you have genuinely dead parameters buried in commented-out SQL, they will appear in the cache; delete the dead SQL and re-run with --prune.

Note: parameter names are treated as case-sensitive, matching the engine. ${report_id} and ${REPORT_ID} are two different parameters, and a file containing both earns a warning, because it is usually a typo rather than intent.

1 Like