Metadata Structure of Stream
- Former user (Deleted)
- Former user (Deleted)
Metadata Structure of Stream
This is a step to read the metadata of the incoming stream.
Metadata Structure
Option |
Definition |
|---|---|
Step Name |
The name of the step |
Output Row |
Do you want to output the rows (Y/N) |
Field for Row |
If "output row" is selected, name the field that will contain the output row data |
Outputs
Field Name |
Contents |
|---|---|
Position |
The position of the field in the inbound data stream |
Fieldname |
The field name in the inbound data stream |
Comments |
|
Type |
String, Integer, Number etc |
Length |
The field length |
Precision |
Applicable for Integer, Number and -1 if not applicable (e.g. Strings) |
Origin |
The step where the meta data was derived from |
Example
You can create an input step from a database of your choice and then create an input hop to this step.
You may choose to create inputs from two different databases which *should* have the same structure and use a "Merge rows (diff)" step to compare the structure. If doing this, note that the case of field names are sometimes different and that the data types are also detected differently from different database engines. In this instance, I use a modified javascript to perform a "lower" on the "Fieldname" before comparing this to the other stream in the "Merge rows (diff)". It may help to sort the result by "position" to help identify where a column mismatch causes a "new" and "deleted" record.