Avro Input0



The Avro Input step decodes binary or JSON Avro data and extracts fields from the structure it defines, either from flat files or incoming fields.

Source tab

Option

Definition

Avro source is in file

Indicates the source data comes from a file.

Avro source is defined in a field

Indicates the source data comes from a field, and you can select an incoming field to decode from the Avro field to decode from drop-down box. In this mode of operation, a schema file must be specified in the Schema file field.

Avro file

Specifies the file to decode.

Avro field to decode from

Specifies the incoming field containing Avro data to decode.

JSON encoded

Indicates the Avro data has been encoded in JSON.




Schema tab

Option

Definition

Schema file

Indicates an Avro schema file.

Schema is defined in a field

Indicates the schema specified to use for decoding an incoming Avro object is found within a field. When checked, this option enables the Schema in field is a path and Cache schemas options. This also changes the Schema file label to Default schema file, which the user can specify if an incoming schema is missing.

Schema in field is a path

Indicates that the incoming schema specifies a path to a schema file. If left unchecked, the step assumes the incoming schema is the actual schema definition in JSON format.

Cache schemas in memory

Enables the step to retain all schemas seen in memory and uses this before loading or parsing an incoming schema.

Field containing schema

Indicates which field contains the Avro schema.




Avro fields tab

Option

Definition

Do not complain about fields not present in the schema

Disables issuing an exception when specified paths or fields are not present in the active Avro schema. Instead a null value is returned. OR Instead the system returns a null value.

Preview

Displays a review of the fields or data from the designated source file.

Get fields

Populates the fields available from the designated source file or schema and gives each extracted field a name that reflects the path used to extract it.




Lookup fields tab

Option

Definition

Get incoming fields

Populates the Name column of the table with the names of incoming Kettle fields. The Variable column of the table allows you to assign the values of these incoming fields to variable. A default value (to use in case the incoming field value is null) can be supplied in the Default value column. These variables can then be used anywhere in the Avro paths defined in the Avro fields tab.