Regex Evaluation

Step name

Name of the step.

Note: This name has to be unique in a single transformation.

Field to evaluate

Name of the field from the incoming stream which is to be matched against the regular expression

Result Fieldname

The name of the output field (boolean). This field will be added to the output stream and indicate whether the value of the input field matched the regular expression. Y means the value of the input field matched the regular expression, N means it didn't match.

Create fields for capture groups

Enable this if you want to create new fields based on capture groups in the regular expression. Capturing groups are those parts of the regular expression pattern which are enclosed in a pair of left and right parenthesis. If this option is enabled, substrings of the input field value corresponding to the capturing groups in the regular expression will be extracted and stored in new output fields. If this option is enabled, the "Capture group fields" grid needs to define one field for each capturing group. 

Replace previous fields

This option is available in case the "Create fields for capture groups" option is enabled. When the "Replace previous fields" option is checked, fields created for capturing groups will replace existing fields in the incoming stream with the same name. If not enabled, new fields will be added to the output stream for each capturing group field.

Regular expression

Put here the regular expression to match. See the java.util.regex.Pattern javadoc for reference documentation of the particular regular expression syntax used by this step.

Use variable substitution

Enable this if your regular expression contains variable references. By enabling this, variable references will be expanded to their value before evaluating the regular expression pattern.

Capture group fields

Here you can specify the new fields for any substrings captured by the regular expression from the input string. If the "Create fields for capture groups" option is enabled, you need to use this grid to enter a field definition corresponding to each capturing group in the regular expression. The order of the fields is the same as the order of the capturing groups in the regular expression. The columns in the grid allow you to change to the required data type right away.

Note:  A capture group is a part of the expression between a matching pair of left and right parenthesis.

For example, suppose your input field contains a text value like "Author, Ann" - 53 posts. The following regular expression contains 4 capturing groups and can be used to parse out the different parts:
^"(([^{"]), ([}"]))" - (\d+) posts\.$

This expression implies 4 fields, one for each capturing group:

• Fullname: (([^{"]), ([}"]))
• Lastname: ([^"]+)
• Firstname: ([^"]+)
• Number of posts: (\d+)
  In this particualr example, a field definition must be present for each of these capturing groups. It is an error in case the number of field definitions does not match up with the number of capturing groups in the regular expression.
  Note that capturing groups can be nested. In the example above the fields Lastname and Firstname are correspond to the capturing groups that are themselves contained inside the Fullname capturing group

Ignore differences in Unicode encodings

Check to ignore differences.

Note: This may improve performance, but be sure you data only contains US ASCII characters.

Enables case-insensitive matching

By default, case-insensitive matching assumes that only characters in the US-ASCII charset are being matched. Unicode-aware case-insensitive matching can be enabled by specifying the 'Unicode-aware case...' flag in conjunction with this flag.

Note: You can also enable this via the embedded flag expression (?i).

Permit whitespace and and comments in pattern

When enabled, the step will ignore whitespace and embedded comments starting with # through the end of the line.
In this mode, you must use the \s token to match whitespace. (If this option is not enabled, any whitespace characters appearing in the regular expression are matched as-is).

Note: Comments mode can also be enabled via the embedded flag expression (?x).

Enable dotall mode

When enabled, the expression '.' matches any character including the line terminator. By default, this expression matches any character except line terminators.

Note: Dotall mode can also be enabled via the flag expression (?s).

Enable multiline mode

When enabled, the expressions '^' and '$' match just after or just before, respectively, a line terminator or the end of the input sequence. By default, these expressions only match at the beginning and the end of the entire input sequence.

Note: Multiline mode can also be enabled via the flag expression (?m)

Enable Unicode-aware case folding

When enabled, in conjunction with the Case-insensitive flag, case-insensitive matching is done in a manner consistent with the Unicode standard. By default, case-insensitive matching assumes that only characters in the US-ASCII charset are being matched.

Note: Unicode-aware case folding can also be enabled via the embedded flag expression (?u).

Enables Unix lines mode

When enabled, only the line terminator is recognized in the behavior of '.', '^', and '$'.

Note: Unix lines mode can also be enabled via the embedded flag mode (?d).