Text Preprocessing

The Text Preprocessing component uses extremely fast regex-based text processing to remove specific types of characters from a String column and normalize the data as much as possible without over-processing.

This component eliminates the need to convert text to a Document type in order to preprocess it. It also executes extremely quickly compared to various other approaches, promoting scalability.

Each option in the configuration is processed independently of the others. There is an order of operations (which can be audited or edited by drilling into the component).

Input ports

1. Type: Table
   Input Table
   A table of input data with at least one String column.

Output ports

1. Type: Table
   Output Table
   A table in which the input table's selected column has been processed and replaced.