StringToWordVector

IDFTransform

Sets whether if the word frequencies in a document should be transformed into:
fij*log(num of Docs/num of Docs with word i)
where fij is the frequency of word i in document (instance) j.

TFTransform

Sets whether if the word frequencies should be transformed into:
log(1+fij)
where fij is the frequency of word i in document (instance) j.

attributeIndices

Specify range of attributes to act on. This is a comma separated list of attribute indices, with "first" and "last" valid values. Specify an inclusive range with "-". E.g: "first-3,5,6-10,last".

attributeNamePrefix

Prefix for the created attribute names. (default: "")

doNotOperateOnPerClassBasis

If this is set, the maximum number of words and the minimum term frequency is not enforced on a per-class basis but based on the documents in all the classes (even if a class attribute is set).

invertSelection

Set attribute selection mode. If false, only selected attributes in the range will be worked on; if true, only non-selected attributes will be processed.

lowerCaseTokens

If set then all the word tokens are converted to lower case before being added to the dictionary.

minTermFreq

Sets the minimum term frequency. This is enforced on a per-class basis.

normalizeDocLength

Sets whether if the word frequencies for a document (instance) should be normalized or not.

outputWordCounts

Output word counts rather than boolean 0 or 1(indicating presence or absence of a word).

periodicPruning

Specify the rate (x% of the input dataset) at which to periodically prune the dictionary. wordsToKeep prunes after creating a full dictionary. You may not have enough memory for this approach.

stemmer

The stemming algorithm to use on the words.

stopwords

The file containing the stopwords (if this is a directory then the default ones are used).

tokenizer

The tokenizing algorithm to use on the strings.

useStoplist

Ignores all the words that are on the stoplist, if set to true.

wordsToKeep

The number of words (per class if there is a class attribute assigned) to attempt to keep.

Class

No class, Relational class, Unary class, Binary class, Numeric class, Empty nominal class, Date class, Missing class values, Nominal class, String class

Attributes

Relational attributes, Empty nominal attributes, Date attributes, Binary attributes, String attributes, Missing values, Nominal attributes, Unary attributes, Numeric attributes

Min # of instances

0