Chapter 4/Exercise 3. Document Vector Hashing Creation from the Movie Review Dataset

Read the Large Movie Review Dataset [1]. The dataset contains labeled reviews as positive or negative, as well unlabeled reviews.
Use the Strings to Document node to transforms the strings into documents. Tag all the words available in the documents and pre-process them by filtering the numbers, erase the punctuations, filter the stop words, convert the words in lower case, apply snowball stemmer and use the Tag Filter node to keep only the tagged words. Create the bag of words for the terms that have been tagged. Continue the analysis by filtering the Bag of Words to keep only the terms that occur at least 5 times in the documents. Split the collection of documents in two different sets of data. The Rule-based Row Splitter node needs to split the data in a way that the labels NEG or POS in the variable Category are available in the top output port. Instead the bottom output port should contain only the missing values for the variable Category.

[1] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. (2011). Learning Word Vectors for Sentiment Analysis. The 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011)

External resources

• www.knime.com/knimepress/from-words-to-wisdom