Challenge 21: Summarizing KNIME Forum Topics
Level: Hard
Description: The KNIME Forum contains thousands of topics on a variety of subjects, with interactions across many users. These features make it a great resource to experiment with text processing algorithms for natural language. This week, you will use your API interaction skills to fetch the latest forum topics, and then experiment with different text summarization techniques to check what subjects are the most talked about in the forum currently. You can even go one step further and create visualizations for these summaries. Hint 1: Check the Discourse API documentation. Hint 2: Here's an example of an initial request URL, as documented here.
Beginner-friendly objective(s): 1. Set up the initial data retrieval process by configuring the GET Request node to fetch the latest topics from the KNIME Forum. 2. Parse the JSON response to extract topic IDs. 3. Using the topic IDs from the first step, retrieve all posts in each topic along with topics' relevant details like title and author in a second request (per topic).
Intermediate-friendly objective(s): 1. Implement text processing techniques to clean and prepare the extracted data, including removing HTML tags, punctuation, and stop words. 2. Create a visualization of the most frequent bigrams using the NGram Creator and Tag Cloud nodes.
Advanced objective(s): 1. Integrate LLMs to summarize conversations from the forum topics, showcasing the power of LLMs in text analysis. 2. Develop a comprehensive visualization using the KNIME View nodes to display the summarized topics alongside their associated tag cloud and metadata.
Author: Armin Ghassemi Rudd
Remember to upload your solution with tag JKISeason4-21 to your public space on KNIME Community Hub. To increase the visibility of your solution, also post it to this challenge thread on KNIME Forum.