Spark Entropy Scorer

Scorer for clustering results given a reference clustering. Connect the Spark DataFrame/RDD containing a column with the reference cluster IDs as well as a column containing the clustering results to the input port. The respective columns can be selected in the dialog. After successful execution, the view will show entropy values (the smaller the better) and a quality value (in [0,1] - with 1 being the best possible value, as used in Fuzzy Clustering in Parallel Universes , section 6: "Experimental results").

Input ports

1. Type: Spark Data
   Spark DataFrame/RDD
   Arbitrary input Spark DataFrame/RDD with at least two columns, where one column contains the reference clustering and one the clustering that shall be scored.

Output ports

1. Type: Table
   Quality Table
   Table containing entropy values for each cluster. The last row contains statistics on the entire clustering. It corresponds to the table shown in the Statistics View.