Spark Scorer

Compares two columns by their attribute value pairs and shows the confusion matrix, i.e. how many rows of which attribute and their classification match. The dialog allows you to select two columns for comparison; the values from the first selected column are represented in the confusion matrix's rows and the values from the second column by the confusion matrix's columns. The output of the node is the confusion matrix with the number of matches in each cell. Additionally, the second out-port reports a number of accuracy statistics such as True-Positives, False-Positives, True-Negatives, False-Negatives, Recall, Precision, Sensitivity, Specificity, F-measure, as well as the overall accuracy and Cohen's kappa.

Input ports

1. Type: Spark Data
   Spark DataFrame/RDD
   Arbitrary input Spark DataFrame/RDD with at least two columns to compare.

Output ports

1. Type: Table
   Confusion matrix
   The confusion matrix.
2. Type: Table
   Accuracy statistics
   The accuracy statistics table.