Hub
NodeNode / Visualizer

Spark Statistics

Tools & Services Apache Spark Statistics
Drag & drop
Like
Copy short link

This node computes summary statistics for the selected input columns using the MLlib Statistics package.

Computed statistics:

  • Minimum value
  • Maximum value
  • Sample mean
  • Sample variance
  • L1 norm
  • L2 norm
  • Number of nonzero elements
  • Number of zero elements
  • Row count

Node details

Input ports
  1. Type: Spark Data
    Spark DataFrame/RDD to compute
    Spark DataFrame/RDD to compute statistics for.
Output ports
  1. Type: Table
    Statistics table
    Table with numeric values.

Extension

The Spark Statistics node is part of this extension:

Related workflows & nodes

  1. Will They Blend? Hadoop Hive meets Excel.
    Hadoop Excel Data blending
    +1
    Your flight is boarding now! This workflow demonstrates how the data stored in local big …
    knime > Examples > 10_Big_Data > 02_Spark_Executor > 12_Hadoop Hive meets Excel-Your Flight is boarding now
    knime
  2. Fraud_Detection_logit_spark
    andyhe8 > Public > 30416_Fraud_LOGIT > Fraud_Detection_logit_spark
    andyhe8
  3. Spark Label Encoding - prepare the data in local Big Data environment
    Knime Spark Hive
    +5
    s_401 - prepare label encoding with spark prepare the preparation of data in a big data e…
    mlauber71 > Public > kn_example_bigdata_h2o_automl_spark > s_401_spark_label_encoder
    mlauber71
  4. s_601 - Sparkling predictions and encoded labels - "the poor man's ML Ops" (on a Big Data System)
    Knime Spark Hive
    +7
    Sparkling predictions and encoded labels - "the poor man's ML Ops" (on a Big Data System)…
    mlauber71 > Public > kn_example_bigdata_h2o_automl_spark_46 > s_601_spark_label_encoder
    mlauber71