Parameter Optimization Loop Start (Table)

This loop starts a parameter optimization loop. It takes a table as input. In the dialog you can map the parameters of the table to the outgoing flow variables. The loop will vary these parameters following a certain search strategy. Each parameter is output as a flow variable. The parameters can then be used inside the loop body either directly or by converting them with a Variable to Table node into a data table.
If a parameter shall be available inside the loop, but not be optimized, you can set start = stop within the input table.
Step size equals 0 is not allowed unless start = stop .
Currently four search strategies are available:

• Brute Force: All possible parameter combination (given the intervals and the step sizes) are checked and the best is returned.
• Hillclimbing: A random start combination is created and the direct neighbors (respecting the given intervals and step sizes) are evaluated. The best combination among the neighbors is the start point for the next iteration. If no neighbor improves the objective function the loop terminates.
• Random Search: Parameter combinations are randomly chosen and evaluated. The specified start and stop values define the parameter space from which a parameter combination is randomly drawn. Additionally, an optional step size can be defined to restrict the possible parameter values. The loop terminates after a specified number of iterations or, if early stopping is activated, when for a specified number of rounds the objective value has not improved. Note, that it is drawn with replacement. While duplicate parameter combinations will be processed just once, each of them still counts as an iteration. Due to that, it may happen that actually less loop iterations are processed than defined.
• Bayesian Optimization (TPE): This strategy consists of two phases. The first one is the warm-up in which parameter combinations are randomly chosen and evaluated. Based on the scores of the warm-up rounds, the second phase tries to find promising parameter combinations which are then evaluated. The algorithm used by this strategy is based on an algorithm published by Bergstra et al. (see link further down) and uses Tree-structured Parzen Estimation (TPE) in the second phase to find good parameter combinations. The specified start and stop values define the parameter space from which a parameter combination is randomly drawn. Additionally, an optional step size can be defined to restrict the possible parameter values. The loop terminates after a specified number of iterations. Note, that it is drawn with replacement. While duplicate parameter combinations will be processed just once, each of them still counts as an iteration. Due to that, it may happen that actually less loop iterations are processed than defined.

See also for Bayesian optimization: Algorithms for Hyper-Parameter Optimization by Bergstra et al.

Input ports

1. Type: Table
   Parameter Table
   A table containing the parameters and their attributes name , start , stop , stepsize and the optional parameter type for this loop. Parameter type should be either Number (integer) or Number (double) .

Output ports

1. Type: Flow Variable
   Parameters
   A parameter combination as flow variables