This loop starts a parameter optimization loop. In the dialog you can enter several parameters with an interval and a step size. The loop will vary these parameters following a certain search strategy. Each parameter is output as a flow variable. The parameters can then be used inside the loop body either directly or by converting them with a Variable to Table node into a data table.
Currently four search strategies are available:
- Brute Force: All possible parameter combination (given the intervals and the step sizes) are checked and the best is returned.
- Hillclimbing: A random start combination is created and the direct neighbors (respecting the given intervals and step sizes) are evaluated. The best combination among the neighbors is the start point for the next iteration. If no neighbor improves the objective function the loop terminates.
- Random Search: Parameter combinations are randomly chosen and evaluated. The specified start and stop values define the parameter space from which a parameter combination is randomly drawn. Additionally, an optional step size can be defined to restrict the possible parameter values. The loop terminates after a specified number of iterations or, if early stopping is activated, when for a specified number of rounds the objective value has not improved. Note, that it is drawn with replacement. While duplicate parameter combinations will be processed just once, each of them still counts as an iteration. Due to that, it may happen that actually less loop iterations are processed than defined.
- Bayesian Optimization (TPE): This strategy consists of two phases. The first one is the warm-up in which parameter combinations are randomly chosen and evaluated. Based on the scores of the warm-up rounds, the second phase tries to find promising parameter combinations which are then evaluated. The algorithm used by this strategy is based on an algorithm published by Bergstra et al. (see link further down) and uses Tree-structured Parzen Estimation (TPE) in the second phase to find good parameter combinations. The specified start and stop values define the parameter space from which a parameter combination is randomly drawn. Additionally, an optional step size can be defined to restrict the possible parameter values. The loop terminates after a specified number of iterations. Note, that it is drawn with replacement. While duplicate parameter combinations will be processed just once, each of them still counts as an iteration. Due to that, it may happen that actually less loop iterations are processed than defined.