Bayesian Optimization: An Approach for Optimal Hyperparameter Tuning
RE-WORK: AI in Finance 2023
Bayesian optimization is an approach for optimizing objective functions that are expensive to evaluate. “Vanilla” Bayesian optimization is typically best-suited for optimization over continuous domains of less than 20 dimensions and is tolerable to noisy function evaluations. It works by fitting a surrogate model to the objective function and quantifying uncertainty in the surrogate using a Bayesian machine learning technique, Gaussian process regression; we then build an acquisition function from this surrogate to decide where to sample. Although not limited to hyperparameter tuning, we will discuss how Bayesian Optimization is used to find optimal hyperparameters for machine learning models.