Bayesian optimization

[4][5] The term is generally attributed to Jonas Mockus [lt] and is coined in his work from a series of publications on global optimization in the 1970s and 1980s.

, is continuous and takes the form of some unknown structure, referred to as a "black box".

The posterior distribution, in turn, is used to construct an acquisition function (often also referred to as infill sampling criteria) that determines the next query point.

Another less expensive method uses the Parzen-Tree Estimator to construct two distributions for 'high' and 'low' points, and then finds the location that maximizes the expected improvement.

[11] They all trade-off exploration and exploitation so as to minimize the number of function queries.

The maximum of the acquisition function is typically found by resorting to discretization or by means of an auxiliary optimizer.

The approach has been applied to solve a wide range of problems,[12] including learning to rank,[13] computer graphics and visual design,[14][15][16] robotics,[17][18][19][20] sensor networks,[21][22] automatic algorithm configuration,[23][24] automatic machine learning toolboxes,[25][26][27] reinforcement learning,[28] planning, visual attention, architecture configuration in deep learning, static program analysis, experimental particle physics,[29][30] quality-diversity optimization,[31][32][33] chemistry, material design, and drug development.

[36] The performance of the Histogram of Oriented Gradients (HOG) algorithm, a popular feature extraction method, heavily relies on its parameter settings.

Optimizing these parameters can be challenging but crucial for achieving high accuracy.

[36] A novel approach to optimize the HOG algorithm parameters and image size for facial recognition using a Tree-structured Parzen Estimator (TPE) based Bayesian optimization technique has been proposed.

[36] This optimized approach has the potential to be adapted for other computer vision applications and contributes to the ongoing development of hand-crafted parameter-based feature extraction algorithms in computer vision.