Bayesian Framework for Optimization and Generalization Techniques for Microwave Design

Many of the conventional global optimization methods cannot handle complex problems with high dimensionality as they are prone to challenges regarding initial point selection, non-convex behavior or expensive system queries. Most of these techniques take a step-by-step approach to black-box optimization, meaning the search for next parameter only involves analyzing the current point. This is done via simulations that are performed in proximity of the current best point to identify patterns or approximate gradients. This can lead to excessive simulations or inaccurate approximations in high dimensional systems that result in convergence to local extrema. Recently, machine learning techniques based on Bayesian Optimization (BO) has gained attention in the microwave community for design optimization of complex systems. In the context of optimization, this means that the next set of parameters are determined by analyzing all the previous observations, which enables capability of differentiating between local and global optima as well as intelligent parameter elimination and selection to avoid excessive searches. In BO, these previous observations are used to train a Gaussian Process (GP), which creates a predictive model of the underlying function to be optimized. The predictions provided by GP, along with the point-wise confidence intervals, are then used to guide optimization and mathematically ensure convergence to the global optima with high probability. In design space exploration, one of the challenges is generalization where results can be extrapolated outside of the training data range. This becomes an especially important problem when a surrogate model is already available but the response needs to be calculated outside the range of data validity without having to recreate a surrogate model. In this talk, we will first introduce GP theory and its application to microwave systems. We then present different training methods (probabilistic vs deterministic) and compare them in terms of model quality, model interpretability and computational load. Then, we introduce BO and present state-of-the-art methods such as Two-Stage Bayesian Optimization, Additive GP optimization for addressing high dimensional problems which covers the first part of the talk. In the second part of the talk we focus on generalization of a solution using Deep Neural Networks (DNN) along with the use of penalty functions and error bounds. By using “data is the model” paradigm, we modify the architecture of the network to minimize the expected error to enable extrapolation. We apply these methods to integrated wireless power transfer solutions and microwave interconnect structures that cover a broad frequency range.