Lopata Hall, Room 101
"Hyperparameter Optimization for Bayesian Quadrature of Constrained Integrands"
Adviser: Roman Garnett
Quadrature is the problem of estimating intractable integrals, a problem that frequently arises in Bayesian machine learning settings. In this work, we present a method of setting model hyperparameters for a Bayesian quadrature (BQ) algorithm used to estimate specific kinds of intractable integrals, namely ones where the integrand is known a priori to be non-negative everywhere. The BQ algorithm models a transformation of the integrands instead of the integrand itself; by doing so, the algorithm weakly incorporates the prior knowledge. Our proposed hyperparameter optimization method optimizes the hyperparameters in the original space of the integrand as opposed to in the transformed space, resulting in a model that better explains the actual data. Experiments on both synthetic and real-world data demonstrate that our proposed methodology results in more-accurate integral estimates than other commonly-used methods for hyperparameter optimization.
"Deep Embedding Logistic Regression"
Adviser: Yixin Chen
Logistic regression (LR) is used in many areas due to its simplicity and interpretability. While at the same time, those two properties limit its classification accuracy. Deep neural networks (DNNs), instead, achieve state-of-the-art performance in many domains. However, the nonlinearity and complexity of DNNs makes it less interpretable. To balance interpretability and classification performance, we propose a novel nonlinear model, Deep Feature Embedding Logistic Regression (DELR), which augments LR with a nonlinear dimension-wise feature embedding. In DELR, each feature embedding is learnt through a deep and narrow neural network and LR is attached to decide feature importance. A compact and yet powerful model, DELR offers great interpretability: it can tell the importance of each input feature, yield meaningful embedding of categorical features, and extract actionable changes, making it attractive for tasks such as market analysis and clinical prediction.