Skip to menu

머신러닝 동계 워크샵


Inference with Normalized Random Measures

최승진 교수, POSTECH 

약력 Professor, Department of Computer Science and Engineering, POSTECH
Ph.D. in Electrical Engineering, 1996 University of Notre Dame, Indiana, USA
MS. in Electrical Engineering, 1989 Seoul National University
BS. in Electrical Engineering, 1987 Seoul National University
초록 Normalized random measures (NRMs) provide a broad class of discrete random measures that are often used as priors for Bayesian nonparametric models. A well-known example of NRM is Dirichlet process. Most of posterior inference methods for NRM mixture models rely on MCMC methods since they are easy to implement and their convergence is well studied. However, MCMC often suffers from slow convergence when the acceptance rate is low. Tree-based inference is an alternative deterministic posterior inference method, including where Bayesian hierarchical clustering (BHC) and incremental Bayesian hierarchical clustering (IBHC), which have been developed for DP and NRM mixture (NRMM) models, respectively. In this talk, I begin with an overview of BHC and IBHC. Then, I emphasize a hybrid inference algorithm for NRMM models, which combines the merits of both MCMC and IBHC. Trees built by IBHC outlines partitions of data, which guides Metropolis-Hastings procedure to employ appropriate proposals. Inheriting the nature of MCMC, our tree-guided MCMC (tgMCMC) is guaranteed to converge, and enjoys the fast convergence thanks to the effective proposals guided by trees.
Metric Learning for Nadaraya-Watson Kernel Regression

노영균 교수, SNU

약력 BK assistant professor, Department of Mechanical and Aerospace Engineering, Seoul National University (SNU)
Ph.D. in Computer Science from Interdisciplinary Program in Cognitive Science, SNU
BS. in Physics, POSTECH
초록 One common practice for reducing the mean square error (MSE) in kernel regression is the bandwidth selection method. However, only a handful of studies have been devoted to metric learning despite its significant advantages in reducing MSE, particularly for high-dimensional data. In this talk, I will present the metric learning method in a well-known Nadaraya-Watson (NW) kernel regression. In this work, we propose an effective way of designing a metric, and examine the impact of this choice of metric on MSE. A noteworthy feature of our approach is that the metric is determined from the ‘global’ configuration of data, which is typically unused in normal NW kernel regression. Subsequent theoretical and empirical analysis of our metric-based method confirms efficient reduction of the bias, and thus the MSE.
Belief Propagation for Large-scale Optimization

신진우 교수, KAIST

약력 Assistant Professor, Department of Electrical and Engineering, KAIST (2013 ~)
Postdoctoral Researcher, IBM T. J. Watson Research (2012-2013)
Postdoctoral Researcher, Algorithm and Randomness Center, Georgia Institute of Technology (2010-2012)
Ph.D. in Mathematics, MIT (2005-2010)
BS. in Mathematics and Computer Science, SNU (1997-2001)
초록 Belief propagation (BP) is a popular message-passing algorithm for computing a maximum-a-posteriori assignment in a graphical model. It has been shown that BP can solve a few classes of Linear Programming (LP) formulations to combinatorial optimization problems including maximum weight matching and shortest path. However, it has been not clear what extent these results can be generalized to. In this talk, I first present a generic criteria that BP converges to the optimal solution of given LP, and show that it is satisfied in LP formulations associated to many classical combinatorial optimization problems including maximum weight perfect matching, independent set, shortest path, network flow, traveling salesman, cycle packing and vertex cover. Using the criteria, we also construct the exact distributed algorithm, called Blossom-BP, solving the maximum weight matching problem over arbitrary graphs. In essence, Blossom-BP offers a distributed version of the celebrated Blossom algorithm (Edmonds 1965) jumping at once over many sub-steps of the Blossom-V (most recent implementation of the Blossom algorithm due to Kolmogorov, 2011). Finally, I report the empirical performance of BP for solving large-scale combinatorial optimization problems. This talk is based on a series of joint works with Sungsoo Ahn (KAIST), Michael Chertkov (LANL) and Sejun Park (KAIST).
Bayesian Reinforcement Learning

김기응 교수, KAIST

초록 TBA
The Automatic Statistician: A Relational Perspective

최재식 교수, UNIST

약력 Assistant Professor, School of Electrical and Computer Engineering, UNIST, 2013-present
Affiliate Research, Computational Research Division, Lawrence Berkeley National Lab, 2013-present
Postdoc Fellow, Computational Research Division, Lawrence Berkeley National Lab, 2013
Ph.D., Department of Computer Science, University of Illinois at Urbana-Champaign, 2012
B.S., Department of Computer Engineering, Seoul National University, 2004
초록 Gaussian processes (GPs) provide a general and analytically tractable way of capturing complex time-varying, nonparametric functions. The time varying parameters of GPs can be explained by a composition of base kernels such as linear, smoothness and periodicity in that covariance kernels are closed under addition and multiplication. The Automatic Bayesian Covariance Discovery (ABCD or the Automatic Statistician) system constructs natural-language description of time-series data by treating unknown time-series data nonparametrically using GPs. In this talk, I will introduce a relational kernel learning for the ABCD system which enable to model relationships among sets of time series data by finding shared structures. Also, I will demonstrate how the shared structure helps learning models more accurate for sets of regression problems with some synthetic data, US top market capitalization stock data and US house sales index data.
Trends in Deep Recurrent Neural Networks

윤성로 교수, SNU

약력 Associate Professor, Electrical and Computer Engineering, Seoul National University
Assistant Professor, Electrical Engineering, Korea University (2007-2012)
Senior Engineer, Intel Corporation, Santa Clara, California, USA (2006-2007)
Postdoctoral Scholar, Stanford University, California, USA (2006)
Ph.D. in Electrical Engineering, Stanford University, California, USA (2006)
M.S. in Electrical Engineering, Stanford University, California, USA (2002)
B.S. in Electrical Engineering, Seoul National University (1996)
초록 In this presentation, I will introduce the fundamentals of recurrent neural networks (RNNs) detailing their representation and training methods, followed by recent advances in deep RNN structures, training techniques, and other details. Emerging applications in various disciplines and underlying deep RNN technologies will also be presented, most of which are based on clever use of interconnected explicit memory units and heuristics for alleviating challenges in training. Additionally, I will update the recent progress of DeepSpark, a distributed deep learning framework under development at SNU in collaboration with POSTECH ML Center.