2018 14th Symposium on Neural Networks and Applications (NEUREL)

Invited lecture 1

Deep Gaussian processes: Theory and applications

Lecturer: Petar M. Djurić
Department of Electrical and Computer Engineering
Stony Brook University, Stony Brook, NY 11794

Abstract

Gaussian processes are an infinite-dimensional generalization of multivariate normal distributions. They provide a principled approach to learning with kernel machines and they have found wide applications in many fields. More recently, with the advance of deep learning, the concept of deep Gaussian processes has emerged. Deep Gaussian processes have improved capacity for prediction and classification over standard Gaussian processes and models based on them preserve the features of allowing for full Bayesian treatment and for applications when the amount of available data is limited. The theory of recent progress in deep Gaussian processes will be presented and selected applications to problems in medicine will be provided.

Biosketch

Petar M. Djurić received the B.S. and M.S. degrees in electrical engineering from the University of Belgrade, Belgrade, Yugoslavia, respectively, and the Ph.D. degree in electrical engineering from the University of Rhode Island, Kingston, RI, USA. He is a SUNY Distinguished Professor and currently, he is a Chair of the Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY, USA. Djurić was a recipient of the IEEE Signal Processing Magazine Best Paper Award in 2007 and the EURASIP Technical Achievement Award in 2012. From 2008 to 2009, he was a Distinguished Lecturer of the IEEE Signal Processing Society. He is the Editor-in-Chief of the IEEE Transactions on Signal and Information Processing over Networks. Professor Djurić is a Fellow of IEEE and EURASIP.


Invited lecture 2

Quantum Low Entropy based Associative Reasoning –QLEAR Learning

Lecturer: Marko V. Jankovic
Department of Emergency Medicine
ARTORG Center for Biomedical Engineering Research, Diabetes Technology Group, University of Bern, Bern, Switzerland

Abstract

It is well known that the field of pattern recognition is concerned with the automatic discovery of regularities in data through the use of computer algorithms and with the use of these regularities to take actions such as classifying the data into different categories. There are many algorithms based on different theoretical backgrounds that could be used for pattern recognition in practical applications. Generally, most of the algorithms are applied in areas like classification, regression or change point detection. Recently, it has been shown that a probabilistic model based on two of the main concepts in quantum physics – a density matrix and the Born rule, can be suitable for the modeling of learning algorithms in biologically plausible artificial neural networks framework. It has been shown that the proposed probabilistic interpretation is suitable for modeling on-line learning algorithms for Independent /Principal/Minor Component Analysis, which could be realized on parallel hardware based on very simple computational units. Also, it has been shown that the quantum entropy of the system, related to that model, can be successfully used in the problems like change point or anomalies detection as well as simple classification problems. Here another application of the proposed quantum probabilistic model is going to be presented. A general paradigm called QLEAR learning (Quantum Low Entropy based Associative Reasoning) would be presented and tested in classification context. Proposed method potentially can overcome the problem that classifier performance depends greatly on the characteristics of the data to be classified. It is known that until now, there is no single classifier that works best on all given problems (a phenomenon that may be explained by the no-free-lunch theorem). Here we will try to propose a classification algorithm that, actually, automatically adjusts its performance according to characteristics of the data on which it is applied. An interesting aspect is that proposed method inherently solves the problem of unbalanced classes (classes that have significantly different size). The proposed paradigm can be applied in any area in which standard classification techniques are applied. The method is going to be analyzed in the context of classification, prediction and solving some mathematical problems. We’ll analyze, mainly, the case in which data is represented by vectors. Generalization toward multiway data would be discussed only on one example. The approach is based on the idea that classification can be understood as supervised clustering, where quantum entropy, in the context of the quantum probabilistic model, will be used as a “capturer” (measure, or external index) of the “natural structure” of the data. By using quantum entropy we don’t make any assumption about linear separability of the data that are going to be classified. The basic idea is to find close neighbours to a query sample and then use relative change in the quantum entropy as a measure of similarity of the newly arrived sample with the representatives of interest. In other words, method is based on calculation of quantum entropy of the referent system and its relative change with the addition of the newly arrived sample. Referent system consists of vectors/matrices that represent individual classes and that are the most similar, in Euclidean distance sense, to the vector that is analyzed. The classification problem is analysed in the context of measuring similarities to prototype examples of categories. The proposed method could be seen as a hybrid of nearest neighbor and optimization machine learning technique which deals naturally with the multiclass setting, has reasonable computational complexity both in training and at run time, and yields excellent results in practice.

Biosketch

Marko Jankovic received the B.S. , M.S. and PhD degrees in electrical engineering from the University of Belgrade, Belgrade (Serbia). Since November 1991 he works in the Institute of Electrical Engineering “Nikola Tesla”, Belgrade. From September 1997 till March 1999 he was a Research Student at Saitama University and the Electrotechnical Laboratory in Tsukuba. He was a Visiting Associate with the Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology, in July 2003. From January 2007 till September 2007 he was a part-time researcher in LABSP, BSI, RIKEN, Wakoshi, Japan. He was a JSPS postdoctoral research fellow with the Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology, from October 2007 till September 2009. Since May 2015 he is a senior researcher at ARTORG Biomedical Engineering Research, University of Bern – diabetes technology group and the Department of Emergency Medicine, Bern University Hospital “Inselspital”, Bern. He is interested in machine learning based on quantum probabilistic model, biologically inspired neural networks and their implementation in signal processing and brain modeling, real time control, design of software and hardware for microprocessor and PC based systems in power electronics and industry. He realized more than 25 real time control applications in power electronics and factory automation as the leader or a member of a team. He has published more than 90 scientific papers in peer reviewed journals and conferences.

NEUREL 2018

November 20 - 21, 2018

SAVA Center

Milentija Popovića 9

11070, Belgrade, Serbia

Contact:

Email: neurel@etf.rs

IEEE logo ICEF logo ETF logo DT logo