Paper Group ANR 422
Mixing Complexity and its Applications to Neural Networks. Fast-Forward Video Based on Semantic Extraction. Detecting Qualia in Natural and Artificial Agents. How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models. Automatic Segmentation and Disease Classification Using Cardiac Ci …
Mixing Complexity and its Applications to Neural Networks
| Title | Mixing Complexity and its Applications to Neural Networks |
| Authors | Michal Moshkovitz, Naftali Tishby |
| Abstract | We suggest analyzing neural networks through the prism of space constraints. We observe that most training algorithms applied in practice use bounded memory, which enables us to use a new notion introduced in the study of space-time tradeoffs that we call mixing complexity. This notion was devised in order to measure the (in)ability to learn using a bounded-memory algorithm. In this paper we describe how we use mixing complexity to obtain new results on what can and cannot be learned using neural networks. |
| Tasks | |
| Published | 2017-03-02 |
| URL | http://arxiv.org/abs/1703.00729v1 |
| http://arxiv.org/pdf/1703.00729v1.pdf | |
| PWC | https://paperswithcode.com/paper/mixing-complexity-and-its-applications-to |
| Repo | |
| Framework | |
Fast-Forward Video Based on Semantic Extraction
| Title | Fast-Forward Video Based on Semantic Extraction |
| Authors | Washington Luis Souza Ramos, Michel Melo Silva, Mario Fernando Montenegro Campos, Erickson Rangel Nascimento |
| Abstract | Thanks to the low operational cost and large storage capacity of smartphones and wearable devices, people are recording many hours of daily activities, sport actions and home videos. These videos, also known as egocentric videos, are generally long-running streams with unedited content, which make them boring and visually unpalatable, bringing up the challenge to make egocentric videos more appealing. In this work we propose a novel methodology to compose the new fast-forward video by selecting frames based on semantic information extracted from images. The experiments show that our approach outperforms the state-of-the-art as far as semantic information is concerned and that it is also able to produce videos that are more pleasant to be watched. |
| Tasks | |
| Published | 2017-08-14 |
| URL | http://arxiv.org/abs/1708.04160v3 |
| http://arxiv.org/pdf/1708.04160v3.pdf | |
| PWC | https://paperswithcode.com/paper/fast-forward-video-based-on-semantic |
| Repo | |
| Framework | |
Detecting Qualia in Natural and Artificial Agents
| Title | Detecting Qualia in Natural and Artificial Agents |
| Authors | Roman V. Yampolskiy |
| Abstract | The Hard Problem of consciousness has been dismissed as an illusion. By showing that computers are capable of experiencing, we show that they are at least rudimentarily conscious with potential to eventually reach superconsciousness. The main contribution of the paper is a test for confirming certain subjective experiences in a tested agent. We follow with analysis of benefits and problems with conscious machines and implications of such capability on future of computing, machine rights and artificial intelligence safety. |
| Tasks | |
| Published | 2017-12-11 |
| URL | http://arxiv.org/abs/1712.04020v1 |
| http://arxiv.org/pdf/1712.04020v1.pdf | |
| PWC | https://paperswithcode.com/paper/detecting-qualia-in-natural-and-artificial |
| Repo | |
| Framework | |
How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models
| Title | How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models |
| Authors | Kathrin Grosse, David Pfaff, Michael Thomas Smith, Michael Backes |
| Abstract | Machine learning models are vulnerable to Adversarial Examples: minor perturbations to input samples intended to deliberately cause misclassification. Current defenses against adversarial examples, especially for Deep Neural Networks (DNN), are primarily derived from empirical developments, and their security guarantees are often only justified retroactively. Many defenses therefore rely on hidden assumptions that are subsequently subverted by increasingly elaborate attacks. This is not surprising: deep learning notoriously lacks a comprehensive mathematical framework to provide meaningful guarantees. In this paper, we leverage Gaussian Processes to investigate adversarial examples in the framework of Bayesian inference. Across different models and datasets, we find deviating levels of uncertainty reflect the perturbation introduced to benign samples by state-of-the-art attacks, including novel white-box attacks on Gaussian Processes. Our experiments demonstrate that even unoptimized uncertainty thresholds already reject adversarial examples in many scenarios. Comment: Thresholds can be broken in a modified attack, which was done in arXiv:1812.02606 (The limitations of model uncertainty in adversarial settings). |
| Tasks | Bayesian Inference, Gaussian Processes |
| Published | 2017-11-17 |
| URL | http://arxiv.org/abs/1711.06598v4 |
| http://arxiv.org/pdf/1711.06598v4.pdf | |
| PWC | https://paperswithcode.com/paper/how-wrong-am-i-studying-adversarial-examples |
| Repo | |
| Framework | |
Automatic Segmentation and Disease Classification Using Cardiac Cine MR Images
| Title | Automatic Segmentation and Disease Classification Using Cardiac Cine MR Images |
| Authors | Jelmer M. Wolterink, Tim Leiner, Max A. Viergever, Ivana Isgum |
| Abstract | Segmentation of the heart in cardiac cine MR is clinically used to quantify cardiac function. We propose a fully automatic method for segmentation and disease classification using cardiac cine MR images. A convolutional neural network (CNN) was designed to simultaneously segment the left ventricle (LV), right ventricle (RV) and myocardium in end-diastole (ED) and end-systole (ES) images. Features derived from the obtained segmentations were used in a Random Forest classifier to label patients as suffering from dilated cardiomyopathy, hypertrophic cardiomyopathy, heart failure following myocardial infarction, right ventricular abnormality, or no cardiac disease. The method was developed and evaluated using a balanced dataset containing images of 100 patients, which was provided in the MICCAI 2017 automated cardiac diagnosis challenge (ACDC). The segmentation and classification pipeline were evaluated in a four-fold stratified cross-validation. Average Dice scores between reference and automatically obtained segmentations were 0.94, 0.88 and 0.87 for the LV, RV and myocardium. The classifier assigned 91% of patients to the correct disease category. Segmentation and disease classification took 5 s per patient. The results of our study suggest that image-based diagnosis using cine MR cardiac scans can be performed automatically with high accuracy. |
| Tasks | |
| Published | 2017-08-03 |
| URL | http://arxiv.org/abs/1708.01141v1 |
| http://arxiv.org/pdf/1708.01141v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-segmentation-and-disease |
| Repo | |
| Framework | |
SGD Learns the Conjugate Kernel Class of the Network
| Title | SGD Learns the Conjugate Kernel Class of the Network |
| Authors | Amit Daniely |
| Abstract | We show that the standard stochastic gradient decent (SGD) algorithm is guaranteed to learn, in polynomial time, a function that is competitive with the best function in the conjugate kernel space of the network, as defined in Daniely, Frostig and Singer. The result holds for log-depth networks from a rich family of architectures. To the best of our knowledge, it is the first polynomial-time guarantee for the standard neural network learning algorithm for networks of depth more that two. As corollaries, it follows that for neural networks of any depth between $2$ and $\log(n)$, SGD is guaranteed to learn, in polynomial time, constant degree polynomials with polynomially bounded coefficients. Likewise, it follows that SGD on large enough networks can learn any continuous function (not in polynomial time), complementing classical expressivity results. |
| Tasks | |
| Published | 2017-02-27 |
| URL | http://arxiv.org/abs/1702.08503v2 |
| http://arxiv.org/pdf/1702.08503v2.pdf | |
| PWC | https://paperswithcode.com/paper/sgd-learns-the-conjugate-kernel-class-of-the |
| Repo | |
| Framework | |
A Hybrid Supervised-unsupervised Method on Image Topic Visualization with Convolutional Neural Network and LDA
| Title | A Hybrid Supervised-unsupervised Method on Image Topic Visualization with Convolutional Neural Network and LDA |
| Authors | Kai Zhen, Mridul Birla, David Crandall, Bingjing Zhang, Judy Qiu |
| Abstract | Given the progress in image recognition with recent data driven paradigms, it’s still expensive to manually label a large training data to fit a convolutional neural network (CNN) model. This paper proposes a hybrid supervised-unsupervised method combining a pre-trained AlexNet with Latent Dirichlet Allocation (LDA) to extract image topics from both an unlabeled life-logging dataset and the COCO dataset. We generate the bag-of-words representations of an egocentric dataset from the softmax layer of AlexNet and use LDA to visualize the subject’s living genre with duplicated images. We use a subset of COCO on 4 categories as ground truth, and define consistent rate to quantitatively analyze the performance of the method, it achieves 84% for consistent rate on average comparing to 18.75% from a raw CNN model. The method is capable of detecting false labels and multi-labels from COCO dataset. For scalability test, parallelization experiments are conducted with Harp-LDA on a Intel Knights Landing cluster: to extract 1,000 topic assignments for 241,035 COCO images, it takes 10 minutes with 60 threads. |
| Tasks | |
| Published | 2017-03-15 |
| URL | http://arxiv.org/abs/1703.05243v2 |
| http://arxiv.org/pdf/1703.05243v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-hybrid-supervised-unsupervised-method-on |
| Repo | |
| Framework | |
Effects of Images with Different Levels of Familiarity on EEG
| Title | Effects of Images with Different Levels of Familiarity on EEG |
| Authors | Ali Saeedi, Ehsan Arbabi |
| Abstract | Evaluating human brain potentials during watching different images can be used for memory evaluation, information retrieving, guilty-innocent identification and examining the brain response. In this study, the effects of watching images, with different levels of familiarity, on subjects’ Electroencephalogram (EEG) have been studied. Three different groups of images with three familiarity levels of “unfamiliar”, “familiar” and “very familiar” have been considered for this study. EEG signals of 21 subjects (14 men) were recorded. After signal acquisition, pre-processing, including noise and artifact removal, were performed on epochs of data. Features, including spatial-statistical, wavelet, frequency and harmonic parameters, and also correlation between recording channels, were extracted from the data. Then, we evaluated the efficiency of the extracted features by using p-value and also an orthogonal feature selection method (combination of Gram-Schmitt method and Fisher discriminant ratio) for feature dimensional reduction. As the final step of feature selection, we used ‘add-r take-away l’ method for choosing the most discriminative features. For data classification, including all two-class and three-class cases, we applied Support Vector Machine (SVM) on the extracted features. The correct classification rates (CCR) for “unfamiliar-familiar”, “unfamiliar-very familiar” and “familiar-very familiar” cases were 85.6%, 92.6%, and 70.6%, respectively. The best results of classifications were obtained in pre-frontal and frontal regions of brain. Also, wavelet, frequency and harmonic features were among the most discriminative features. Finally, in three-class case, the best CCR was 86.8%. |
| Tasks | EEG, Feature Selection |
| Published | 2017-10-12 |
| URL | http://arxiv.org/abs/1710.04462v1 |
| http://arxiv.org/pdf/1710.04462v1.pdf | |
| PWC | https://paperswithcode.com/paper/effects-of-images-with-different-levels-of |
| Repo | |
| Framework | |
Consistency of Lipschitz learning with infinite unlabeled data and finite labeled data
| Title | Consistency of Lipschitz learning with infinite unlabeled data and finite labeled data |
| Authors | Jeff Calder |
| Abstract | We study the consistency of Lipschitz learning on graphs in the limit of infinite unlabeled data and finite labeled data. Previous work has conjectured that Lipschitz learning is well-posed in this limit, but is insensitive to the distribution of the unlabeled data, which is undesirable for semi-supervised learning. We first prove that this conjecture is true in the special case of a random geometric graph model with kernel-based weights. Then we go on to show that on a random geometric graph with self-tuning weights, Lipschitz learning is in fact highly sensitive to the distribution of the unlabeled data, and we show how the degree of sensitivity can be adjusted by tuning the weights. In both cases, our results follow from showing that the sequence of learned functions converges to the viscosity solution of an $\infty$-Laplace type equation, and studying the structure of the limiting equation. |
| Tasks | |
| Published | 2017-10-28 |
| URL | https://arxiv.org/abs/1710.10364v3 |
| https://arxiv.org/pdf/1710.10364v3.pdf | |
| PWC | https://paperswithcode.com/paper/consistency-of-lipschitz-learning-with |
| Repo | |
| Framework | |
Active Sampling of Pairs and Points for Large-scale Linear Bipartite Ranking
| Title | Active Sampling of Pairs and Points for Large-scale Linear Bipartite Ranking |
| Authors | Wei-Yuan Shen, Hsuan-Tien Lin |
| Abstract | Bipartite ranking is a fundamental ranking problem that learns to order relevant instances ahead of irrelevant ones. The pair-wise approach for bi-partite ranking construct a quadratic number of pairs to solve the problem, which is infeasible for large-scale data sets. The point-wise approach, albeit more efficient, often results in inferior performance. That is, it is difficult to conduct bipartite ranking accurately and efficiently at the same time. In this paper, we develop a novel active sampling scheme within the pair-wise approach to conduct bipartite ranking efficiently. The scheme is inspired from active learning and can reach a competitive ranking performance while focusing only on a small subset of the many pairs during training. Moreover, we propose a general Combined Ranking and Classification (CRC) framework to accurately conduct bipartite ranking. The framework unifies point-wise and pair-wise approaches and is simply based on the idea of treating each instance point as a pseudo-pair. Experiments on 14 real-word large-scale data sets demonstrate that the proposed algorithm of Active Sampling within CRC, when coupled with a linear Support Vector Machine, usually outperforms state-of-the-art point-wise and pair-wise ranking approaches in terms of both accuracy and efficiency. |
| Tasks | Active Learning |
| Published | 2017-08-24 |
| URL | http://arxiv.org/abs/1708.07336v1 |
| http://arxiv.org/pdf/1708.07336v1.pdf | |
| PWC | https://paperswithcode.com/paper/active-sampling-of-pairs-and-points-for-large |
| Repo | |
| Framework | |
Research on Bi-mode Biometrics Based on Deep Learning
| Title | Research on Bi-mode Biometrics Based on Deep Learning |
| Authors | Hao Jiang |
| Abstract | In view of the fact that biological characteristics have excellent independent distinguishing characteristics,biometric identification technology involves almost all the relevant areas of human distinction. Fingerprints, iris, face, voice-print and other biological features have been widely used in the public security departments to detect detection, mobile equipment unlock, target tracking and other fields. With the use of electronic devices more and more widely and the frequency is getting higher and higher. Only the Biometrics identification technology with excellent recognition rate can guarantee the long-term development of these fields. |
| Tasks | |
| Published | 2017-05-16 |
| URL | http://arxiv.org/abs/1705.05619v1 |
| http://arxiv.org/pdf/1705.05619v1.pdf | |
| PWC | https://paperswithcode.com/paper/research-on-bi-mode-biometrics-based-on-deep |
| Repo | |
| Framework | |
Coherent Online Video Style Transfer
| Title | Coherent Online Video Style Transfer |
| Authors | Dongdong Chen, Jing Liao, Lu Yuan, Nenghai Yu, Gang Hua |
| Abstract | Training a feed-forward network for fast neural style transfer of images is proven to be successful. However, the naive extension to process video frame by frame is prone to producing flickering results. We propose the first end-to-end network for online video style transfer, which generates temporally coherent stylized video sequences in near real-time. Two key ideas include an efficient network by incorporating short-term coherence, and propagating short-term coherence to long-term, which ensures the consistency over larger period of time. Our network can incorporate different image stylization networks. We show that the proposed method clearly outperforms the per-frame baseline both qualitatively and quantitatively. Moreover, it can achieve visually comparable coherence to optimization-based video style transfer, but is three orders of magnitudes faster in runtime. |
| Tasks | Image Stylization, Style Transfer, Video Style Transfer |
| Published | 2017-03-27 |
| URL | http://arxiv.org/abs/1703.09211v2 |
| http://arxiv.org/pdf/1703.09211v2.pdf | |
| PWC | https://paperswithcode.com/paper/coherent-online-video-style-transfer |
| Repo | |
| Framework | |
Preserving Intermediate Objectives: One Simple Trick to Improve Learning for Hierarchical Models
| Title | Preserving Intermediate Objectives: One Simple Trick to Improve Learning for Hierarchical Models |
| Authors | Abhilasha Ravichander, Shruti Rijhwani, Rajat Kulshreshtha, Chirag Nagpal, Tadas Baltrušaitis, Louis-Philippe Morency |
| Abstract | Hierarchical models are utilized in a wide variety of problems which are characterized by task hierarchies, where predictions on smaller subtasks are useful for trying to predict a final task. Typically, neural networks are first trained for the subtasks, and the predictions of these networks are subsequently used as additional features when training a model and doing inference for a final task. In this work, we focus on improving learning for such hierarchical models and demonstrate our method on the task of speaker trait prediction. Speaker trait prediction aims to computationally identify which personality traits a speaker might be perceived to have, and has been of great interest to both the Artificial Intelligence and Social Science communities. Persuasiveness prediction in particular has been of interest, as persuasive speakers have a large amount of influence on our thoughts, opinions and beliefs. In this work, we examine how leveraging the relationship between related speaker traits in a hierarchical structure can help improve our ability to predict how persuasive a speaker is. We present a novel algorithm that allows us to backpropagate through this hierarchy. This hierarchical model achieves a 25% relative error reduction in classification accuracy over current state-of-the art methods on the publicly available POM dataset. |
| Tasks | |
| Published | 2017-06-23 |
| URL | http://arxiv.org/abs/1706.07867v1 |
| http://arxiv.org/pdf/1706.07867v1.pdf | |
| PWC | https://paperswithcode.com/paper/preserving-intermediate-objectives-one-simple |
| Repo | |
| Framework | |
An SDP-Based Algorithm for Linear-Sized Spectral Sparsification
| Title | An SDP-Based Algorithm for Linear-Sized Spectral Sparsification |
| Authors | Yin Tat Lee, He Sun |
| Abstract | For any undirected and weighted graph $G=(V,E,w)$ with $n$ vertices and $m$ edges, we call a sparse subgraph $H$ of $G$, with proper reweighting of the edges, a $(1+\varepsilon)$-spectral sparsifier if [ (1-\varepsilon)x^{\intercal}L_Gx\leq x^{\intercal} L_{H} x\leq (1+\varepsilon) x^{\intercal} L_Gx ] holds for any $x\in\mathbb{R}^n$, where $L_G$ and $L_{H}$ are the respective Laplacian matrices of $G$ and $H$. Noticing that $\Omega(m)$ time is needed for any algorithm to construct a spectral sparsifier and a spectral sparsifier of $G$ requires $\Omega(n)$ edges, a natural question is to investigate, for any constant $\varepsilon$, if a $(1+\varepsilon)$-spectral sparsifier of $G$ with $O(n)$ edges can be constructed in $\tilde{O}(m)$ time, where the $\tilde{O}$ notation suppresses polylogarithmic factors. All previous constructions on spectral sparsification require either super-linear number of edges or $m^{1+\Omega(1)}$ time. In this work we answer this question affirmatively by presenting an algorithm that, for any undirected graph $G$ and $\varepsilon>0$, outputs a $(1+\varepsilon)$-spectral sparsifier of $G$ with $O(n/\varepsilon^2)$ edges in $\tilde{O}(m/\varepsilon^{O(1)})$ time. Our algorithm is based on three novel techniques: (1) a new potential function which is much easier to compute yet has similar guarantees as the potential functions used in previous references; (2) an efficient reduction from a two-sided spectral sparsifier to a one-sided spectral sparsifier; (3) constructing a one-sided spectral sparsifier by a semi-definite program. |
| Tasks | |
| Published | 2017-02-27 |
| URL | http://arxiv.org/abs/1702.08415v1 |
| http://arxiv.org/pdf/1702.08415v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-sdp-based-algorithm-for-linear-sized |
| Repo | |
| Framework | |
AutoDIAL: Automatic DomaIn Alignment Layers
| Title | AutoDIAL: Automatic DomaIn Alignment Layers |
| Authors | Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, Samuel Rota Bulò |
| Abstract | Classifiers trained on given databases perform poorly when tested on data acquired in different settings. This is explained in domain adaptation through a shift among distributions of the source and target domains. Attempts to align them have traditionally resulted in works reducing the domain shift by introducing appropriate loss terms, measuring the discrepancies between source and target distributions, in the objective function. Here we take a different route, proposing to align the learned representations by embedding in any given network specific Domain Alignment Layers, designed to match the source and target feature distributions to a reference one. Opposite to previous works which define a priori in which layers adaptation should be performed, our method is able to automatically learn the degree of feature alignment required at different levels of the deep network. Thorough experiments on different public benchmarks, in the unsupervised setting, confirm the power of our approach. |
| Tasks | Domain Adaptation |
| Published | 2017-04-26 |
| URL | http://arxiv.org/abs/1704.08082v3 |
| http://arxiv.org/pdf/1704.08082v3.pdf | |
| PWC | https://paperswithcode.com/paper/autodial-automatic-domain-alignment-layers |
| Repo | |
| Framework | |
