January 26, 2020

3302 words 16 mins read

Paper Group ANR 1391

Inverse Projection Representation and Category Contribution Rate for Robust Tumor Recognition. Scalable Model Compression by Entropy Penalized Reparameterization. The Use of Mutual Coherence to Prove $\ell^1/\ell^0$-Equivalence in Classification Problems. Structure-adaptive manifold estimation. DGSAN: Discrete Generative Self-Adversarial Network. L …

Inverse Projection Representation and Category Contribution Rate for Robust Tumor Recognition


Title	Inverse Projection Representation and Category Contribution Rate for Robust Tumor Recognition
Authors	Xiao-Hui Yang, Li Tian, Yun-Mei Chen, Li-Jun Yang, Shuang Xu, Wen-Ming Wu
Abstract	Sparse representation based classification (SRC) methods have achieved remarkable results. SRC, however, still suffer from requiring enough training samples, insufficient use of test samples and instability of representation. In this paper, a stable inverse projection representation based classification (IPRC) is presented to tackle these problems by effectively using test samples. An IPR is firstly proposed and its feasibility and stability are analyzed. A classification criterion named category contribution rate is constructed to match the IPR and complete classification. Moreover, a statistical measure is introduced to quantify the stability of representation-based classification methods. Based on the IPRC technique, a robust tumor recognition framework is presented by interpreting microarray gene expression data, where a two-stage hybrid gene selection method is introduced to select informative genes. Finally, the functional analysis of candidate’s pathogenicity-related genes is given. Extensive experiments on six public tumor microarray gene expression datasets demonstrate the proposed technique is competitive with state-of-the-art methods.
Tasks	Sparse Representation-based Classification
Published	2019-02-09
URL	https://arxiv.org/abs/1902.03510v2
PDF	https://arxiv.org/pdf/1902.03510v2.pdf
PWC	https://paperswithcode.com/paper/inverse-projection-representation-and
Repo
Framework

Scalable Model Compression by Entropy Penalized Reparameterization


Title	Scalable Model Compression by Entropy Penalized Reparameterization
Authors	Deniz Oktay, Johannes Ballé, Saurabh Singh, Abhinav Shrivastava
Abstract	We describe a simple and general neural network weight compression approach, in which the network parameters (weights and biases) are represented in a “latent” space, amounting to a reparameterization. This space is equipped with a learned probability model, which is used to impose an entropy penalty on the parameter representation during training, and to compress the representation using a simple arithmetic coder after training. Classification accuracy and model compressibility is maximized jointly, with the bitrate–accuracy trade-off specified by a hyperparameter. We evaluate the method on the MNIST, CIFAR-10 and ImageNet classification benchmarks using six distinct model architectures. Our results show that state-of-the-art model compression can be achieved in a scalable and general way without requiring complex procedures such as multi-stage training.
Tasks	Model Compression
Published	2019-06-15
URL	https://arxiv.org/abs/1906.06624v3
PDF	https://arxiv.org/pdf/1906.06624v3.pdf
PWC	https://paperswithcode.com/paper/model-compression-by-entropy-penalized
Repo
Framework

The Use of Mutual Coherence to Prove $\ell^1/\ell^0$-Equivalence in Classification Problems


Title	The Use of Mutual Coherence to Prove $\ell^1/\ell^0$-Equivalence in Classification Problems
Authors	Chelsea Weaver, Naoki Saito
Abstract	We consider the decomposition of a signal over an overcomplete set of vectors. Minimization of the $\ell^1$-norm of the coefficient vector can often retrieve the sparsest solution (so-called “$\ell^1/\ell^0$-equivalence”), a generally NP-hard task, and this fact has powered the field of compressed sensing. Wright et al.‘s sparse representation-based classification (SRC) applies this relationship to machine learning, wherein the signal to be decomposed represents the test sample and columns of the dictionary are training samples. We investigate the relationships between $\ell^1$-minimization, sparsity, and classification accuracy in SRC. After proving that the tractable, deterministic approach to verifying $\ell^1/\ell^0$-equivalence fundamentally conflicts with the high coherence between same-class training samples, we demonstrate that $\ell^1$-minimization can still recover the sparsest solution when the classes are well-separated. Further, using a nonlinear transform so that sparse recovery conditions may be satisfied, we demonstrate that approximate (not strict) equivalence is key to the success of SRC.
Tasks	Sparse Representation-based Classification
Published	2019-01-09
URL	http://arxiv.org/abs/1901.02783v1
PDF	http://arxiv.org/pdf/1901.02783v1.pdf
PWC	https://paperswithcode.com/paper/the-use-of-mutual-coherence-to-prove-ell1ell0
Repo
Framework

Structure-adaptive manifold estimation


Title	Structure-adaptive manifold estimation
Authors	Nikita Puchkin, Vladimir Spokoiny
Abstract	We consider a problem of manifold estimation from noisy observations. Many manifold learning procedures locally approximate a manifold by a weighted average over a small neighborhood. However, in the presence of large noise, the assigned weights become so corrupted that the averaged estimate shows very poor performance. We suggest a novel computationally efficient structure-adaptive procedure, which simultaneously reconstructs a smooth manifold and estimates projections of the point cloud onto this manifold. The proposed approach iteratively refines the weights on each step, using the structural information obtained at previous steps. After several iterations, we obtain nearly “oracle” weights, so that the final estimates are nearly efficient even in the presence of relatively large noise. In our theoretical study we establish tight lower and upper bounds proving asymptotic optimality of the method for manifold estimation under the Hausdorff loss. Our finite sample study confirms a very reasonable performance of the procedure in comparison with the other methods of manifold estimation.
Tasks
Published	2019-06-12
URL	https://arxiv.org/abs/1906.05014v3
PDF	https://arxiv.org/pdf/1906.05014v3.pdf
PWC	https://paperswithcode.com/paper/structure-adaptive-manifold-estimation
Repo
Framework

DGSAN: Discrete Generative Self-Adversarial Network


Title	DGSAN: Discrete Generative Self-Adversarial Network
Authors	Ehsan Montahaei, Danial Alihosseini, Mahdieh Soleymani Baghshah
Abstract	Although GAN-based methods have received many achievements in the last few years, they have not been such successful in generating discrete data. The most important challenge of these methods is the difficulty of passing the gradient from the discriminator to the generator when the generator outputs are discrete. Despite several attempts done to alleviate this problem, none of the existing GAN-based methods has improved the performance of text generation (using measures that evaluate both the quality and the diversity of generated samples) compared to a generative RNN that is simply trained by the maximum likelihood approach. In this paper, we propose a new framework for generating discrete data by an adversarial approach in which we do not need to pass the gradient to the generator. In the proposed method, the update of either the generator or the discriminator can be accomplished straightforwardly. Moreover, we leverage the discreteness of data to explicitly model the data distribution and ensure the normalization of the generated distribution and consequently the convergence properties of the proposed method. Experimental results generally show the superiority of the proposed DGSAN method compared to the other GAN-based approaches for generating discrete sequential data.
Tasks	Text Generation
Published	2019-08-24
URL	https://arxiv.org/abs/1908.09127v1
PDF	https://arxiv.org/pdf/1908.09127v1.pdf
PWC	https://paperswithcode.com/paper/dgsan-discrete-generative-self-adversarial
Repo
Framework

Learning $\textit{Ex Nihilo}$


Title	Learning $\textit{Ex Nihilo}$
Authors	Selmer Bringsjord, Naveen Sundar Govindarajulu
Abstract	This paper introduces, philosophically and to a degree formally, the novel concept of learning $\textit{ex nihilo}$, intended (obviously) to be analogous to the concept of creation $\textit{ex nihilo}$. Learning $\textit{ex nihilo}$ is an agent’s learning “from nothing,” by the suitable employment of schemata for deductive and inductive reasoning. This reasoning must be in machine-verifiable accord with a formal proof/argument theory in a $\textit{cognitive calculus}$ (i.e., roughly, an intensional higher-order multi-operator quantified logic), and this reasoning is applied to percepts received by the agent, in the context of both some prior knowledge, and some prior and current interests. Learning $\textit{ex nihilo}$ is a challenge to contemporary forms of ML, indeed a severe one, but the challenge is offered in the spirt of seeking to stimulate attempts, on the part of non-logicist ML researchers and engineers, to collaborate with those in possession of learning-$\textit{ex nihilo}$ frameworks, and eventually attempts to integrate directly with such frameworks at the implementation level. Such integration will require, among other things, the symbiotic interoperation of state-of-the-art automated reasoners and high-expressivity planners, with statistical/connectionist ML technology.
Tasks
Published	2019-03-04
URL	http://arxiv.org/abs/1903.03515v2
PDF	http://arxiv.org/pdf/1903.03515v2.pdf
PWC	https://paperswithcode.com/paper/learning-textitex-nihilo
Repo
Framework

Automated Inline Analysis of Myocardial Perfusion MRI with Deep Learning


Title	Automated Inline Analysis of Myocardial Perfusion MRI with Deep Learning
Authors	Hui Xue, Rhodri Davies, Louis AE Brown, Kristopher D Knott, Tushar Kotecha, Marianna Fontana, Sven Plein, James C Moon, Peter Kellman
Abstract	Recent development of quantitative myocardial blood flow (MBF) mapping allows direct evaluation of absolute myocardial perfusion, by computing pixel-wise flow maps. Clinical studies suggest quantitative evaluation would be more desirable for objectivity and efficiency. Objective assessment can be further facilitated by segmenting the myocardium and automatically generating reports following the AHA model. This will free user interaction for analysis and lead to a ‘one-click’ solution to improve workflow. This paper proposes a deep neural network based computational workflow for inline myocardial perfusion analysis. Adenosine stress and rest perfusion scans were acquired from three hospitals. Training set included N=1,825 perfusion series from 1,034 patients. Independent test set included 200 scans from 105 patients. Data were consecutively acquired at each site. A convolution neural net (CNN) model was trained to provide segmentation for LV cavity, myocardium and right ventricular by processing incoming 2D+T perfusion Gd series. Model outputs were compared to manual ground-truth for accuracy of segmentation and flow measures derived on global and per-sector basis. The trained models were integrated onto MR scanners for effective inference. Segmentation accuracy and myocardial flow measures were compared between CNN models and manual ground-truth. The mean Dice ratio of CNN derived myocardium was 0.93 +/- 0.04. Both global flow and per-sector values showed no significant difference, compared to manual results. The AHA 16 segment model was automatically generated and reported on the MR scanner. As a result, the fully automated analysis of perfusion flow mapping was achieved. This solution was integrated on the MR scanner, enabling ‘one-click’ analysis and reporting of myocardial blood flow.
Tasks
Published	2019-11-02
URL	https://arxiv.org/abs/1911.00625v1
PDF	https://arxiv.org/pdf/1911.00625v1.pdf
PWC	https://paperswithcode.com/paper/automated-inline-analysis-of-myocardial
Repo
Framework

Recognizing Variables from their Data via Deep Embeddings of Distributions


Title	Recognizing Variables from their Data via Deep Embeddings of Distributions
Authors	Jonas Mueller, Alex Smola
Abstract	A key obstacle in automated analytics and meta-learning is the inability to recognize when different datasets contain measurements of the same variable. Because provided attribute labels are often uninformative in practice, this task may be more robustly addressed by leveraging the data values themselves rather than just relying on their arbitrarily selected variable names. Here, we present a computationally efficient method to identify high-confidence variable matches between a given set of data values and a large repository of previously encountered datasets. Our approach enjoys numerous advantages over distributional similarity based techniques because we leverage learned vector embeddings of datasets which adaptively account for natural forms of data variation encountered in practice. Based on the neural architecture of deep sets, our embeddings can be computed for both numeric and string data. In dataset search and schema matching tasks, our methods outperform standard statistical techniques and we find that the learned embeddings generalize well to new data sources.
Tasks	Meta-Learning
Published	2019-09-11
URL	https://arxiv.org/abs/1909.04844v1
PDF	https://arxiv.org/pdf/1909.04844v1.pdf
PWC	https://paperswithcode.com/paper/recognizing-variables-from-their-data-via
Repo
Framework

PABO: Pseudo Agent-Based Multi-Objective Bayesian Hyperparameter Optimization for Efficient Neural Accelerator Design


Title	PABO: Pseudo Agent-Based Multi-Objective Bayesian Hyperparameter Optimization for Efficient Neural Accelerator Design
Authors	Maryam Parsa, Aayush Ankit, Amirkoushyar Ziabari, Kaushik Roy
Abstract	The ever increasing computational cost of Deep Neural Networks (DNN) and the demand for energy efficient hardware for DNN acceleration has made accuracy and hardware cost co-optimization for DNNs tremendously important, especially for edge devices. Owing to the large parameter space and cost of evaluating each parameter in the search space, manually tuning of DNN hyperparameters is impractical. Automatic joint DNN and hardware hyperparameter optimization is indispensable for such problems. Bayesian optimization-based approaches have shown promising results for hyperparameter optimization of DNNs. However, most of these techniques have been developed without considering the underlying hardware, thereby leading to inefficient designs. Further, the few works that perform joint optimization are not generalizable and mainly focus on CMOS-based architectures. In this work, we present a novel pseudo agent-based multi-objective hyperparameter optimization (PABO) for maximizing the DNN performance while obtaining low hardware cost. Compared to the existing methods, our work poses a theoretically different approach for joint optimization of accuracy and hardware cost and focuses on memristive crossbar-based accelerators. PABO uses a supervisor agent to establish connections between the posterior Gaussian distribution models of network accuracy and hardware cost requirements. The agent reduces the mathematical complexity of the co-optimization problem by removing unnecessary computations and updates of acquisition functions, thereby achieving significant speed-ups for the optimization procedure. PABO outputs a Pareto frontier that underscores the trade-offs between designing high-accuracy and hardware efficiency. Our results demonstrate a superior performance compared to the state-of-the-art methods both in terms of accuracy and computational speed (~100x speed up).
Tasks	Hyperparameter Optimization
Published	2019-06-11
URL	https://arxiv.org/abs/1906.08167v1
PDF	https://arxiv.org/pdf/1906.08167v1.pdf
PWC	https://paperswithcode.com/paper/pabo-pseudo-agent-based-multi-objective
Repo
Framework

Sequential Gaussian Processes for Online Learning of Nonstationary Functions


Title	Sequential Gaussian Processes for Online Learning of Nonstationary Functions
Authors	Michael Minyi Zhang, Bianca Dumitrascu, Sinead A. Williamson, Barbara E. Engelhardt
Abstract	Many machine learning problems can be framed in the context of estimating functions, and often these are time-dependent functions that are estimated in real-time as observations arrive. Gaussian processes (GPs) are an attractive choice for modeling real-valued nonlinear functions due to their flexibility and uncertainty quantification. However, the typical GP regression model suffers from several drawbacks: i) Conventional GP inference scales $O(N^{3})$ with respect to the number of observations; ii) updating a GP model sequentially is not trivial; and iii) covariance kernels often enforce stationarity constraints on the function, while GPs with non-stationary covariance kernels are often intractable to use in practice. To overcome these issues, we propose an online sequential Monte Carlo algorithm to fit mixtures of GPs that capture non-stationary behavior while allowing for fast, distributed inference. By formulating hyperparameter optimization as a multi-armed bandit problem, we accelerate mixing for real time inference. Our approach empirically improves performance over state-of-the-art methods for online GP estimation in the context of prediction for simulated non-stationary data and hospital time series data.
Tasks	Gaussian Processes, Hyperparameter Optimization, Time Series
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10003v2
PDF	https://arxiv.org/pdf/1905.10003v2.pdf
PWC	https://paperswithcode.com/paper/sequential-gaussian-processes-for-online
Repo
Framework

Software Engineering for Fairness: A Case Study with Hyperparameter Optimization


Title	Software Engineering for Fairness: A Case Study with Hyperparameter Optimization
Authors	Joymallya Chakraborty, Tianpei Xia, Fahmid M. Fahid, Tim Menzies
Abstract	We assert that it is the ethical duty of software engineers to strive to reduce software discrimination. This paper discusses how that might be done. This is an important topic since machine learning software is increasingly being used to make decisions that affect people’s lives. Potentially, the application of that software will result in fairer decisions because (unlike humans) machine learning software is not biased. However, recent results show that the software within many data mining packages exhibits “group discrimination”; i.e. their decisions are inappropriately affected by “protected attributes”(e.g., race, gender, age, etc.). There has been much prior work on validating the fairness of machine-learning models (by recognizing when such software discrimination exists). But after detection, comes mitigation. What steps can ethical software engineers take to reduce discrimination in the software they produce? This paper shows that making \textit{fairness} as a goal during hyperparameter optimization can (a) preserve the predictive power of a model learned from a data miner while also (b) generates fairer results. To the best of our knowledge, this is the first application of hyperparameter optimization as a tool for software engineers to generate fairer software.
Tasks	Hyperparameter Optimization
Published	2019-05-14
URL	https://arxiv.org/abs/1905.05786v2
PDF	https://arxiv.org/pdf/1905.05786v2.pdf
PWC	https://paperswithcode.com/paper/software-engineering-for-fairness-a-case
Repo
Framework

Towards More Realistic Human-Robot Conversation: A Seq2Seq-based Body Gesture Interaction System


Title	Towards More Realistic Human-Robot Conversation: A Seq2Seq-based Body Gesture Interaction System
Authors	Minjie Hua, Fuyuan Shi, Yibing Nan, Kai Wang, Hao Chen, Shiguo Lian
Abstract	This paper presents a novel system that enables intelligent robots to exhibit realistic body gestures while communicating with humans. The proposed system consists of a listening model and a speaking model used in corresponding conversational phases. Both models are adapted from the sequence-to-sequence (seq2seq) architecture to synthesize body gestures represented by the movements of twelve upper-body keypoints. All the extracted 2D keypoints are firstly 3D-transformed, then rotated and normalized to discard irrelevant information. Substantial videos of human conversations from Youtube are collected and preprocessed to train the listening and speaking models separately, after which the two models are evaluated using metrics of mean squared error (MSE) and cosine similarity on the test dataset. The tuned system is implemented to drive a virtual avatar as well as Pepper, a physical humanoid robot, to demonstrate the improvement on conversational interaction abilities of our method in practice.
Tasks
Published	2019-05-05
URL	https://arxiv.org/abs/1905.01641v3
PDF	https://arxiv.org/pdf/1905.01641v3.pdf
PWC	https://paperswithcode.com/paper/towards-more-realistic-human-robot
Repo
Framework

Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning


Title	Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning
Authors	Guillermo A. Castillo, Bowen Weng, Wei Zhang, Ayonga Hereid
Abstract	This paper presents a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from the hybrid nature of the walking dynamics and the well-established hybrid zero dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, short training time, and less dependence on prior knowledge. We demonstrate the effectiveness of the proposed method on Cassie, a challenging 3D bipedal robot. The proposed solution produces stable limit walking cycles that can track various walking speed in different directions. Surprisingly, without specifically trained with disturbances to achieve robustness, it also performs robustly against various adversarial forces applied to the torso towards both the forward and the backward directions.
Tasks
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01748v1
PDF	https://arxiv.org/pdf/1910.01748v1.pdf
PWC	https://paperswithcode.com/paper/hybrid-zero-dynamics-inspired-feedback
Repo
Framework

Advances in Online Audio-Visual Meeting Transcription


Title	Advances in Online Audio-Visual Meeting Transcription
Authors	Takuya Yoshioka, Igor Abramovski, Cem Aksoylar, Zhuo Chen, Moshe David, Dimitrios Dimitriadis, Yifan Gong, Ilya Gurvich, Xuedong Huang, Yan Huang, Aviv Hurvitz, Li Jiang, Sharon Koubi, Eyal Krupka, Ido Leichter, Changliang Liu, Partha Parthasarathy, Alon Vinnikov, Lingfeng Wu, Xiong Xiao, Wayne Xiong, Huaming Wang, Zhenghao Wang, Jun Zhang, Yong Zhao, Tianyan Zhou
Abstract	This paper describes a system that generates speaker-annotated transcripts of meetings by using a microphone array and a 360-degree camera. The hallmark of the system is its ability to handle overlapped speech, which has been an unsolved problem in realistic settings for over a decade. We show that this problem can be addressed by using a continuous speech separation approach. In addition, we describe an online audio-visual speaker diarization method that leverages face tracking and identification, sound source localization, speaker identification, and, if available, prior speaker information for robustness to various real world challenges. All components are integrated in a meeting transcription framework called SRD, which stands for “separate, recognize, and diarize”. Experimental results using recordings of natural meetings involving up to 11 attendees are reported. The continuous speech separation improves a word error rate (WER) by 16.1% compared with a highly tuned beamformer. When a complete list of meeting attendees is available, the discrepancy between WER and speaker-attributed WER is only 1.0%, indicating accurate word-to-speaker association. This increases marginally to 1.6% when 50% of the attendees are unknown to the system.
Tasks	Speaker Diarization, Speaker Identification, Speech Separation
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04979v1
PDF	https://arxiv.org/pdf/1912.04979v1.pdf
PWC	https://paperswithcode.com/paper/advances-in-online-audio-visual-meeting
Repo
Framework

The Speed Submission to DIHARD II: Contributions & Lessons Learned


Title	The Speed Submission to DIHARD II: Contributions & Lessons Learned
Authors	Md Sahidullah, Jose Patino, Samuele Cornell, Ruiqing Yin, Sunit Sivasankaran, Hervé Bredin, Pavel Korshunov, Alessio Brutti, Romain Serizel, Emmanuel Vincent, Nicholas Evans, Sébastien Marcel, Stefano Squartini, Claude Barras
Abstract	This paper describes the speaker diarization systems developed for the Second DIHARD Speech Diarization Challenge (DIHARD II) by the Speed team. Besides describing the system, which considerably outperformed the challenge baselines, we also focus on the lessons learned from numerous approaches that we tried for single and multi-channel systems. We present several components of our diarization system, including categorization of domains, speech enhancement, speech activity detection, speaker embeddings, clustering methods, resegmentation, and system fusion. We analyze and discuss the effect of each such component on the overall diarization performance within the realistic settings of the challenge.
Tasks	Action Detection, Activity Detection, Speaker Diarization, Speech Enhancement
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02388v1
PDF	https://arxiv.org/pdf/1911.02388v1.pdf
PWC	https://paperswithcode.com/paper/the-speed-submission-to-dihard-ii
Repo
Framework