July 26, 2019

3283 words 16 mins read

Paper Group ANR 788

Regularized Residual Quantization: a multi-layer sparse dictionary learning approach. Selection of training populations (and other subset selection problems) with an accelerated genetic algorithm (STPGA: An R-package for selection of training populations with a genetic algorithm). Mixed one-bit compressive sensing with applications to overexposure …

Regularized Residual Quantization: a multi-layer sparse dictionary learning approach


Title	Regularized Residual Quantization: a multi-layer sparse dictionary learning approach
Authors	Sohrab Ferdowsi, Slava Voloshynovskiy, Dimche Kostadinov
Abstract	The Residual Quantization (RQ) framework is revisited where the quantization distortion is being successively reduced in multi-layers. Inspired by the reverse-water-filling paradigm in rate-distortion theory, an efficient regularization on the variances of the codewords is introduced which allows to extend the RQ for very large numbers of layers and also for high dimensional data, without getting over-trained. The proposed Regularized Residual Quantization (RRQ) results in multi-layer dictionaries which are additionally sparse, thanks to the soft-thresholding nature of the regularization when applied to variance-decaying data which can arise from de-correlating transformations applied to correlated data. Furthermore, we also propose a general-purpose pre-processing for natural images which makes them suitable for such quantization. The RRQ framework is first tested on synthetic variance-decaying data to show its efficiency in quantization of high-dimensional data. Next, we use the RRQ in super-resolution of a database of facial images where it is shown that low-resolution facial images from the test set quantized with codebooks trained on high-resolution images from the training set show relevant high-frequency content when reconstructed with those codebooks.
Tasks	Dictionary Learning, Quantization, Super-Resolution
Published	2017-05-01
URL	http://arxiv.org/abs/1705.00522v1
PDF	http://arxiv.org/pdf/1705.00522v1.pdf
PWC	https://paperswithcode.com/paper/regularized-residual-quantization-a-multi
Repo
Framework

Selection of training populations (and other subset selection problems) with an accelerated genetic algorithm (STPGA: An R-package for selection of training populations with a genetic algorithm)


Title	Selection of training populations (and other subset selection problems) with an accelerated genetic algorithm (STPGA: An R-package for selection of training populations with a genetic algorithm)
Authors	Deniz Akdemir
Abstract	Optimal subset selection is an important task that has numerous algorithms designed for it and has many application areas. STPGA contains a special genetic algorithm supplemented with a tabu memory property (that keeps track of previously tried solutions and their fitness for a number of iterations), and with a regression of the fitness of the solutions on their coding that is used to form the ideal estimated solution (look ahead property) to search for solutions of generic optimal subset selection problems. I have initially developed the programs for the specific problem of selecting training populations for genomic prediction or association problems, therefore I give discussion of the theory behind optimal design of experiments to explain the default optimization criteria in STPGA, and illustrate the use of the programs in this endeavor. Nevertheless, I have picked a few other areas of application: supervised and unsupervised variable selection based on kernel alignment, supervised variable selection with design criteria, influential observation identification for regression, solving mixed integer quadratic optimization problems, balancing gains and inbreeding in a breeding population. Some of these illustrations pertain new statistical approaches.
Tasks
Published	2017-02-26
URL	http://arxiv.org/abs/1702.08088v1
PDF	http://arxiv.org/pdf/1702.08088v1.pdf
PWC	https://paperswithcode.com/paper/selection-of-training-populations-and-other
Repo
Framework

Mixed one-bit compressive sensing with applications to overexposure correction for CT reconstruction


Title	Mixed one-bit compressive sensing with applications to overexposure correction for CT reconstruction
Authors	Xiaolin Huang, Yan Xia, Lei Shi, Yixing Huang, Ming Yan, Joachim Hornegger, Andreas Maier
Abstract	When a measurement falls outside the quantization or measurable range, it becomes saturated and cannot be used in classical reconstruction methods. For example, in C-arm angiography systems, which provide projection radiography, fluoroscopy, digital subtraction angiography, and are widely used for medical diagnoses and interventions, the limited dynamic range of C-arm flat detectors leads to overexposure in some projections during an acquisition, such as imaging relatively thin body parts (e.g., the knee). Aiming at overexposure correction for computed tomography (CT) reconstruction, we in this paper propose a mixed one-bit compressive sensing (M1bit-CS) to acquire information from both regular and saturated measurements. This method is inspired by the recent progress on one-bit compressive sensing, which deals with only sign observations. Its successful applications imply that information carried by saturated measurements is useful to improve recovery quality. For the proposed M1bit-CS model, alternating direction methods of multipliers is developed and an iterative saturation detection scheme is established. Then we evaluate M1bit-CS on one-dimensional signal recovery tasks. In some experiments, the performance of the proposed algorithms on mixed measurements is almost the same as recovery on unsaturated ones with the same amount of measurements. Finally, we apply the proposed method to overexposure correction for CT reconstruction on a phantom and a simulated clinical image. The results are promising, as the typical streaking artifacts and capping artifacts introduced by saturated projection data are effectively reduced, yielding significant error reduction compared with existing algorithms based on extrapolation.
Tasks	Compressive Sensing, Computed Tomography (CT), Quantization
Published	2017-01-03
URL	http://arxiv.org/abs/1701.00694v1
PDF	http://arxiv.org/pdf/1701.00694v1.pdf
PWC	https://paperswithcode.com/paper/mixed-one-bit-compressive-sensing-with
Repo
Framework

A Coordinate-wise Optimization Algorithm for Sparse Inverse Covariance Selection


Title	A Coordinate-wise Optimization Algorithm for Sparse Inverse Covariance Selection
Authors	Ganzhao Yuan, Haoxian Tan, Wei-Shi Zheng
Abstract	Sparse inverse covariance selection is a fundamental problem for analyzing dependencies in high dimensional data. However, such a problem is difficult to solve since it is NP-hard. Existing solutions are primarily based on convex approximation and iterative hard thresholding, which only lead to sub-optimal solutions. In this work, we propose a coordinate-wise optimization algorithm to solve this problem which is guaranteed to converge to a coordinate-wise minimum point. The algorithm iteratively and greedily selects one variable or swaps two variables to identify the support set, and then solves a reduced convex optimization problem over the support set to achieve the greatest descent. As a side contribution of this paper, we propose a Newton-like algorithm to solve the reduced convex sub-problem, which is proven to always converge to the optimal solution with global linear convergence rate and local quadratic convergence rate. Finally, we demonstrate the efficacy of our method on synthetic data and real-world data sets. As a result, the proposed method consistently outperforms existing solutions in terms of accuracy.
Tasks
Published	2017-11-19
URL	http://arxiv.org/abs/1711.07038v2
PDF	http://arxiv.org/pdf/1711.07038v2.pdf
PWC	https://paperswithcode.com/paper/a-coordinate-wise-optimization-algorithm-for
Repo
Framework

Exception-Based Knowledge Updates


Title	Exception-Based Knowledge Updates
Authors	Martin Slota, Joao Leite
Abstract	Existing methods for dealing with knowledge updates differ greatly depending on the underlying knowledge representation formalism. When Classical Logic is used, updates are typically performed by manipulating the knowledge base on the model-theoretic level. On the opposite side of the spectrum stand the semantics for updating Answer-Set Programs that need to rely on rule syntax. Yet, a unifying perspective that could embrace both these branches of research is of great importance as it enables a deeper understanding of all involved methods and principles and creates room for their cross-fertilisation, ripening and further development. This paper bridges the seemingly irreconcilable approaches to updates. It introduces a novel monotonic characterisation of rules, dubbed RE-models, and shows it to be a more suitable semantic foundation for rule updates than SE-models. Then it proposes a generic scheme for specifying semantic rule update operators, based on the idea of viewing a program as the set of sets of RE-models of its rules; updates are performed by introducing additional interpretations - exceptions - to the sets of RE-models of rules in the original program. The introduced scheme is used to define rule update operators that are closely related to both classical update principles and traditional approaches to rules updates, and serve as a basis for a solution to the long-standing problem of state condensing, showing how they can be equivalently defined as binary operators on some class of logic programs. Finally, the essence of these ideas is extracted to define an abstract framework for exception-based update operators, viewing a knowledge base as the set of sets of models of its elements, which can capture a wide range of both model- and formula-based classical update operators, and thus serves as the first firm formal ground connecting classical and rule updates.
Tasks
Published	2017-06-02
URL	http://arxiv.org/abs/1706.00585v1
PDF	http://arxiv.org/pdf/1706.00585v1.pdf
PWC	https://paperswithcode.com/paper/exception-based-knowledge-updates
Repo
Framework

Transfer from Multiple Linear Predictive State Representations (PSR)


Title	Transfer from Multiple Linear Predictive State Representations (PSR)
Authors	Sri Ramana Sekharan, Ramkumar Natarajan, Siddharthan Rajasekaran
Abstract	In this paper, we tackle the problem of transferring policy from multiple partially observable source environments to a partially observable target environment modeled as predictive state representation. This is an entirely new approach with no previous work, other than the case of transfer in fully observable domains. We develop algorithms to successfully achieve policy transfer when we have the model of both the source and target tasks and discuss in detail their performance and shortcomings. These algorithms could be a starting point for the field of transfer learning in partial observability.
Tasks	Transfer Learning
Published	2017-02-07
URL	http://arxiv.org/abs/1702.02184v1
PDF	http://arxiv.org/pdf/1702.02184v1.pdf
PWC	https://paperswithcode.com/paper/transfer-from-multiple-linear-predictive
Repo
Framework

Dependency Grammar Induction with Neural Lexicalization and Big Training Data


Title	Dependency Grammar Induction with Neural Lexicalization and Big Training Data
Authors	Wenjuan Han, Yong Jiang, Kewei Tu
Abstract	We study the impact of big models (in terms of the degree of lexicalization) and big data (in terms of the training corpus size) on dependency grammar induction. We experimented with L-DMV, a lexicalized version of Dependency Model with Valence and L-NDMV, our lexicalized extension of the Neural Dependency Model with Valence. We find that L-DMV only benefits from very small degrees of lexicalization and moderate sizes of training corpora. L-NDMV can benefit from big training data and lexicalization of greater degrees, especially when enhanced with good model initialization, and it achieves a result that is competitive with the current state-of-the-art.
Tasks	Dependency Grammar Induction
Published	2017-08-02
URL	http://arxiv.org/abs/1708.00801v1
PDF	http://arxiv.org/pdf/1708.00801v1.pdf
PWC	https://paperswithcode.com/paper/dependency-grammar-induction-with-neural
Repo
Framework

Online Learning via the Differential Privacy Lens


Title	Online Learning via the Differential Privacy Lens
Authors	Jacob Abernethy, Young Hun Jung, Chansoo Lee, Audra McMillan, Ambuj Tewari
Abstract	In this paper, we use differential privacy as a lens to examine online learning in both full and partial information settings. The differential privacy framework is, at heart, less about privacy and more about algorithmic stability, and thus has found application in domains well beyond those where information security is central. Here we develop an algorithmic property called one-step differential stability which facilitates a more refined regret analysis for online learning methods. We show that tools from the differential privacy literature can yield regret bounds for many interesting online learning problems including online convex optimization and online linear optimization. Our stability notion is particularly well-suited for deriving first-order regret bounds for follow-the-perturbed-leader algorithms, something that all previous analyses have struggled to achieve. We also generalize the standard max-divergence to obtain a broader class called Tsallis max-divergences. These define stronger notions of stability that are useful in deriving bounds in partial information settings such as multi-armed bandits and bandits with experts.
Tasks	Multi-Armed Bandits
Published	2017-11-27
URL	https://arxiv.org/abs/1711.10019v4
PDF	https://arxiv.org/pdf/1711.10019v4.pdf
PWC	https://paperswithcode.com/paper/online-learning-via-differential-privacy
Repo
Framework

GUN: Gradual Upsampling Network for Single Image Super-Resolution


Title	GUN: Gradual Upsampling Network for Single Image Super-Resolution
Authors	Yang Zhao, Guoqing Li, Wenjun Xie, Wei Jia, Hai Min, Xiaoping Liu
Abstract	In this paper, an efficient super-resolution (SR) method based on deep convolutional neural network (CNN) is proposed, namely Gradual Upsampling Network (GUN). Recent CNN based SR methods often preliminarily magnify the low resolution (LR) input to high resolution (HR) and then reconstruct the HR input, or directly reconstruct the LR input and then recover the HR result at the last layer. The proposed GUN utilizes a gradual process instead of these two commonly used frameworks. The GUN consists of an input layer, multiple upsampling and convolutional layers, and an output layer. By means of the gradual process, the proposed network can simplify the direct SR problem to multistep easier upsampling tasks with very small magnification factor in each step. Furthermore, a gradual training strategy is presented for the GUN. In the proposed training process, an initial network can be easily trained with edge-like samples, and then the weights are gradually tuned with more complex samples. The GUN can recover fine and vivid results, and is easy to be trained. The experimental results on several image sets demonstrate the effectiveness of the proposed network.
Tasks	Image Super-Resolution, Super-Resolution
Published	2017-03-13
URL	http://arxiv.org/abs/1703.04244v2
PDF	http://arxiv.org/pdf/1703.04244v2.pdf
PWC	https://paperswithcode.com/paper/gun-gradual-upsampling-network-for-single
Repo
Framework

Neural Machine Translation Training in a Multi-Domain Scenario


Title	Neural Machine Translation Training in a Multi-Domain Scenario
Authors	Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Yonatan Belinkov, Stephan Vogel
Abstract	In this paper, we explore alternative ways to train a neural machine translation system in a multi-domain scenario. We investigate data concatenation (with fine tuning), model stacking (multi-level fine tuning), data selection and multi-model ensemble. Our findings show that the best translation quality can be achieved by building an initial system on a concatenation of available out-of-domain data and then fine-tuning it on in-domain data. Model stacking works best when training begins with the furthest out-of-domain data and the model is incrementally fine-tuned with the next furthest domain and so on. Data selection did not give the best results, but can be considered as a decent compromise between training time and translation quality. A weighted ensemble of different individual models performed better than data selection. It is beneficial in a scenario when there is no time for fine-tuning an already trained model.
Tasks	Machine Translation
Published	2017-08-29
URL	http://arxiv.org/abs/1708.08712v3
PDF	http://arxiv.org/pdf/1708.08712v3.pdf
PWC	https://paperswithcode.com/paper/neural-machine-translation-training-in-a
Repo
Framework

Deep Learning the Indus Script


Title	Deep Learning the Indus Script
Authors	Satish Palaniappan, Ronojoy Adhikari
Abstract	Standardized corpora of undeciphered scripts, a necessary starting point for computational epigraphy, requires laborious human effort for their preparation from raw archaeological records. Automating this process through machine learning algorithms can be of significant aid to epigraphical research. Here, we take the first steps in this direction and present a deep learning pipeline that takes as input images of the undeciphered Indus script, as found in archaeological artifacts, and returns as output a string of graphemes, suitable for inclusion in a standard corpus. The image is first decomposed into regions using Selective Search and these regions are classified as containing textual and/or graphical information using a convolutional neural network. Regions classified as potentially containing text are hierarchically merged and trimmed to remove non-textual information. The remaining textual part of the image is segmented using standard image processing techniques to isolate individual graphemes. This set is finally passed to a second convolutional neural network to classify the graphemes, based on a standard corpus. The classifier can identify the presence or absence of the most frequent Indus grapheme, the “jar” sign, with an accuracy of 92%. Our results demonstrate the great potential of deep learning approaches in computational epigraphy and, more generally, in the digital humanities.
Tasks
Published	2017-02-02
URL	http://arxiv.org/abs/1702.00523v1
PDF	http://arxiv.org/pdf/1702.00523v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-the-indus-script
Repo
Framework

Can Computers overcome Humans? Consciousness interaction and its implications


Title	Can Computers overcome Humans? Consciousness interaction and its implications
Authors	Camilo Miguel Signorelli
Abstract	Can computers overcome human capabilities? This is a paradoxical and controversial question, particularly because there are many hidden assumptions. This article focuses on that issue putting on evidence some misconception related with future generations of machines and the understanding of the brain. It will be discussed to what extent computers might reach human capabilities, and how it could be possible only if the computer is a conscious machine. However, it will be shown that if the computer is conscious, an interference process due to consciousness would affect the information processing of the system. Therefore, it might be possible to make conscious machines to overcome human capabilities, which will have limitations as well as humans. In other words, trying to overcome human capabilities with computers implies the paradoxical conclusion that a computer will never overcome human capabilities at all, or if the computer does, it should not be considered as a computer anymore.
Tasks
Published	2017-06-07
URL	http://arxiv.org/abs/1706.02274v2
PDF	http://arxiv.org/pdf/1706.02274v2.pdf
PWC	https://paperswithcode.com/paper/can-computers-overcome-humans-consciousness
Repo
Framework

Distributed Stratified Locality Sensitive Hashing for Critical Event Prediction in the Cloud


Title	Distributed Stratified Locality Sensitive Hashing for Critical Event Prediction in the Cloud
Authors	Alessandro De Palma, Erik Hemberg, Una-May O’Reilly
Abstract	The availability of massive healthcare data repositories calls for efficient tools for data-driven medicine. We introduce a distributed system for Stratified Locality Sensitive Hashing to perform fast similarity-based prediction on large medical waveform datasets. Our implementation, for an ICU use case, prioritizes latency over throughput and is targeted at a cloud environment. We demonstrate our system on Acute Hypotensive Episode prediction from Arterial Blood Pressure waveforms. On a dataset of $1.37$ million points, we show scaling up to $40$ processors and a $21\times$ speedup in number of comparisons to parallel exhaustive search at the price of a $10%$ Matthews correlation coefficient (MCC) loss. Furthermore, if additional MCC loss can be tolerated, our system achieves speedups up to two orders of magnitude.
Tasks
Published	2017-12-01
URL	http://arxiv.org/abs/1712.00206v1
PDF	http://arxiv.org/pdf/1712.00206v1.pdf
PWC	https://paperswithcode.com/paper/distributed-stratified-locality-sensitive
Repo
Framework

An Introduction to Deep Visual Explanation


Title	An Introduction to Deep Visual Explanation
Authors	Housam Khalifa Bashier Babiker, Randy Goebel
Abstract	The practical impact of deep learning on complex supervised learning problems has been significant, so much so that almost every Artificial Intelligence problem, or at least a portion thereof, has been somehow recast as a deep learning problem. The applications appeal is significant, but this appeal is increasingly challenged by what some call the challenge of explainability, or more generally the more traditional challenge of debuggability: if the outcomes of a deep learning process produce unexpected results (e.g., less than expected performance of a classifier), then there is little available in the way of theories or tools to help investigate the potential causes of such unexpected behavior, especially when this behavior could impact people’s lives. We describe a preliminary framework to help address this issue, which we call “deep visual explanation” (DVE). “Deep,” because it is the development and performance of deep neural network models that we want to understand. “Visual,” because we believe that the most rapid insight into a complex multi-dimensional model is provided by appropriate visualization techniques, and “Explanation,” because in the spectrum from instrumentation by inserting print statements to the abductive inference of explanatory hypotheses, we believe that the key to understanding deep learning relies on the identification and exposure of hypotheses about the performance behavior of a learned deep model. In the exposition of our preliminary framework, we use relatively straightforward image classification examples and a variety of choices on initial configuration of a deep model building scenario. By careful but not complicated instrumentation, we expose classification outcomes of deep models using visualization, and also show initial results for one potential application of interpretability.
Tasks	Image Classification
Published	2017-11-26
URL	http://arxiv.org/abs/1711.09482v2
PDF	http://arxiv.org/pdf/1711.09482v2.pdf
PWC	https://paperswithcode.com/paper/an-introduction-to-deep-visual-explanation
Repo
Framework

Image Compression: Sparse Coding vs. Bottleneck Autoencoders


Title	Image Compression: Sparse Coding vs. Bottleneck Autoencoders
Authors	Yijing Watkins, Mohammad Sayeh, Oleksandr Iaroshenko, Garrett Kenyon
Abstract	Bottleneck autoencoders have been actively researched as a solution to image compression tasks. However, we observed that bottleneck autoencoders produce subjectively low quality reconstructed images. In this work, we explore the ability of sparse coding to improve reconstructed image quality for the same degree of compression. We observe that sparse image compression produces visually superior reconstructed images and yields higher values of pixel-wise measures of reconstruction quality (PSNR and SSIM) compared to bottleneck autoencoders. % In addition, we find that using alternative metrics that correlate better with human perception, such as feature perceptual loss and the classification accuracy, sparse image compression scores up to 18.06% and 2.7% higher, respectively, compared to bottleneck autoencoders. Although computationally much more intensive, we find that sparse coding is otherwise superior to bottleneck autoencoders for the same degree of compression.
Tasks	Image Compression
Published	2017-10-26
URL	http://arxiv.org/abs/1710.09926v2
PDF	http://arxiv.org/pdf/1710.09926v2.pdf
PWC	https://paperswithcode.com/paper/image-compression-sparse-coding-vs-bottleneck
Repo
Framework