May 5, 2019

3019 words 15 mins read

Paper Group ANR 496

Iterative Judgment Aggregation. The geometry of learning. Comparing Convolutional Neural Networks to Traditional Models for Slot Filling. Group Sparse Regularization for Deep Neural Networks. An Adaptive Psychoacoustic Model for Automatic Speech Recognition. Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism. Direc …

Iterative Judgment Aggregation


Title	Iterative Judgment Aggregation
Authors	Marija Slavkovik, Wojciech Jamroga
Abstract	Judgment aggregation problems form a class of collective decision-making problems represented in an abstract way, subsuming some well known problems such as voting. A collective decision can be reached in many ways, but a direct one-step aggregation of individual decisions is arguably most studied. Another way to reach collective decisions is by iterative consensus building – allowing each decision-maker to change their individual decision in response to the choices of the other agents until a consensus is reached. Iterative consensus building has so far only been studied for voting problems. Here we propose an iterative judgment aggregation algorithm, based on movements in an undirected graph, and we study for which instances it terminates with a consensus. We also compare the computational complexity of our iterative procedure with that of related judgment aggregation operators.
Tasks	Decision Making
Published	2016-04-21
URL	http://arxiv.org/abs/1604.06356v3
PDF	http://arxiv.org/pdf/1604.06356v3.pdf
PWC	https://paperswithcode.com/paper/iterative-judgment-aggregation
Repo
Framework

The geometry of learning


Title	The geometry of learning
Authors	Gianluca Calcagni
Abstract	We establish a correspondence between Pavlovian conditioning processes and fractals. The association strength at a training trial corresponds to a point in a disconnected set at a given iteration level. In this way, one can represent a training process as a hopping on a fractal set, instead of the traditional learning curve as a function of the trial. The main advantage of this novel perspective is to provide an elegant classification of associative theories in terms of the geometric features of fractal sets. In particular, the dimension of fractals can measure the efficiency of conditioning models. We illustrate the correspondence with the examples of the Hull, Rescorla-Wagner, and Mackintosh models and show that they are equivalent to a Cantor set. More generally, conditioning programs are described by the geometry of their associated fractal, which gives much more information than just its dimension. We show this in several examples of random fractals and also comment on a possible relation between our formalism and other “fractal” findings in the cognitive literature.
Tasks
Published	2016-05-02
URL	http://arxiv.org/abs/1605.00591v3
PDF	http://arxiv.org/pdf/1605.00591v3.pdf
PWC	https://paperswithcode.com/paper/the-geometry-of-learning
Repo
Framework

Comparing Convolutional Neural Networks to Traditional Models for Slot Filling


Title	Comparing Convolutional Neural Networks to Traditional Models for Slot Filling
Authors	Heike Adel, Benjamin Roth, Hinrich Schütze
Abstract	We address relation classification in the context of slot filling, the task of finding and evaluating fillers like “Steve Jobs” for the slot X in “X founded Apple”. We propose a convolutional neural network which splits the input sentence into three parts according to the relation arguments and compare it to state-of-the-art and traditional approaches of relation classification. Finally, we combine different methods and show that the combination is better than individual approaches. We also analyze the effect of genre differences on performance.
Tasks	Relation Classification, Slot Filling
Published	2016-03-16
URL	http://arxiv.org/abs/1603.05157v2
PDF	http://arxiv.org/pdf/1603.05157v2.pdf
PWC	https://paperswithcode.com/paper/comparing-convolutional-neural-networks-to
Repo
Framework

Group Sparse Regularization for Deep Neural Networks


Title	Group Sparse Regularization for Deep Neural Networks
Authors	Simone Scardapane, Danilo Comminiello, Amir Hussain, Aurelio Uncini
Abstract	In this paper, we consider the joint task of simultaneously optimizing (i) the weights of a deep neural network, (ii) the number of neurons for each hidden layer, and (iii) the subset of active input features (i.e., feature selection). While these problems are generally dealt with separately, we present a simple regularized formulation allowing to solve all three of them in parallel, using standard optimization routines. Specifically, we extend the group Lasso penalty (originated in the linear regression literature) in order to impose group-level sparsity on the network’s connections, where each group is defined as the set of outgoing weights from a unit. Depending on the specific case, the weights can be related to an input variable, to a hidden neuron, or to a bias unit, thus performing simultaneously all the aforementioned tasks in order to obtain a compact network. We perform an extensive experimental evaluation, by comparing with classical weight decay and Lasso penalties. We show that a sparse version of the group Lasso penalty is able to achieve competitive performances, while at the same time resulting in extremely compact networks with a smaller number of input features. We evaluate both on a toy dataset for handwritten digit recognition, and on multiple realistic large-scale classification problems.
Tasks	Feature Selection, Handwritten Digit Recognition
Published	2016-07-02
URL	http://arxiv.org/abs/1607.00485v1
PDF	http://arxiv.org/pdf/1607.00485v1.pdf
PWC	https://paperswithcode.com/paper/group-sparse-regularization-for-deep-neural
Repo
Framework

An Adaptive Psychoacoustic Model for Automatic Speech Recognition


Title	An Adaptive Psychoacoustic Model for Automatic Speech Recognition
Authors	Peng Dai, Xue Teng, Frank Rudzicz, Ing Yann Soon
Abstract	Compared with automatic speech recognition (ASR), the human auditory system is more adept at handling noise-adverse situations, including environmental noise and channel distortion. To mimic this adeptness, auditory models have been widely incorporated in ASR systems to improve their robustness. This paper proposes a novel auditory model which incorporates psychoacoustics and otoacoustic emissions (OAEs) into ASR. In particular, we successfully implement the frequency-dependent property of psychoacoustic models and effectively improve resulting system performance. We also present a novel double-transform spectrum-analysis technique, which can qualitatively predict ASR performance for different noise types. Detailed theoretical analysis is provided to show the effectiveness of the proposed algorithm. Experiments are carried out on the AURORA2 database and show that the word recognition rate using our proposed feature extraction method is significantly increased over the baseline. Given models trained with clean speech, our proposed method achieves up to 85.39% word recognition accuracy on noisy data.
Tasks	Speech Recognition
Published	2016-09-14
URL	http://arxiv.org/abs/1609.04417v1
PDF	http://arxiv.org/pdf/1609.04417v1.pdf
PWC	https://paperswithcode.com/paper/an-adaptive-psychoacoustic-model-for
Repo
Framework

Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism


Title	Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Authors	Orhan Firat, Kyunghyun Cho, Yoshua Bengio
Abstract	We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is shared across all language pairs. We train the proposed multi-way, multilingual model on ten language pairs from WMT’15 simultaneously and observe clear performance improvements over models trained on only one language pair. In particular, we observe that the proposed model significantly improves the translation quality of low-resource language pairs.
Tasks	Machine Translation
Published	2016-01-06
URL	http://arxiv.org/abs/1601.01073v1
PDF	http://arxiv.org/pdf/1601.01073v1.pdf
PWC	https://paperswithcode.com/paper/multi-way-multilingual-neural-machine
Repo
Framework

Directed expected utility networks


Title	Directed expected utility networks
Authors	Manuele Leonelli, Jim Q. Smith
Abstract	A variety of statistical graphical models have been defined to represent the conditional independences underlying a random vector of interest. Similarly, many different graphs embedding various types of preferential independences, as for example conditional utility independence and generalized additive independence, have more recently started to appear. In this paper we define a new graphical model, called a directed expected utility network, whose edges depict both probabilistic and utility conditional independences. These embed a very flexible class of utility models, much larger than those usually conceived in standard influence diagrams. Our graphical representation, and various transformations of the original graph into a tree structure, are then used to guide fast routines for the computation of a decision problem’s expected utilities. We show that our routines generalize those usually utilized in standard influence diagrams’ evaluations under much more restrictive conditions. We then proceed with the construction of a directed expected utility network to support decision makers in the domain of household food security.
Tasks
Published	2016-08-02
URL	http://arxiv.org/abs/1608.00810v2
PDF	http://arxiv.org/pdf/1608.00810v2.pdf
PWC	https://paperswithcode.com/paper/directed-expected-utility-networks
Repo
Framework

Paraconsistency and Word Puzzles


Title	Paraconsistency and Word Puzzles
Authors	Tiantian Gao, Paul Fodor, Michael Kifer
Abstract	Word puzzles and the problem of their representations in logic languages have received considerable attention in the last decade (Ponnuru et al. 2004; Shapiro 2011; Baral and Dzifcak 2012; Schwitter 2013). Of special interest is the problem of generating such representations directly from natural language (NL) or controlled natural language (CNL). An interesting variation of this problem, and to the best of our knowledge, scarcely explored variation in this context, is when the input information is inconsistent. In such situations, the existing encodings of word puzzles produce inconsistent representations and break down. In this paper, we bring the well-known type of paraconsistent logics, called Annotated Predicate Calculus (APC) (Kifer and Lozinskii 1992), to bear on the problem. We introduce a new kind of non-monotonic semantics for APC, called consistency preferred stable models and argue that it makes APC into a suitable platform for dealing with inconsistency in word puzzles and, more generally, in NL sentences. We also devise a number of general principles to help the user choose among the different representations of NL sentences, which might seem equivalent but, in fact, behave differently when inconsistent information is taken into account. These principles can be incorporated into existing CNL translators, such as Attempto Controlled English (ACE) (Fuchs et al. 2008) and PENG Light (White and Schwitter 2009). Finally, we show that APC with the consistency preferred stable model semantics can be equivalently embedded in ASP with preferences over stable models, and we use this embedding to implement this version of APC in Clingo (Gebser et al. 2011) and its Asprin add-on (Brewka et al. 2015).
Tasks
Published	2016-08-03
URL	http://arxiv.org/abs/1608.01338v2
PDF	http://arxiv.org/pdf/1608.01338v2.pdf
PWC	https://paperswithcode.com/paper/paraconsistency-and-word-puzzles
Repo
Framework

Training Spiking Deep Networks for Neuromorphic Hardware


Title	Training Spiking Deep Networks for Neuromorphic Hardware
Authors	Eric Hunsberger, Chris Eliasmith
Abstract	We describe a method to train spiking deep networks that can be run using leaky integrate-and-fire (LIF) neurons, achieving state-of-the-art results for spiking LIF networks on five datasets, including the large ImageNet ILSVRC-2012 benchmark. Our method for transforming deep artificial neural networks into spiking networks is scalable and works with a wide range of neural nonlinearities. We achieve these results by softening the neural response function, such that its derivative remains bounded, and by training the network with noise to provide robustness against the variability introduced by spikes. Our analysis shows that implementations of these networks on neuromorphic hardware will be many times more power-efficient than the equivalent non-spiking networks on traditional hardware.
Tasks
Published	2016-11-16
URL	http://arxiv.org/abs/1611.05141v1
PDF	http://arxiv.org/pdf/1611.05141v1.pdf
PWC	https://paperswithcode.com/paper/training-spiking-deep-networks-for
Repo
Framework

A correlation coefficient of belief functions


Title	A correlation coefficient of belief functions
Authors	Wen Jiang
Abstract	How to manage conflict is still an open issue in Dempster-Shafer evidence theory. The correlation coefficient can be used to measure the similarity of evidence in Dempster-Shafer evidence theory. However, existing correlation coefficients of belief functions have some shortcomings. In this paper, a new correlation coefficient is proposed with many desirable properties. One of its applications is to measure the conflict degree among belief functions. Some numerical examples and comparisons demonstrate the effectiveness of the correlation coefficient.
Tasks
Published	2016-12-16
URL	http://arxiv.org/abs/1612.05497v2
PDF	http://arxiv.org/pdf/1612.05497v2.pdf
PWC	https://paperswithcode.com/paper/a-correlation-coefficient-of-belief-functions
Repo
Framework

On Random Weights for Texture Generation in One Layer Neural Networks


Title	On Random Weights for Texture Generation in One Layer Neural Networks
Authors	Mihir Mongia, Kundan Kumar, Akram Erraqabi, Yoshua Bengio
Abstract	Recent work in the literature has shown experimentally that one can use the lower layers of a trained convolutional neural network (CNN) to model natural textures. More interestingly, it has also been experimentally shown that only one layer with random filters can also model textures although with less variability. In this paper we ask the question as to why one layer CNNs with random filters are so effective in generating textures? We theoretically show that one layer convolutional architectures (without a non-linearity) paired with the an energy function used in previous literature, can in fact preserve and modulate frequency coefficients in a manner so that random weights and pretrained weights will generate the same type of images. Based on the results of this analysis we question whether similar properties hold in the case where one uses one convolution layer with a non-linearity. We show that in the case of ReLu non-linearity there are situations where only one input will give the minimum possible energy whereas in the case of no nonlinearity, there are always infinite solutions that will give the minimum possible energy. Thus we can show that in certain situations adding a ReLu non-linearity generates less variable images.
Tasks	Texture Synthesis
Published	2016-12-19
URL	http://arxiv.org/abs/1612.06070v1
PDF	http://arxiv.org/pdf/1612.06070v1.pdf
PWC	https://paperswithcode.com/paper/on-random-weights-for-texture-generation-in
Repo
Framework

Optimization Beyond Prediction: Prescriptive Price Optimization


Title	Optimization Beyond Prediction: Prescriptive Price Optimization
Authors	Shinji Ito, Ryohei Fujimaki
Abstract	This paper addresses a novel data science problem, prescriptive price optimization, which derives the optimal price strategy to maximize future profit/revenue on the basis of massive predictive formulas produced by machine learning. The prescriptive price optimization first builds sales forecast formulas of multiple products, on the basis of historical data, which reveal complex relationships between sales and prices, such as price elasticity of demand and cannibalization. Then, it constructs a mathematical optimization problem on the basis of those predictive formulas. We present that the optimization problem can be formulated as an instance of binary quadratic programming (BQP). Although BQP problems are NP-hard in general and computationally intractable, we propose a fast approximation algorithm using a semi-definite programming (SDP) relaxation, which is closely related to the Goemans-Williamson’s Max-Cut approximation. Our experiments on simulation and real retail datasets show that our prescriptive price optimization simultaneously derives the optimal prices of tens/hundreds products with practical computational time, that potentially improve 8.2% of gross profit of those products.
Tasks
Published	2016-05-18
URL	http://arxiv.org/abs/1605.05422v2
PDF	http://arxiv.org/pdf/1605.05422v2.pdf
PWC	https://paperswithcode.com/paper/optimization-beyond-prediction-prescriptive
Repo
Framework

Context-Aware Proactive Content Caching with Service Differentiation in Wireless Networks


Title	Context-Aware Proactive Content Caching with Service Differentiation in Wireless Networks
Authors	Sabrina Müller, Onur Atan, Mihaela van der Schaar, Anja Klein
Abstract	Content caching in small base stations or wireless infostations is considered to be a suitable approach to improve the efficiency in wireless content delivery. Placing the optimal content into local caches is crucial due to storage limitations, but it requires knowledge about the content popularity distribution, which is often not available in advance. Moreover, local content popularity is subject to fluctuations since mobile users with different interests connect to the caching entity over time. Which content a user prefers may depend on the user’s context. In this paper, we propose a novel algorithm for context-aware proactive caching. The algorithm learns context-specific content popularity online by regularly observing context information of connected users, updating the cache content and observing cache hits subsequently. We derive a sublinear regret bound, which characterizes the learning speed and proves that our algorithm converges to the optimal cache content placement strategy in terms of maximizing the number of cache hits. Furthermore, our algorithm supports service differentiation by allowing operators of caching entities to prioritize customer groups. Our numerical results confirm that our algorithm outperforms state-of-the-art algorithms in a real world data set, with an increase in the number of cache hits of at least 14%.
Tasks
Published	2016-06-14
URL	http://arxiv.org/abs/1606.04236v2
PDF	http://arxiv.org/pdf/1606.04236v2.pdf
PWC	https://paperswithcode.com/paper/context-aware-proactive-content-caching-with
Repo
Framework

Assessing Threat of Adversarial Examples on Deep Neural Networks


Title	Assessing Threat of Adversarial Examples on Deep Neural Networks
Authors	Abigail Graese, Andras Rozsa, Terrance E. Boult
Abstract	Deep neural networks are facing a potential security threat from adversarial examples, inputs that look normal but cause an incorrect classification by the deep neural network. For example, the proposed threat could result in hand-written digits on a scanned check being incorrectly classified but looking normal when humans see them. This research assesses the extent to which adversarial examples pose a security threat, when one considers the normal image acquisition process. This process is mimicked by simulating the transformations that normally occur in acquiring the image in a real world application, such as using a scanner to acquire digits for a check amount or using a camera in an autonomous car. These small transformations negate the effect of the carefully crafted perturbations of adversarial examples, resulting in a correct classification by the deep neural network. Thus just acquiring the image decreases the potential impact of the proposed security threat. We also show that the already widely used process of averaging over multiple crops neutralizes most adversarial examples. Normal preprocessing, such as text binarization, almost completely neutralizes adversarial examples. This is the first paper to show that for text driven classification, adversarial examples are an academic curiosity, not a security threat.
Tasks
Published	2016-10-13
URL	http://arxiv.org/abs/1610.04256v1
PDF	http://arxiv.org/pdf/1610.04256v1.pdf
PWC	https://paperswithcode.com/paper/assessing-threat-of-adversarial-examples-on
Repo
Framework

A constrained clustering based approach for matching a collection of feature sets


Title	A constrained clustering based approach for matching a collection of feature sets
Authors	Junchi Yan, Zhe Ren, Hongyuan Zha, Stephen Chu
Abstract	In this paper, we consider the problem of finding the feature correspondences among a collection of feature sets, by using their point-wise unary features. This is a fundamental problem in computer vision and pattern recognition, which also closely relates to other areas such as operational research. Different from two-set matching which can be transformed to a quadratic assignment programming task that is known NP-hard, inclusion of merely unary attributes leads to a linear assignment problem for matching two feature sets. This problem has been well studied and there are effective polynomial global optimum solvers such as the Hungarian method. However, it becomes ill-posed when the unary attributes are (heavily) corrupted. The global optimal correspondence concerning the best score defined by the attribute affinity/cost between the two sets can be distinct to the ground truth correspondence since the score function is biased by noises. To combat this issue, we devise a method for matching a collection of feature sets by synergetically exploring the information across the sets. In general, our method can be perceived from a (constrained) clustering perspective: in each iteration, it assigns the features of one set to the clusters formed by the rest of feature sets, and updates the cluster centers in turn. Results on both synthetic data and real images suggest the efficacy of our method against state-of-the-arts.
Tasks
Published	2016-06-12
URL	http://arxiv.org/abs/1606.03731v1
PDF	http://arxiv.org/pdf/1606.03731v1.pdf
PWC	https://paperswithcode.com/paper/a-constrained-clustering-based-approach-for
Repo
Framework