January 31, 2020

3185 words 15 mins read

Paper Group ANR 166

Learning Compact Neural Networks Using Ordinary Differential Equations as Activation Functions. EmoBed: Strengthening Monomodal Emotion Recognition via Training with Crossmodal Emotion Embeddings. Learning high-dimensional probability distributions using tree tensor networks. Making AI meaningful again. Learning Part Generation and Assembly for Str …

Learning Compact Neural Networks Using Ordinary Differential Equations as Activation Functions


Title	Learning Compact Neural Networks Using Ordinary Differential Equations as Activation Functions
Authors	MohamadAli Torkamani, Phillip Wallis, Shiv Shankar, Amirmohammad Rooshenas
Abstract	Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation function from a family of solutions to an ordinary differential equation. Specifically, each neuron may change its functional form during training based on the behavior of the other parts of the network. We show that using neurons with DEU activation functions results in a more compact network capable of achieving comparable, if not superior, performance when is compared to much larger networks.
Tasks
Published	2019-05-19
URL	https://arxiv.org/abs/1905.07685v1
PDF	https://arxiv.org/pdf/1905.07685v1.pdf
PWC	https://paperswithcode.com/paper/learning-compact-neural-networks-using
Repo
Framework

EmoBed: Strengthening Monomodal Emotion Recognition via Training with Crossmodal Emotion Embeddings


Title	EmoBed: Strengthening Monomodal Emotion Recognition via Training with Crossmodal Emotion Embeddings
Authors	Jing Han, Zixing Zhang, Zhao Ren, Björn Schuller
Abstract	Despite remarkable advances in emotion recognition, they are severely restrained from either the essentially limited property of the employed single modality, or the synchronous presence of all involved multiple modalities. Motivated by this, we propose a novel crossmodal emotion embedding framework called EmoBed, which aims to leverage the knowledge from other auxiliary modalities to improve the performance of an emotion recognition system at hand. The framework generally includes two main learning components, i. e., joint multimodal training and crossmodal training. Both of them tend to explore the underlying semantic emotion information but with a shared recognition network or with a shared emotion embedding space, respectively. In doing this, the enhanced system trained with this approach can efficiently make use of the complementary information from other modalities. Nevertheless, the presence of these auxiliary modalities is not demanded during inference. To empirically investigate the effectiveness and robustness of the proposed framework, we perform extensive experiments on the two benchmark databases RECOLA and OMG-Emotion for the tasks of dimensional emotion regression and categorical emotion classification, respectively. The obtained results show that the proposed framework significantly outperforms related baselines in monomodal inference, and are also competitive or superior to the recently reported systems, which emphasises the importance of the proposed crossmodal learning for emotion recognition.
Tasks	Emotion Classification, Emotion Recognition
Published	2019-07-23
URL	https://arxiv.org/abs/1907.10428v1
PDF	https://arxiv.org/pdf/1907.10428v1.pdf
PWC	https://paperswithcode.com/paper/emobed-strengthening-monomodal-emotion
Repo
Framework

Learning high-dimensional probability distributions using tree tensor networks


Title	Learning high-dimensional probability distributions using tree tensor networks
Authors	Erwan Grelier, Anthony Nouy, Régis Lebrun
Abstract	We consider the problem of the estimation of a high-dimensional probability distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density with respect to a product measure, possibly discrete for handling the case of discrete random variables. After discussing the representation of classical model classes in tree-based tensor formats, we present learning algorithms based on empirical risk minimization using a $L^2$ contrast. These algorithms exploit the multilinear parametrization of the formats to recast the nonlinear minimization problem into a sequence of empirical risk minimization problems with linear models. A suitable parametrization of the tensor in tree-based tensor format allows to obtain a linear model with orthogonal bases, so that each problem admits an explicit expression of the solution and cross-validation risk estimates. These estimations of the risk enable the model selection, for instance when exploiting sparsity in the coefficients of the representation. A strategy for the adaptation of the tensor format (dimension tree and tree-based ranks) is provided, which allows to discover and exploit some specific structures of high-dimensional probability distributions such as independence or conditional independence. We illustrate the performances of the proposed algorithms for the approximation of classical probabilistic models (such as Gaussian distribution, graphical models, Markov chain).
Tasks	Model Selection, Tensor Networks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.07913v2
PDF	https://arxiv.org/pdf/1912.07913v2.pdf
PWC	https://paperswithcode.com/paper/learning-high-dimensional-probability
Repo
Framework

Making AI meaningful again


Title	Making AI meaningful again
Authors	Jobst Landgrebe, Barry Smith
Abstract	Artificial intelligence (AI) research enjoyed an initial period of enthusiasm in the 1970s and 80s. But this enthusiasm was tempered by a long interlude of frustration when genuinely useful AI applications failed to be forthcoming. Today, we are experiencing once again a period of enthusiasm, fired above all by the successes of the technology of deep neural networks or deep machine learning. In this paper we draw attention to what we take to be serious problems underlying current views of artificial intelligence encouraged by these successes, especially in the domain of language processing. We then show an alternative approach to language-centric AI, in which we identify a role for philosophy.
Tasks
Published	2019-01-09
URL	http://arxiv.org/abs/1901.02918v3
PDF	http://arxiv.org/pdf/1901.02918v3.pdf
PWC	https://paperswithcode.com/paper/making-ai-meaningful-again
Repo
Framework

Learning Part Generation and Assembly for Structure-aware Shape Synthesis


Title	Learning Part Generation and Assembly for Structure-aware Shape Synthesis
Authors	Jun Li, Chengjie Niu, Kai Xu
Abstract	Learning powerful deep generative models for 3D shape synthesis is largely hindered by the difficulty in ensuring plausibility encompassing correct topology and reasonable geometry. Indeed, learning the distribution of plausible 3D shapes seems a daunting task for the holistic approaches, given the significant topological variations of 3D objects even within the same category. Enlightened by the fact that 3D shape structure is characterized as part composition and placement, we propose to model 3D shape variations with a part-aware deep generative network, coined as PAGENet. The network is composed of an array of per-part VAE-GANs, generating semantic parts composing a complete shape, followed by a part assembly module that estimates a transformation for each part to correlate and assemble them into a plausible structure. Through delegating the learning of part composition and part placement into separate networks, the difficulty of modeling structural variations of 3D shapes is greatly reduced. We demonstrate through both qualitative and quantitative evaluations that PAGENet generates 3D shapes with plausible, diverse and detailed structure, and show two applications, i.e., semantic shape segmentation and part-based shape editing.
Tasks	3D Shape Analysis
Published	2019-06-16
URL	https://arxiv.org/abs/1906.06693v4
PDF	https://arxiv.org/pdf/1906.06693v4.pdf
PWC	https://paperswithcode.com/paper/learning-part-generation-and-assembly-for
Repo
Framework

Automatic Inference of Minimalist Grammars using an SMT-Solver


Title	Automatic Inference of Minimalist Grammars using an SMT-Solver
Authors	Sagar Indurkhya
Abstract	We introduce (1) a novel parser for Minimalist Grammars (MG), encoded as a system of first-order logic formulae that may be evaluated using an SMT-solver, and (2) a novel procedure for inferring Minimalist Grammars using this parser. The input to this procedure is a sequence of sentences that have been annotated with syntactic relations such as semantic role labels (connecting arguments to predicates) and subject-verb agreement. The output of this procedure is a set of minimalist grammars, each of which is able to parse the sentences in the input sequence such that the parse for a sentence has the same syntactic relations as those specified in the annotation for that sentence. We applied this procedure to a set of sentences annotated with syntactic relations and evaluated the inferred grammars using cost functions inspired by the Minimum Description Length principle and the Subset principle. Inferred grammars that were optimal with respect to certain combinations of these cost functions were found to align with contemporary theories of syntax.
Tasks
Published	2019-05-08
URL	https://arxiv.org/abs/1905.02869v1
PDF	https://arxiv.org/pdf/1905.02869v1.pdf
PWC	https://paperswithcode.com/paper/automatic-inference-of-minimalist-grammars
Repo
Framework

Boosting Throughput and Efficiency of Hardware Spiking Neural Accelerators using Time Compression Supporting Multiple Spike Codes


Title	Boosting Throughput and Efficiency of Hardware Spiking Neural Accelerators using Time Compression Supporting Multiple Spike Codes
Authors	Changqing Xu, Wenrui Zhang, Yu Liu, Peng Li
Abstract	Spiking neural networks (SNNs) are the third generation of neural networks and can explore both rate and temporal coding for energy-efficient event-driven computation. However, the decision accuracy of existing SNN designs is contingent upon processing a large number of spikes over a long period. Nevertheless, the switching power of SNN hardware accelerators is proportional to the number of spikes processed while the length of spike trains limits throughput and static power efficiency. This paper presents the first study on developing temporal compression to significantly boost throughput and reduce energy dissipation of digital hardware SNN accelerators while being applicable to multiple spike codes. The proposed compression architectures consist of low-cost input spike compression units, novel input-and-output-weighted spiking neurons, and reconfigurable time constant scaling to support large and flexible time compression ratios. Our compression architectures can be transparently applied to any given pre-designed SNNs employing either rate or temporal codes while incurring minimal modification of the neural models, learning algorithms, and hardware design. Using spiking speech and image recognition datasets, we demonstrate the feasibility of supporting large time compression ratios of up to 16x, delivering up to 15.93x, 13.88x, and 86.21x improvements in throughput, energy dissipation, the tradeoffs between hardware area, runtime, energy, and classification accuracy, respectively based on different spike codes on a Xilinx Zynq-7000 FPGA. These results are achieved while incurring little extra hardware overhead.
Tasks
Published	2019-09-10
URL	https://arxiv.org/abs/1909.04757v1
PDF	https://arxiv.org/pdf/1909.04757v1.pdf
PWC	https://paperswithcode.com/paper/boosting-throughput-and-efficiency-of
Repo
Framework

Generalized Planning With Procedural Domain Control Knowledge


Title	Generalized Planning With Procedural Domain Control Knowledge
Authors	Javier Segovia-Aguas, Sergio Jiménez, Anders Jonsson
Abstract	Generalized planning is the task of generating a single solution that is valid for a set of planning problems. In this paper we show how to represent and compute generalized plans using procedural Domain Control Knowledge (DCK). We define a {\it divide and conquer} approach that first generates the procedural DCK solving a set of planning problems representative of certain subtasks and then compile it as callable procedures of the overall generalized planning problem. Our procedure calling mechanism allows nested and recursive procedure calls and is implemented in PDDL so that classical planners can compute and exploit procedural DCK. Experiments show that an off-the-shelf classical planner, using procedural DCK as callable procedures, can compute generalized plans in a wide range of domains including non-trivial ones, such as sorting variable-size lists or DFS traversal of binary trees with variable size.
Tasks
Published	2019-10-11
URL	https://arxiv.org/abs/1910.04999v1
PDF	https://arxiv.org/pdf/1910.04999v1.pdf
PWC	https://paperswithcode.com/paper/generalized-planning-with-procedural-domain
Repo
Framework


Title	Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections
Authors	Mennatallah El-Assady, Rebecca Kehlbeck, Christopher Collins, Daniel Keim, Oliver Deussen
Abstract	We present a framework that allows users to incorporate the semantics of their domain knowledge for topic model refinement while remaining model-agnostic. Our approach enables users to (1) understand the semantic space of the model, (2) identify regions of potential conflicts and problems, and (3) readjust the semantic relation of concepts based on their understanding, directly influencing the topic modeling. These tasks are supported by an interactive visual analytics workspace that uses word-embedding projections to define concept regions which can then be refined. The user-refined concepts are independent of a particular document collection and can be transferred to related corpora. All user interactions within the concept space directly affect the semantic relations of the underlying vector space model, which, in turn, change the topic modeling. In addition to direct manipulation, our system guides the users’ decision-making process through recommended interactions that point out potential improvements. This targeted refinement aims at minimizing the feedback required for an efficient human-in-the-loop process. We confirm the improvements achieved through our approach in two user studies that show topic model quality improvements through our visual knowledge externalization and learning process.
Tasks	Decision Making
Published	2019-08-01
URL	https://arxiv.org/abs/1908.00475v1
PDF	https://arxiv.org/pdf/1908.00475v1.pdf
PWC	https://paperswithcode.com/paper/semantic-concept-spaces-guided-topic-model
Repo
Framework

A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression


Title	A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
Authors	Kai Yang, Tao Fan, Tianjian Chen, Yuanming Shi, Qiang Yang
Abstract	Data privacy and security becomes a major concern in building machine learning models from different data providers. Federated learning shows promise by leaving data at providers locally and exchanging encrypted information. This paper studies the vertical federated learning structure for logistic regression where the data sets at two parties have the same sample IDs but own disjoint subsets of features. Existing frameworks adopt the first-order stochastic gradient descent algorithm, which requires large number of communication rounds. To address the communication challenge, we propose a quasi-Newton method based vertical federated learning framework for logistic regression under the additively homomorphic encryption scheme. Our approach can considerably reduce the number of communication rounds with a little additional communication cost per round. Numerical results demonstrate the advantages of our approach over the first-order method.
Tasks
Published	2019-12-01
URL	https://arxiv.org/abs/1912.00513v2
PDF	https://arxiv.org/pdf/1912.00513v2.pdf
PWC	https://paperswithcode.com/paper/a-quasi-newton-method-based-vertical
Repo
Framework

Learning Robust Representations by Projecting Superficial Statistics Out


Title	Learning Robust Representations by Projecting Superficial Statistics Out
Authors	Haohan Wang, Zexue He, Zachary C. Lipton, Eric P. Xing
Abstract	Despite impressive performance as evaluated on i.i.d. holdout data, deep neural networks depend heavily on superficial statistics of the training data and are liable to break under distribution shift. For example, subtle changes to the background or texture of an image can break a seemingly powerful classifier. Building on previous work on domain generalization, we hope to produce a classifier that will generalize to previously unseen domains, even when domain identifiers are not available during training. This setting is challenging because the model may extract many distribution-specific (superficial) signals together with distribution-agnostic (semantic) signals. To overcome this challenge, we incorporate the gray-level co-occurrence matrix (GLCM) to extract patterns that our prior knowledge suggests are superficial: they are sensitive to the texture but unable to capture the gestalt of an image. Then we introduce two techniques for improving our networks’ out-of-sample performance. The first method is built on the reverse gradient method that pushes our model to learn representations from which the GLCM representation is not predictable. The second method is built on the independence introduced by projecting the model’s representation onto the subspace orthogonal to GLCM representation’s. We test our method on the battery of standard domain generalization data sets and, interestingly, achieve comparable or better performance as compared to other domain generalization methods that explicitly require samples from the target distribution for training.
Tasks	Domain Generalization
Published	2019-03-02
URL	http://arxiv.org/abs/1903.06256v1
PDF	http://arxiv.org/pdf/1903.06256v1.pdf
PWC	https://paperswithcode.com/paper/learning-robust-representations-by-projecting-1
Repo
Framework

Shapley regressions: A framework for statistical inference on machine learning models


Title	Shapley regressions: A framework for statistical inference on machine learning models
Authors	Andreas Joseph
Abstract	Machine learning models often excel in the accuracy of their predictions but are opaque due to their non-linear and non-parametric structure. This makes statistical inference challenging and disqualifies them from many applications where model interpretability is crucial. This paper proposes the Shapley regression framework as an approach for statistical inference on non-linear or non-parametric models. Inference is performed based on the Shapley value decomposition of a model, a pay-off concept from cooperative game theory. I show that universal approximators from machine learning are estimation consistent and introduce hypothesis tests for individual variable contributions, model bias and parametric functional forms. The inference properties of state-of-the-art machine learning models - like artificial neural networks, support vector machines and random forests - are investigated using numerical simulations and real-world data. The proposed framework is unique in the sense that it is identical to the conventional case of statistical inference on a linear model if the model is linear in parameters. This makes it a well-motivated extension to more general models and strengthens the case for the use of machine learning to inform decisions.
Tasks
Published	2019-03-11
URL	http://arxiv.org/abs/1903.04209v1
PDF	http://arxiv.org/pdf/1903.04209v1.pdf
PWC	https://paperswithcode.com/paper/shapley-regressions-a-framework-for
Repo
Framework

GANalyze: Toward Visual Definitions of Cognitive Image Properties


Title	GANalyze: Toward Visual Definitions of Cognitive Image Properties
Authors	Authors, :, Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola
Abstract	We introduce a framework that uses Generative Adversarial Networks (GANs) to study cognitive properties like memorability, aesthetics, and emotional valence. These attributes are of interest because we do not have a concrete visual definition of what they entail. What does it look like for a dog to be more or less memorable? GANs allow us to generate a manifold of natural-looking images with fine-grained differences in their visual attributes. By navigating this manifold in directions that increase memorability, we can visualize what it looks like for a particular generated image to become more or less memorable. The resulting `visual definitions" surface image properties (like` object size”) that may underlie memorability. Through behavioral experiments, we verify that our method indeed discovers image manipulations that causally affect human memory performance. We further demonstrate that the same framework can be used to analyze image aesthetics and emotional valence. Visit the GANalyze website at http://ganalyze.csail.mit.edu/.
Tasks
Published	2019-06-24
URL	https://arxiv.org/abs/1906.10112v1
PDF	https://arxiv.org/pdf/1906.10112v1.pdf
PWC	https://paperswithcode.com/paper/ganalyze-toward-visual-definitions-of
Repo
Framework

Fast and Exact Nearest Neighbor Search in Hamming Space on Full-Text Search Engines


Title	Fast and Exact Nearest Neighbor Search in Hamming Space on Full-Text Search Engines
Authors	Cun Mu, Jun Zhao, Guang Yang, Binwei Yang, Zheng Yan
Abstract	A growing interest has been witnessed recently from both academia and industry in building nearest neighbor search (NNS) solutions on top of full-text search engines. Compared with other NNS systems, such solutions are capable of effectively reducing main memory consumption, coherently supporting multi-model search and being immediately ready for production deployment. In this paper, we continue the journey to explore specifically how to empower full-text search engines with fast and exact NNS in Hamming space (i.e., the set of binary codes). By revisiting three techniques (bit operation, subs-code filtering and data preprocessing with permutation) in information retrieval literature, we develop a novel engineering solution for full-text search engines to efficiently accomplish this special but important NNS task. In the experiment, we show that our proposed approach enables full-text search engines to achieve significant speed-ups over its state-of-the-art term match approach for NNS within binary codes.
Tasks	Information Retrieval, Representation Learning
Published	2019-02-20
URL	https://arxiv.org/abs/1902.08498v2
PDF	https://arxiv.org/pdf/1902.08498v2.pdf
PWC	https://paperswithcode.com/paper/empowering-elasticsearch-with-exact-and-fast
Repo
Framework

Grey matter sublayer thickness estimation in themouse cerebellum


Title	Grey matter sublayer thickness estimation in themouse cerebellum
Authors	Da Ma, Manuel J. Cardoso, Maria A. Zuluaga, Marc Modat, Nick. Powell, Frances Wiseman, Victor Tybulewicz, Elizabeth Fisher, Mark. F. Lythgoe, Sebastien Ourselin
Abstract	The cerebellar grey matter morphology is an important feature to study neurodegenerative diseases such as Alzheimer’s disease or Down’s syndrome. Its volume or thickness is commonly used as a surrogate imaging biomarker for such diseases. Most studies about grey matter thickness estimation focused on the cortex, and little attention has been drawn on the morphology of the cerebellum. Using ex vivo high-resolution MRI, it is now possible to visualise the different cell layers in the mouse cerebellum. In this work, we introduce a framework to extract the Purkinje layer within the grey matter, enabling the estimation of the thickness of the cerebellar grey matter, the granular layer and molecular layer from gadolinium-enhanced ex vivo mouse brain MRI. Application to mouse model of Down’s syndrome found reduced cortical and layer thicknesses in the transchromosomic group.
Tasks
Published	2019-01-08
URL	http://arxiv.org/abs/1901.02499v1
PDF	http://arxiv.org/pdf/1901.02499v1.pdf
PWC	https://paperswithcode.com/paper/grey-matter-sublayer-thickness-estimation-in
Repo
Framework