Paper Group ANR 544
Learning Relational Representations with Auto-encoding Logic Programs. Nonlinear Dipole Inversion (NDI) enables Quantitative Susceptibility Mapping (QSM) without parameter tuning. Are Nearby Neighbors Relatives?: Testing Deep Music Embeddings. Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry. LAVAE: Disentangling Location and Appea …
Learning Relational Representations with Auto-encoding Logic Programs
| Title | Learning Relational Representations with Auto-encoding Logic Programs |
| Authors | Sebastijan Dumancic, Tias Guns, Wannes Meert, Hendrik Blockeel |
| Abstract | Deep learning methods capable of handling relational data have proliferated over the last years. In contrast to traditional relational learning methods that leverage first-order logic for representing such data, these deep learning methods aim at re-representing symbolic relational data in Euclidean spaces. They offer better scalability, but can only numerically approximate relational structures and are less flexible in terms of reasoning tasks supported. This paper introduces a novel framework for relational representation learning that combines the best of both worlds. This framework, inspired by the auto-encoding principle, uses first-order logic as a data representation language, and the mapping between the original and latent representation is done by means of logic programs instead of neural networks. We show how learning can be cast as a constraint optimisation problem for which existing solvers can be used. The use of logic as a representation language makes the proposed framework more accurate (as the representation is exact, rather than approximate), more flexible, and more interpretable than deep learning methods. We experimentally show that these latent representations are indeed beneficial in relational learning tasks. |
| Tasks | Relational Reasoning, Representation Learning |
| Published | 2019-03-29 |
| URL | https://arxiv.org/abs/1903.12577v2 |
| https://arxiv.org/pdf/1903.12577v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-relational-representations-with-auto |
| Repo | |
| Framework | |
Nonlinear Dipole Inversion (NDI) enables Quantitative Susceptibility Mapping (QSM) without parameter tuning
| Title | Nonlinear Dipole Inversion (NDI) enables Quantitative Susceptibility Mapping (QSM) without parameter tuning |
| Authors | Daniel Polak, Itthi Chatnuntawech, Jaeyeon Yoon, Siddharth Srinivasan Iyer, Jongho Lee, Peter Bachert, Elfar Adalsteinsson, Kawin Setsompop, Berkin Bilgic |
| Abstract | We propose Nonlinear Dipole Inversion (NDI) for high-quality Quantitative Susceptibility Mapping (QSM) without regularization tuning, while matching the image quality of state-of-the-art reconstruction techniques. In addition to avoiding over-smoothing that these techniques often suffer from, we also obviate the need for parameter selection. NDI is flexible enough to allow for reconstruction from an arbitrary number of head orientations, and outperforms COSMOS even when using as few as 1-direction data. This is made possible by a nonlinear forward-model that uses the magnitude as an effective prior, for which we derived a simple gradient descent update rule. We synergistically combine this physics-model with a Variational Network (VN) to leverage the power of deep learning in the VaNDI algorithm. This technique adopts the simple gradient descent rule from NDI and learns the network parameters during training, hence requires no additional parameter tuning. Further, we evaluate NDI at 7T using highly accelerated Wave-CAIPI acquisitions at 0.5 mm isotropic resolution and demonstrate high-quality QSM from as few as 2-direction data. |
| Tasks | |
| Published | 2019-09-30 |
| URL | https://arxiv.org/abs/1909.13692v1 |
| https://arxiv.org/pdf/1909.13692v1.pdf | |
| PWC | https://paperswithcode.com/paper/nonlinear-dipole-inversion-ndi-enables |
| Repo | |
| Framework | |
Are Nearby Neighbors Relatives?: Testing Deep Music Embeddings
| Title | Are Nearby Neighbors Relatives?: Testing Deep Music Embeddings |
| Authors | Jaehun Kim, Julián Urbano, Cynthia C. S. Liem, Alan Hanjalic |
| Abstract | Deep neural networks have frequently been used to directly learn representations useful for a given task from raw input data. In terms of overall performance metrics, machine learning solutions employing deep representations frequently have been reported to greatly outperform those using hand-crafted feature representations. At the same time, they may pick up on aspects that are predominant in the data, yet not actually meaningful or interpretable. In this paper, we therefore propose a systematic way to test the trustworthiness of deep music representations, considering musical semantics. The underlying assumption is that in case a deep representation is to be trusted, distance consistency between known related points should be maintained both in the input audio space and corresponding latent deep space. We generate known related points through semantically meaningful transformations, both considering imperceptible and graver transformations. Then, we examine within- and between-space distance consistencies, both considering audio space and latent embedded space, the latter either being a result of a conventional feature extractor or a deep encoder. We illustrate how our method, as a complement to task-specific performance, provides interpretable insight into what a network may have captured from training data signals. |
| Tasks | |
| Published | 2019-04-15 |
| URL | https://arxiv.org/abs/1904.07154v3 |
| https://arxiv.org/pdf/1904.07154v3.pdf | |
| PWC | https://paperswithcode.com/paper/are-nearby-neighbors-relatives-diagnosing |
| Repo | |
| Framework | |
Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry
| Title | Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry |
| Authors | Andre Wibisono |
| Abstract | We study the Proximal Langevin Algorithm (PLA) for sampling from a probability distribution $\nu = e^{-f}$ on $\mathbb{R}^n$ under isoperimetry. We prove a convergence guarantee for PLA in Kullback-Leibler (KL) divergence when $\nu$ satisfies log-Sobolev inequality (LSI) and $f$ has bounded second and third derivatives. This improves on the result for the Unadjusted Langevin Algorithm (ULA), and matches the fastest known rate for sampling under LSI (without Metropolis filter) with a better dependence on the LSI constant. We also prove convergence guarantees for PLA in R'enyi divergence of order $q > 1$ when the biased limit satisfies either LSI or Poincar'e inequality. |
| Tasks | |
| Published | 2019-11-04 |
| URL | https://arxiv.org/abs/1911.01469v1 |
| https://arxiv.org/pdf/1911.01469v1.pdf | |
| PWC | https://paperswithcode.com/paper/proximal-langevin-algorithm-rapid-convergence |
| Repo | |
| Framework | |
LAVAE: Disentangling Location and Appearance
| Title | LAVAE: Disentangling Location and Appearance |
| Authors | Andrea Dittadi, Ole Winther |
| Abstract | We propose a probabilistic generative model for unsupervised learning of structured, interpretable, object-based representations of visual scenes. We use amortized variational inference to train the generative model end-to-end. The learned representations of object location and appearance are fully disentangled, and objects are represented independently of each other in the latent space. Unlike previous approaches that disentangle location and appearance, ours generalizes seamlessly to scenes with many more objects than encountered in the training regime. We evaluate the proposed model on multi-MNIST and multi-dSprites data sets. |
| Tasks | |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11813v2 |
| https://arxiv.org/pdf/1909.11813v2.pdf | |
| PWC | https://paperswithcode.com/paper/lavae-disentangling-location-and-appearance-1 |
| Repo | |
| Framework | |
Plackett-Luce model for learning-to-rank task
| Title | Plackett-Luce model for learning-to-rank task |
| Authors | Tian Xia, Shaodan Zhai, Shaojun Wang |
| Abstract | List-wise based learning to rank methods are generally supposed to have better performance than point- and pair-wise based. However, in real-world applications, state-of-the-art systems are not from list-wise based camp. In this paper, we propose a new non-linear algorithm in the list-wise based framework called ListMLE, which uses the Plackett-Luce (PL) loss. Our experiments are conducted on the two largest publicly available real-world datasets, Yahoo challenge 2010 and Microsoft 30K. This is the first time in the single model level for a list-wise based system to match or overpass state-of-the-art systems in real-world datasets. |
| Tasks | Learning-To-Rank |
| Published | 2019-09-15 |
| URL | https://arxiv.org/abs/1909.06722v1 |
| https://arxiv.org/pdf/1909.06722v1.pdf | |
| PWC | https://paperswithcode.com/paper/plackett-luce-model-for-learning-to-rank-task |
| Repo | |
| Framework | |
Multi-View Fuzzy Clustering with The Alternative Learning between Shared Hidden Space and Partition
| Title | Multi-View Fuzzy Clustering with The Alternative Learning between Shared Hidden Space and Partition |
| Authors | Zhaohong Deng, Chen Cui, Peng Xu, Ling Liang, Haoran Chen, Te Zhang, Shitong Wang |
| Abstract | As the multi-view data grows in the real world, multi-view clus-tering has become a prominent technique in data mining, pattern recognition, and machine learning. How to exploit the relation-ship between different views effectively using the characteristic of multi-view data has become a crucial challenge. Aiming at this, a hidden space sharing multi-view fuzzy clustering (HSS-MVFC) method is proposed in the present study. This method is based on the classical fuzzy c-means clustering model, and obtains associ-ated information between different views by introducing shared hidden space. Especially, the shared hidden space and the fuzzy partition can be learned alternatively and contribute to each other. Meanwhile, the proposed method uses maximum entropy strategy to control the weights of different views while learning the shared hidden space. The experimental result shows that the proposed multi-view clustering method has better performance than many related clustering methods. |
| Tasks | |
| Published | 2019-08-12 |
| URL | https://arxiv.org/abs/1908.04771v1 |
| https://arxiv.org/pdf/1908.04771v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-view-fuzzy-clustering-with-the |
| Repo | |
| Framework | |
New pointwise convolution in Deep Neural Networks through Extremely Fast and Non Parametric Transforms
| Title | New pointwise convolution in Deep Neural Networks through Extremely Fast and Non Parametric Transforms |
| Authors | Joonhyun Jeong, Sung-Ho Bae |
| Abstract | Some conventional transforms such as Discrete Walsh-Hadamard Transform (DWHT) and Discrete Cosine Transform (DCT) have been widely used as feature extractors in image processing but rarely applied in neural networks. However, we found that these conventional transforms have the ability to capture the cross-channel correlations without any learnable parameters in DNNs. This paper firstly proposes to apply conventional transforms to pointwise convolution, showing that such transforms significantly reduce the computational complexity of neural networks without accuracy performance degradation. Especially for DWHT, it requires no floating point multiplications but only additions and subtractions, which can considerably reduce computation overheads. In addition, its fast algorithm further reduces complexity of floating point addition from $\mathcal{O}(n^2)$ to $\mathcal{O}(n\log n)$. These nice properties construct extremely efficient networks in the number parameters and operations, enjoying accuracy gain. Our proposed DWHT-based model gained 1.49% accuracy increase with 79.1% reduced parameters and 48.4% reduced FLOPs compared with its baseline model (MoblieNet-V1) on the CIFAR 100 dataset. |
| Tasks | |
| Published | 2019-06-25 |
| URL | https://arxiv.org/abs/1906.12172v1 |
| https://arxiv.org/pdf/1906.12172v1.pdf | |
| PWC | https://paperswithcode.com/paper/new-pointwise-convolution-in-deep-neural |
| Repo | |
| Framework | |
Enhanced Input Modeling for Construction Simulation using Bayesian Deep Neural Networks
| Title | Enhanced Input Modeling for Construction Simulation using Bayesian Deep Neural Networks |
| Authors | Yitong Li, Wenying Ji |
| Abstract | This paper aims to propose a novel deep learning-integrated framework for deriving reliable simulation input models through incorporating multi-source information. The framework sources and extracts multisource data generated from construction operations, which provides rich information for input modeling. The framework implements Bayesian deep neural networks to facilitate the purpose of incorporating richer information in input modeling. A case study on road paving operation is performed to test the feasibility and applicability of the proposed framework. Overall, this research enhances input modeling by deriving detailed input models, thereby, augmenting the decision-making processes in construction operations. This research also sheds lights on prompting data-driven simulation through incorporating machine learning techniques. |
| Tasks | Decision Making |
| Published | 2019-06-14 |
| URL | https://arxiv.org/abs/1906.06421v1 |
| https://arxiv.org/pdf/1906.06421v1.pdf | |
| PWC | https://paperswithcode.com/paper/enhanced-input-modeling-for-construction |
| Repo | |
| Framework | |
Accelerating Langevin Sampling with Birth-death
| Title | Accelerating Langevin Sampling with Birth-death |
| Authors | Yulong Lu, Jianfeng Lu, James Nolen |
| Abstract | A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions. Due to metastability, multimodal distributions are difficult to sample using standard Markov chain Monte Carlo methods. We propose a new sampling algorithm based on a birth-death mechanism to accelerate the mixing of Langevin diffusion. Our algorithm is motivated by its mean field partial differential equation (PDE), which is a Fokker-Planck equation supplemented by a nonlocal birth-death term. This PDE can be viewed as a gradient flow of the Kullback-Leibler divergence with respect to the Wasserstein-Fisher-Rao metric. We prove that under some assumptions the asymptotic convergence rate of the nonlocal PDE is independent of the potential barrier, in contrast to the exponential dependence in the case of the Langevin diffusion. We illustrate the efficiency of the birth-death accelerated Langevin method through several analytical examples and numerical experiments. |
| Tasks | Bayesian Inference |
| Published | 2019-05-23 |
| URL | https://arxiv.org/abs/1905.09863v1 |
| https://arxiv.org/pdf/1905.09863v1.pdf | |
| PWC | https://paperswithcode.com/paper/accelerating-langevin-sampling-with-birth |
| Repo | |
| Framework | |
CoverNet: Multimodal Behavior Prediction using Trajectory Sets
| Title | CoverNet: Multimodal Behavior Prediction using Trajectory Sets |
| Authors | Tung Phan-Minh, Elena Corina Grigore, Freddy A. Boulton, Oscar Beijbom, Eric M. Wolff |
| Abstract | We present CoverNet, a new method for multimodal, probabilistic trajectory prediction in urban driving scenarios. Previous work has employed a variety of methods, including multimodal regression, occupancy maps, and 1-step stochastic policies. We instead frame the trajectory prediction problem as classification over a diverse set of trajectories. The size of this set remains manageable, due to the fact that there are a limited number of distinct actions that can be taken over a reasonable prediction horizon. We structure the trajectory set to a) ensure a desired level of coverage of the state space, and b) eliminate physically impossible trajectories. By dynamically generating trajectory sets based on the agent’s current state, we can further improve the efficiency of our method. We demonstrate our approach on public, real-world self-driving datasets, and show that it outperforms state-of-the-art methods. |
| Tasks | Trajectory Prediction |
| Published | 2019-11-23 |
| URL | https://arxiv.org/abs/1911.10298v1 |
| https://arxiv.org/pdf/1911.10298v1.pdf | |
| PWC | https://paperswithcode.com/paper/covernet-multimodal-behavior-prediction-using |
| Repo | |
| Framework | |
Interpretation of Feature Space using Multi-Channel Attentional Sub-Networks
| Title | Interpretation of Feature Space using Multi-Channel Attentional Sub-Networks |
| Authors | Masanari Kimura, Masayuki Tanaka |
| Abstract | Convolutional Neural Networks have achieved impressive results in various tasks, but interpreting the internal mechanism is a challenging problem. To tackle this problem, we exploit a multi-channel attention mechanism in feature space. Our network architecture allows us to obtain an attention mask for each feature while existing CNN visualization methods provide only a common attention mask for all features. We apply the proposed multi-channel attention mechanism to multi-attribute recognition task. We can obtain different attention mask for each feature and for each attribute. Those analyses give us deeper insight into the feature space of CNNs. The experimental results for the benchmark dataset show that the proposed method gives high interpretability to humans while accurately grasping the attributes of the data. |
| Tasks | |
| Published | 2019-04-30 |
| URL | http://arxiv.org/abs/1904.13078v1 |
| http://arxiv.org/pdf/1904.13078v1.pdf | |
| PWC | https://paperswithcode.com/paper/interpretation-of-feature-space-using-multi |
| Repo | |
| Framework | |
Accelerating Transformer Decoding via a Hybrid of Self-attention and Recurrent Neural Network
| Title | Accelerating Transformer Decoding via a Hybrid of Self-attention and Recurrent Neural Network |
| Authors | Chengyi Wang, Shuangzhi Wu, Shujie Liu |
| Abstract | Due to the highly parallelizable architecture, Transformer is faster to train than RNN-based models and popularly used in machine translation tasks. However, at inference time, each output word requires all the hidden states of the previously generated words, which limits the parallelization capability, and makes it much slower than RNN-based ones. In this paper, we systematically analyze the time cost of different components of both the Transformer and RNN-based model. Based on it, we propose a hybrid network of self-attention and RNN structures, in which, the highly parallelizable self-attention is utilized as the encoder, and the simpler RNN structure is used as the decoder. Our hybrid network can decode 4-times faster than the Transformer. In addition, with the help of knowledge distillation, our hybrid network achieves comparable translation quality to the original Transformer. |
| Tasks | Machine Translation |
| Published | 2019-09-05 |
| URL | https://arxiv.org/abs/1909.02279v1 |
| https://arxiv.org/pdf/1909.02279v1.pdf | |
| PWC | https://paperswithcode.com/paper/accelerating-transformer-decoding-via-a |
| Repo | |
| Framework | |
Improved Activity Forecasting for Generating Trajectories
| Title | Improved Activity Forecasting for Generating Trajectories |
| Authors | Daisuke Ogawa, Toru Tamaki, Tsubasa Hirakawa, Bisser Raytchev, Kazufumi Kaneda, Ken Yoda |
| Abstract | An efficient inverse reinforcement learning for generating trajectories is proposed based of 2D and 3D activity forecasting. We modify reward function with $L_p$ norm and propose convolution into value iteration steps, which is called convolutional value iteration. Experimental results with seabird trajectories (43 for training and 10 for test), our method is best in terms of MHD error and performs fastest. Generated trajectories for interpolating missing parts of trajectories look much similar to real seabird trajectories than those by the previous works. |
| Tasks | |
| Published | 2019-12-12 |
| URL | https://arxiv.org/abs/1912.05729v1 |
| https://arxiv.org/pdf/1912.05729v1.pdf | |
| PWC | https://paperswithcode.com/paper/improved-activity-forecasting-for-generating |
| Repo | |
| Framework | |
No Fear of the Dark: Image Retrieval under Varying Illumination Conditions
| Title | No Fear of the Dark: Image Retrieval under Varying Illumination Conditions |
| Authors | Tomas Jenicek, Ondřej Chum |
| Abstract | Image retrieval under varying illumination conditions, such as day and night images, is addressed by image preprocessing, both hand-crafted and learned. Prior to extracting image descriptors by a convolutional neural network, images are photometrically normalised in order to reduce the descriptor sensitivity to illumination changes. We propose a learnable normalisation based on the U-Net architecture, which is trained on a combination of single-camera multi-exposure images and a newly constructed collection of similar views of landmarks during day and night. We experimentally show that both hand-crafted normalisation based on local histogram equalisation and the learnable normalisation outperform standard approaches in varying illumination conditions, while staying on par with the state-of-the-art methods on daylight illumination benchmarks, such as Oxford or Paris datasets. |
| Tasks | Image Retrieval |
| Published | 2019-08-23 |
| URL | https://arxiv.org/abs/1908.08999v1 |
| https://arxiv.org/pdf/1908.08999v1.pdf | |
| PWC | https://paperswithcode.com/paper/no-fear-of-the-dark-image-retrieval-under |
| Repo | |
| Framework | |
