May 6, 2019

3222 words 16 mins read

Paper Group ANR 426

Cost-Optimal Algorithms for Planning with Procedural Control Knowledge. Efficient Convolutional Auto-Encoding via Random Convexification and Frequency-Domain Minimization. HyperNetworks. Proceedings of the third “international Traveling Workshop on Interactions between Sparse models and Technology” (iTWIST’16). Inferring low-dimensional microstruct …

Cost-Optimal Algorithms for Planning with Procedural Control Knowledge


Title	Cost-Optimal Algorithms for Planning with Procedural Control Knowledge
Authors	Vikas Shivashankar, Ron Alford, Mark Roberts, David W. Aha
Abstract	There is an impressive body of work on developing heuristics and other reasoning algorithms to guide search in optimal and anytime planning algorithms for classical planning. However, very little effort has been directed towards developing analogous techniques to guide search towards high-quality solutions in hierarchical planning formalisms like HTN planning, which allows using additional domain-specific procedural control knowledge. In lieu of such techniques, this control knowledge often needs to provide the necessary search guidance to the planning algorithm, which imposes a substantial burden on the domain author and can yield brittle or error-prone domain models. We address this gap by extending recent work on a new hierarchical goal-based planning formalism called Hierarchical Goal Network (HGN) Planning to develop the Hierarchically-Optimal Goal Decomposition Planner (HOpGDP), an HGN planning algorithm that computes hierarchically-optimal plans. HOpGDP is guided by $h_{HL}$, a new HGN planning heuristic that extends existing admissible landmark-based heuristics from classical planning to compute admissible cost estimates for HGN planning problems. Our experimental evaluation across three benchmark planning domains shows that HOpGDP compares favorably to both optimal classical planners due to its ability to use domain-specific procedural knowledge, and a blind-search version of HOpGDP due to the search guidance provided by $h_{HL}$.
Tasks
Published	2016-07-06
URL	http://arxiv.org/abs/1607.01729v2
PDF	http://arxiv.org/pdf/1607.01729v2.pdf
PWC	https://paperswithcode.com/paper/cost-optimal-algorithms-for-planning-with
Repo
Framework

Efficient Convolutional Auto-Encoding via Random Convexification and Frequency-Domain Minimization


Title	Efficient Convolutional Auto-Encoding via Random Convexification and Frequency-Domain Minimization
Authors	Meshia Cédric Oveneke, Mitchel Aliosha-Perez, Yong Zhao, Dongmei Jiang, Hichem Sahli
Abstract	The omnipresence of deep learning architectures such as deep convolutional neural networks (CNN)s is fueled by the synergistic combination of ever-increasing labeled datasets and specialized hardware. Despite the indisputable success, the reliance on huge amounts of labeled data and specialized hardware can be a limiting factor when approaching new applications. To help alleviating these limitations, we propose an efficient learning strategy for layer-wise unsupervised training of deep CNNs on conventional hardware in acceptable time. Our proposed strategy consists of randomly convexifying the reconstruction contractive auto-encoding (RCAE) learning objective and solving the resulting large-scale convex minimization problem in the frequency domain via coordinate descent (CD). The main advantages of our proposed learning strategy are: (1) single tunable optimization parameter; (2) fast and guaranteed convergence; (3) possibilities for full parallelization. Numerical experiments show that our proposed learning strategy scales (in the worst case) linearly with image size, number of filters and filter size.
Tasks
Published	2016-11-28
URL	http://arxiv.org/abs/1611.09232v1
PDF	http://arxiv.org/pdf/1611.09232v1.pdf
PWC	https://paperswithcode.com/paper/efficient-convolutional-auto-encoding-via
Repo
Framework

HyperNetworks


Title	HyperNetworks
Authors	David Ha, Andrew Dai, Quoc V. Le
Abstract	This work explores hypernetworks: an approach of using a one network, also known as a hypernetwork, to generate the weights for another network. Hypernetworks provide an abstraction that is similar to what is found in nature: the relationship between a genotype - the hypernetwork - and a phenotype - the main network. Though they are also reminiscent of HyperNEAT in evolution, our hypernetworks are trained end-to-end with backpropagation and thus are usually faster. The focus of this work is to make hypernetworks useful for deep convolutional networks and long recurrent networks, where hypernetworks can be viewed as relaxed form of weight-sharing across layers. Our main result is that hypernetworks can generate non-shared weights for LSTM and achieve near state-of-the-art results on a variety of sequence modelling tasks including character-level language modelling, handwriting generation and neural machine translation, challenging the weight-sharing paradigm for recurrent networks. Our results also show that hypernetworks applied to convolutional networks still achieve respectable results for image recognition tasks compared to state-of-the-art baseline models while requiring fewer learnable parameters.
Tasks	Language Modelling, Machine Translation
Published	2016-09-27
URL	http://arxiv.org/abs/1609.09106v4
PDF	http://arxiv.org/pdf/1609.09106v4.pdf
PWC	https://paperswithcode.com/paper/hypernetworks
Repo
Framework

Proceedings of the third “international Traveling Workshop on Interactions between Sparse models and Technology” (iTWIST’16)


Title	Proceedings of the third “international Traveling Workshop on Interactions between Sparse models and Technology” (iTWIST’16)
Authors	V. Abrol, O. Absil, P. -A. Absil, S. Anthoine, P. Antoine, T. Arildsen, N. Bertin, F. Bleichrodt, J. Bobin, A. Bol, A. Bonnefoy, F. Caltagirone, V. Cambareri, C. Chenot, V. Crnojević, M. Daňková, K. Degraux, J. Eisert, J. M. Fadili, M. Gabrié, N. Gac, D. Giacobello, A. Gonzalez, C. A. Gomez Gonzalez, A. González, P. -Y. Gousenbourger, M. Græsbøll Christensen, R. Gribonval, S. Guérit, S. Huang, P. Irofti, L. Jacques, U. S. Kamilov, S. Kiticć, M. Kliesch, F. Krzakala, J. A. Lee, W. Liao, T. Lindstrøm Jensen, A. Manoel, H. Mansour, A. Mohammad-Djafari, A. Moshtaghpour, F. Ngolè, B. Pairet, M. Panić, G. Peyré, A. Pižurica, P. Rajmic, M. Roblin, I. Roth, A. K. Sao, P. Sharma, J. -L. Starck, E. W. Tramel, T. van Waterschoot, D. Vukobratovic, L. Wang, B. Wirth, G. Wunder, H. Zhang
Abstract	The third edition of the “international - Traveling Workshop on Interactions between Sparse models and Technology” (iTWIST) took place in Aalborg, the 4th largest city in Denmark situated beautifully in the northern part of the country, from the 24th to 26th of August 2016. The workshop venue was at the Aalborg University campus. One implicit objective of this biennial workshop is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For this third edition, iTWIST’16 gathered about 50 international participants and features 8 invited talks, 12 oral presentations, and 12 posters on the following themes, all related to the theory, application and generalization of the “sparsity paradigm”: Sparsity-driven data sensing and processing (e.g., optics, computer vision, genomics, biomedical, digital communication, channel estimation, astronomy); Application of sparse models in non-convex/non-linear inverse problems (e.g., phase retrieval, blind deconvolution, self calibration); Approximate probabilistic inference for sparse problems; Sparse machine learning and inference; “Blind” inverse problems and dictionary learning; Optimization for sparse modelling; Information theory, geometry and randomness; Sparsity? What’s next? (Discrete-valued signals; Union of low-dimensional spaces, Cosparsity, mixed/group norm, model-based, low-complexity models, …); Matrix/manifold sensing/processing (graph, low-rank approximation, …); Complexity/accuracy tradeoffs in numerical methods/optimization; Electronic/optical compressive sensors (hardware).
Tasks	Calibration, Dictionary Learning
Published	2016-09-14
URL	http://arxiv.org/abs/1609.04167v1
PDF	http://arxiv.org/pdf/1609.04167v1.pdf
PWC	https://paperswithcode.com/paper/proceedings-of-the-third-international
Repo
Framework

Inferring low-dimensional microstructure representations using convolutional neural networks


Title	Inferring low-dimensional microstructure representations using convolutional neural networks
Authors	Nicholas Lubbers, Turab Lookman, Kipton Barros
Abstract	We apply recent advances in machine learning and computer vision to a central problem in materials informatics: The statistical representation of microstructural images. We use activations in a pre-trained convolutional neural network to provide a high-dimensional characterization of a set of synthetic microstructural images. Next, we use manifold learning to obtain a low-dimensional embedding of this statistical characterization. We show that the low-dimensional embedding extracts the parameters used to generate the images. According to a variety of metrics, the convolutional neural network method yields dramatically better embeddings than the analogous method derived from two-point correlations alone.
Tasks
Published	2016-11-08
URL	http://arxiv.org/abs/1611.02764v2
PDF	http://arxiv.org/pdf/1611.02764v2.pdf
PWC	https://paperswithcode.com/paper/inferring-low-dimensional-microstructure
Repo
Framework

Optimal mean-based algorithms for trace reconstruction


Title	Optimal mean-based algorithms for trace reconstruction
Authors	Anindya De, Ryan O’Donnell, Rocco Servedio
Abstract	In the (deletion-channel) trace reconstruction problem, there is an unknown $n$-bit source string $x$. An algorithm is given access to independent traces of $x$, where a trace is formed by deleting each bit of~$x$ independently with probability~$\delta$. The goal of the algorithm is to recover~$x$ exactly (with high probability), while minimizing samples (number of traces) and running time. Previously, the best known algorithm for the trace reconstruction problem was due to Holenstein~et~al.; it uses $\exp(\tilde{O}(n^{1/2}))$ samples and running time for any fixed $0 < \delta < 1$. It is also what we call a “mean-based algorithm”, meaning that it only uses the empirical means of the individual bits of the traces. Holenstein~et~al.~also gave a lower bound, showing that any mean-based algorithm must use at least $n^{\tilde{\Omega}(\log n)}$ samples. In this paper we improve both of these results, obtaining matching upper and lower bounds for mean-based trace reconstruction. For any constant deletion rate $0 < \delta < 1$, we give a mean-based algorithm that uses $\exp(O(n^{1/3}))$ time and traces; we also prove that any mean-based algorithm must use at least $\exp(\Omega(n^{1/3}))$ traces. In fact, we obtain matching upper and lower bounds even for $\delta$ subconstant and $\rho := 1-\delta$ subconstant: when $(\log^3 n)/n \ll \delta \leq 1/2$ the bound is $\exp(-\Theta(\delta n)^{1/3})$, and when $1/\sqrt{n} \ll \rho \leq 1/2$ the bound is $\exp(-\Theta(n/\rho)^{1/3})$. Our proofs involve estimates for the maxima of Littlewood polynomials on complex disks. We show that these techniques can also be used to perform trace reconstruction with random insertions and bit-flips in addition to deletions. We also find a surprising result: for deletion probabilities $\delta > 1/2$, the presence of insertions can actually help with trace reconstruction.
Tasks
Published	2016-12-09
URL	http://arxiv.org/abs/1612.03148v1
PDF	http://arxiv.org/pdf/1612.03148v1.pdf
PWC	https://paperswithcode.com/paper/optimal-mean-based-algorithms-for-trace
Repo
Framework

Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation


Title	Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation
Authors	Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, Zhi Jin
Abstract	Using neural networks to generate replies in human-computer dialogue systems is attracting increasing attention over the past few years. However, the performance is not satisfactory: the neural network tends to generate safe, universally relevant replies which carry little meaning. In this paper, we propose a content-introducing approach to neural network-based generative dialogue systems. We first use pointwise mutual information (PMI) to predict a noun as a keyword, reflecting the main gist of the reply. We then propose seq2BF, a “sequence to backward and forward sequences” model, which generates a reply containing the given keyword. Experimental results show that our approach significantly outperforms traditional sequence-to-sequence models in terms of human evaluation and the entropy measure, and that the predicted keyword can appear at an appropriate position in the reply.
Tasks	Short-Text Conversation
Published	2016-07-04
URL	http://arxiv.org/abs/1607.00970v2
PDF	http://arxiv.org/pdf/1607.00970v2.pdf
PWC	https://paperswithcode.com/paper/sequence-to-backward-and-forward-sequences-a
Repo
Framework

Quad-networks: unsupervised learning to rank for interest point detection


Title	Quad-networks: unsupervised learning to rank for interest point detection
Authors	Nikolay Savinov, Akihito Seki, Lubor Ladicky, Torsten Sattler, Marc Pollefeys
Abstract	Several machine learning tasks require to represent the data using only a sparse set of interest points. An ideal detector is able to find the corresponding interest points even if the data undergo a transformation typical for a given domain. Since the task is of high practical interest in computer vision, many hand-crafted solutions were proposed. In this paper, we ask a fundamental question: can we learn such detectors from scratch? Since it is often unclear what points are “interesting”, human labelling cannot be used to find a truly unbiased solution. Therefore, the task requires an unsupervised formulation. We are the first to propose such a formulation: training a neural network to rank points in a transformation-invariant manner. Interest points are then extracted from the top/bottom quantiles of this ranking. We validate our approach on two tasks: standard RGB image interest point detection and challenging cross-modal interest point detection between RGB and depth images. We quantitatively show that our unsupervised method performs better or on-par with baselines.
Tasks	Interest Point Detection, Learning-To-Rank
Published	2016-11-22
URL	http://arxiv.org/abs/1611.07571v2
PDF	http://arxiv.org/pdf/1611.07571v2.pdf
PWC	https://paperswithcode.com/paper/quad-networks-unsupervised-learning-to-rank
Repo
Framework

Addressing Community Question Answering in English and Arabic


Title	Addressing Community Question Answering in English and Arabic
Authors	Giovanni Da San Martino, Alberto Barrón-Cedeño, Salvatore Romeo, Alessandro Moschitti, Shafiq Joty, Fahad A. Al Obaidli, Kateryna Tymoshenko, Antonio Uva
Abstract	This paper studies the impact of different types of features applied to learning to re-rank questions in community Question Answering. We tested our models on two datasets released in SemEval-2016 Task 3 on “Community Question Answering”. Task 3 targeted real-life Web fora both in English and Arabic. Our models include bag-of-words features (BoW), syntactic tree kernels (TKs), rank features, embeddings, and machine translation evaluation features. To the best of our knowledge, structural kernels have barely been applied to the question reranking task, where they have to model paraphrase relations. In the case of the English question re-ranking task, we compare our learning to rank (L2R) algorithms against a strong baseline given by the Google-generated ranking (GR). The results show that i) the shallow structures used in our TKs are robust enough to noisy data and ii) improving GR is possible, but effective BoW features and TKs along with an accurate model of GR features in the used L2R algorithm are required. In the case of the Arabic question re-ranking task, for the first time we applied tree kernels on syntactic trees of Arabic sentences. Our approaches to both tasks obtained the second best results on SemEval-2016 subtasks B on English and D on Arabic.
Tasks	Community Question Answering, Learning-To-Rank, Machine Translation, Question Answering
Published	2016-10-18
URL	http://arxiv.org/abs/1610.05522v1
PDF	http://arxiv.org/pdf/1610.05522v1.pdf
PWC	https://paperswithcode.com/paper/addressing-community-question-answering-in
Repo
Framework

Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds


Title	Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds
Authors	Ming Yin, Yi Guo, Junbin Gao, Zhaoshui He, Shengli Xie
Abstract	Sparse subspace clustering (SSC), as one of the most successful subspace clustering methods, has achieved notable clustering accuracy in computer vision tasks. However, SSC applies only to vector data in Euclidean space. As such, there is still no satisfactory approach to solve subspace clustering by ${\it self-expressive}$ principle for symmetric positive definite (SPD) matrices which is very useful in computer vision. In this paper, by embedding the SPD matrices into a Reproducing Kernel Hilbert Space (RKHS), a kernel subspace clustering method is constructed on the SPD manifold through an appropriate Log-Euclidean kernel, termed as kernel sparse subspace clustering on the SPD Riemannian manifold (KSSCR). By exploiting the intrinsic Riemannian geometry within data, KSSCR can effectively characterize the geodesic distance between SPD matrices to uncover the underlying subspace structure. Experimental results on two famous database demonstrate that the proposed method achieves better clustering results than the state-of-the-art approaches.
Tasks
Published	2016-01-04
URL	http://arxiv.org/abs/1601.00414v1
PDF	http://arxiv.org/pdf/1601.00414v1.pdf
PWC	https://paperswithcode.com/paper/kernel-sparse-subspace-clustering-on
Repo
Framework

Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation


Title	Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation
Authors	Xun Xu, Timothy M. Hospedales, Shaogang Gong
Abstract	Zero-Shot Learning (ZSL) promises to scale visual recognition by bypassing the conventional model training requirement of annotated examples for every category. This is achieved by establishing a mapping connecting low-level features and a semantic description of the label space, referred as visual-semantic mapping, on auxiliary data. Reusing the learned mapping to project target videos into an embedding space thus allows novel-classes to be recognised by nearest neighbour inference. However, existing ZSL methods suffer from auxiliary-target domain shift intrinsically induced by assuming the same mapping for the disjoint auxiliary and target classes. This compromises the generalisation accuracy of ZSL recognition on the target data. In this work, we improve the ability of ZSL to generalise across this domain shift in both model- and data-centric ways by formulating a visual-semantic mapping with better generalisation properties and a dynamic data re-weighting method to prioritise auxiliary data that are relevant to the target classes. Specifically: (1) We introduce a multi-task visual-semantic mapping to improve generalisation by constraining the semantic mapping parameters to lie on a low-dimensional manifold, (2) We explore prioritised data augmentation by expanding the pool of auxiliary data with additional instances weighted by relevance to the target domain. The proposed new model is applied to the challenging zero-shot action recognition problem to demonstrate its advantages over existing ZSL models.
Tasks	Data Augmentation, Temporal Action Localization, Zero-Shot Learning
Published	2016-11-26
URL	http://arxiv.org/abs/1611.08663v1
PDF	http://arxiv.org/pdf/1611.08663v1.pdf
PWC	https://paperswithcode.com/paper/multi-task-zero-shot-action-recognition-with
Repo
Framework

Context-Dependent Word Representation for Neural Machine Translation


Title	Context-Dependent Word Representation for Neural Machine Translation
Authors	Heeyoul Choi, Kyunghyun Cho, Yoshua Bengio
Abstract	We first observe a potential weakness of continuous vector representations of symbols in neural machine translation. That is, the continuous vector representation, or a word embedding vector, of a symbol encodes multiple dimensions of similarity, equivalent to encoding more than one meaning of the word. This has the consequence that the encoder and decoder recurrent networks in neural machine translation need to spend substantial amount of their capacity in disambiguating source and target words based on the context which is defined by a source sentence. Based on this observation, in this paper we propose to contextualize the word embedding vectors using a nonlinear bag-of-words representation of the source sentence. Additionally, we propose to represent special tokens (such as numbers, proper nouns and acronyms) with typed symbols to facilitate translating those words that are not well-suited to be translated via continuous vectors. The experiments on En-Fr and En-De reveal that the proposed approaches of contextualization and symbolization improves the translation quality of neural machine translation systems significantly.
Tasks	Machine Translation
Published	2016-07-03
URL	http://arxiv.org/abs/1607.00578v1
PDF	http://arxiv.org/pdf/1607.00578v1.pdf
PWC	https://paperswithcode.com/paper/context-dependent-word-representation-for
Repo
Framework

On the Convergent Properties of Word Embedding Methods


Title	On the Convergent Properties of Word Embedding Methods
Authors	Yingtao Tian, Vivek Kulkarni, Bryan Perozzi, Steven Skiena
Abstract	Do word embeddings converge to learn similar things over different initializations? How repeatable are experiments with word embeddings? Are all word embedding techniques equally reliable? In this paper we propose evaluating methods for learning word representations by their consistency across initializations. We propose a measure to quantify the similarity of the learned word representations under this setting (where they are subject to different random initializations). Our preliminary results illustrate that our metric not only measures a intrinsic property of word embedding methods but also correlates well with other evaluation metrics on downstream tasks. We believe our methods are is useful in characterizing robustness – an important property to consider when developing new word embedding methods.
Tasks	Word Embeddings
Published	2016-05-12
URL	http://arxiv.org/abs/1605.03956v1
PDF	http://arxiv.org/pdf/1605.03956v1.pdf
PWC	https://paperswithcode.com/paper/on-the-convergent-properties-of-word
Repo
Framework

Some medical applications of example-based super-resolution


Title	Some medical applications of example-based super-resolution
Authors	Ramin Zabih
Abstract	Example-based super-resolution (EBSR) reconstructs a high-resolution image from a low-resolution image, given a training set of high-resolution images. In this note I propose some applications of EBSR to medical imaging. A particular interesting application, which I call “x-ray voxelization”, approximates the result of a CT scan from an x-ray image.
Tasks	Super-Resolution
Published	2016-04-17
URL	http://arxiv.org/abs/1604.04926v1
PDF	http://arxiv.org/pdf/1604.04926v1.pdf
PWC	https://paperswithcode.com/paper/some-medical-applications-of-example-based
Repo
Framework

Leveraging Network Dynamics for Improved Link Prediction


Title	Leveraging Network Dynamics for Improved Link Prediction
Authors	Alireza Hajibagheri, Gita Sukthankar, Kiran Lakkaraju
Abstract	The aim of link prediction is to forecast connections that are most likely to occur in the future, based on examples of previously observed links. A key insight is that it is useful to explicitly model network dynamics, how frequently links are created or destroyed when doing link prediction. In this paper, we introduce a new supervised link prediction framework, RPM (Rate Prediction Model). In addition to network similarity measures, RPM uses the predicted rate of link modifications, modeled using time series data; it is implemented in Spark-ML and trained with the original link distribution, rather than a small balanced subset. We compare the use of this network dynamics model to directly creating time series of network similarity measures. Our experiments show that RPM, which leverages predicted rates, outperforms the use of network similarity measures, either individually or within a time series.
Tasks	Link Prediction, Time Series
Published	2016-04-08
URL	http://arxiv.org/abs/1604.03221v1
PDF	http://arxiv.org/pdf/1604.03221v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-network-dynamics-for-improved-link
Repo
Framework