Paper Group ANR 398
Object categorization in finer levels requires higher spatial frequencies, and therefore takes longer. Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models. Coding-theorem Like Behaviour and Emergence of the Universal Distribution from Resource-bounded Algorithmic Probability. Improved Representation Learn …
Object categorization in finer levels requires higher spatial frequencies, and therefore takes longer
| Title | Object categorization in finer levels requires higher spatial frequencies, and therefore takes longer |
| Authors | Matin N. Ashtiani, Saeed Reza Kheradpisheh, Timothée Masquelier, Mohammad Ganjtabesh |
| Abstract | The human visual system contains a hierarchical sequence of modules that take part in visual perception at different levels of abstraction, i.e., superordinate, basic, and subordinate levels. One important question is to identify the “entry” level at which the visual representation is commenced in the process of object recognition. For a long time, it was believed that the basic level had advantage over two others; a claim that has been challenged recently. Here we used a series of psychophysics experiments, based on a rapid presentation paradigm, as well as two computational models, with bandpass filtered images to study the processing order of the categorization levels. In these experiments, we investigated the type of visual information required for categorizing objects in each level by varying the spatial frequency bands of the input image. The results of our psychophysics experiments and computational models are consistent. They indicate that the different spatial frequency information had different effects on object categorization in each level. In the absence of high frequency information, subordinate and basic level categorization are performed inaccurately, while superordinate level is performed well. This means that, low frequency information is sufficient for superordinate level, but not for the basic and subordinate levels. These finer levels require high frequency information, which appears to take longer to be processed, leading to longer reaction times. Finally, to avoid the ceiling effect, we evaluated the robustness of the results by adding different amounts of noise to the input images and repeating the experiments. As expected, the categorization accuracy decreased and the reaction time increased significantly, but the trends were the same.This shows that our results are not due to a ceiling effect. |
| Tasks | Object Recognition |
| Published | 2017-03-29 |
| URL | http://arxiv.org/abs/1703.09990v1 |
| http://arxiv.org/pdf/1703.09990v1.pdf | |
| PWC | https://paperswithcode.com/paper/object-categorization-in-finer-levels |
| Repo | |
| Framework | |
Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models
| Title | Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models |
| Authors | Louis Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, Ray Kurzweil |
| Abstract | Sequence-to-sequence models have been applied to the conversation response generation problem where the source sequence is the conversation history and the target sequence is the response. Unlike translation, conversation responding is inherently creative. The generation of long, informative, coherent, and diverse responses remains a hard task. In this work, we focus on the single turn setting. We add self-attention to the decoder to maintain coherence in longer responses, and we propose a practical approach, called the glimpse-model, for scaling to large datasets. We introduce a stochastic beam-search algorithm with segment-by-segment reranking which lets us inject diversity earlier in the generation process. We trained on a combined data set of over 2.3B conversation messages mined from the web. In human evaluation studies, our method produces longer responses overall, with a higher proportion rated as acceptable and excellent as length increases, compared to baseline sequence-to-sequence models with explicit length-promotion. A back-off strategy produces better responses overall, in the full spectrum of lengths. |
| Tasks | |
| Published | 2017-01-11 |
| URL | http://arxiv.org/abs/1701.03185v2 |
| http://arxiv.org/pdf/1701.03185v2.pdf | |
| PWC | https://paperswithcode.com/paper/generating-high-quality-and-informative |
| Repo | |
| Framework | |
Coding-theorem Like Behaviour and Emergence of the Universal Distribution from Resource-bounded Algorithmic Probability
| Title | Coding-theorem Like Behaviour and Emergence of the Universal Distribution from Resource-bounded Algorithmic Probability |
| Authors | Hector Zenil, Liliana Badillo, Santiago Hernández-Orozco, Francisco Hernández-Quiroz |
| Abstract | Previously referred to as `miraculous’ in the scientific literature because of its powerful properties and its wide application as optimal solution to the problem of induction/inference, (approximations to) Algorithmic Probability (AP) and the associated Universal Distribution are (or should be) of the greatest importance in science. Here we investigate the emergence, the rates of emergence and convergence, and the Coding-theorem like behaviour of AP in Turing-subuniversal models of computation. We investigate empirical distributions of computing models in the Chomsky hierarchy. We introduce measures of algorithmic probability and algorithmic complexity based upon resource-bounded computation, in contrast to previously thoroughly investigated distributions produced from the output distribution of Turing machines. This approach allows for numerical approximations to algorithmic (Kolmogorov-Chaitin) complexity-based estimations at each of the levels of a computational hierarchy. We demonstrate that all these estimations are correlated in rank and that they converge both in rank and values as a function of computational power, despite fundamental differences between computational models. In the context of natural processes that operate below the Turing universal level because of finite resources and physical degradation, the investigation of natural biases stemming from algorithmic rules may shed light on the distribution of outcomes. We show that up to 60% of the simplicity/complexity bias in distributions produced even by the weakest of the computational models can be accounted for by Algorithmic Probability in its approximation to the Universal Distribution. | |
| Tasks | |
| Published | 2017-11-06 |
| URL | http://arxiv.org/abs/1711.01711v11 |
| http://arxiv.org/pdf/1711.01711v11.pdf | |
| PWC | https://paperswithcode.com/paper/coding-theorem-like-behaviour-and-emergence |
| Repo | |
| Framework | |
Improved Representation Learning for Predicting Commonsense Ontologies
| Title | Improved Representation Learning for Predicting Commonsense Ontologies |
| Authors | Xiang Li, Luke Vilnis, Andrew McCallum |
| Abstract | Recent work in learning ontologies (hierarchical and partially-ordered structures) has leveraged the intrinsic geometry of spaces of learned representations to make predictions that automatically obey complex structural constraints. We explore two extensions of one such model, the order-embedding model for hierarchical relation learning, with an aim towards improved performance on text data for commonsense knowledge representation. Our first model jointly learns ordering relations and non-hierarchical knowledge in the form of raw text. Our second extension exploits the partial order structure of the training data to find long-distance triplet constraints among embeddings which are poorly enforced by the pairwise training procedure. We find that both incorporating free text and augmented training constraints improve over the original order-embedding model and other strong baselines. |
| Tasks | Representation Learning |
| Published | 2017-08-01 |
| URL | http://arxiv.org/abs/1708.00549v1 |
| http://arxiv.org/pdf/1708.00549v1.pdf | |
| PWC | https://paperswithcode.com/paper/improved-representation-learning-for |
| Repo | |
| Framework | |
Label Propagation on K-partite Graphs with Heterophily
| Title | Label Propagation on K-partite Graphs with Heterophily |
| Authors | Dingxiong Deng, Fan Bai, Yiqi Tang, Shuigeng Zhou, Cyrus Shahabi, Linhong Zhu |
| Abstract | In this paper, for the first time, we study label propagation in heterogeneous graphs under heterophily assumption. Homophily label propagation (i.e., two connected nodes share similar labels) in homogeneous graph (with same types of vertices and relations) has been extensively studied before. Unfortunately, real-life networks are heterogeneous, they contain different types of vertices (e.g., users, images, texts) and relations (e.g., friendships, co-tagging) and allow for each node to propagate both the same and opposite copy of labels to its neighbors. We propose a $\mathcal{K}$-partite label propagation model to handle the mystifying combination of heterogeneous nodes/relations and heterophily propagation. With this model, we develop a novel label inference algorithm framework with update rules in near-linear time complexity. Since real networks change over time, we devise an incremental approach, which supports fast updates for both new data and evidence (e.g., ground truth labels) with guaranteed efficiency. We further provide a utility function to automatically determine whether an incremental or a re-modeling approach is favored. Extensive experiments on real datasets have verified the effectiveness and efficiency of our approach, and its superiority over the state-of-the-art label propagation methods. |
| Tasks | |
| Published | 2017-01-21 |
| URL | http://arxiv.org/abs/1701.06075v1 |
| http://arxiv.org/pdf/1701.06075v1.pdf | |
| PWC | https://paperswithcode.com/paper/label-propagation-on-k-partite-graphs-with |
| Repo | |
| Framework | |
Information Theoretic Properties of Markov Random Fields, and their Algorithmic Applications
| Title | Information Theoretic Properties of Markov Random Fields, and their Algorithmic Applications |
| Authors | Linus Hamilton, Frederic Koehler, Ankur Moitra |
| Abstract | Markov random fields area popular model for high-dimensional probability distributions. Over the years, many mathematical, statistical and algorithmic problems on them have been studied. Until recently, the only known algorithms for provably learning them relied on exhaustive search, correlation decay or various incoherence assumptions. Bresler gave an algorithm for learning general Ising models on bounded degree graphs. His approach was based on a structural result about mutual information in Ising models. Here we take a more conceptual approach to proving lower bounds on the mutual information through setting up an appropriate zero-sum game. Our proof generalizes well beyond Ising models, to arbitrary Markov random fields with higher order interactions. As an application, we obtain algorithms for learning Markov random fields on bounded degree graphs on $n$ nodes with $r$-order interactions in $n^r$ time and $\log n$ sample complexity. The sample complexity is information theoretically optimal up to the dependence on the maximum degree. The running time is nearly optimal under standard conjectures about the hardness of learning parity with noise. |
| Tasks | |
| Published | 2017-05-31 |
| URL | http://arxiv.org/abs/1705.11107v1 |
| http://arxiv.org/pdf/1705.11107v1.pdf | |
| PWC | https://paperswithcode.com/paper/information-theoretic-properties-of-markov |
| Repo | |
| Framework | |
Frangi-Net: A Neural Network Approach to Vessel Segmentation
| Title | Frangi-Net: A Neural Network Approach to Vessel Segmentation |
| Authors | Weilin Fu, Katharina Breininger, Tobias Würfl, Nishant Ravikumar, Roman Schaffert, Andreas Maier |
| Abstract | In this paper, we reformulate the conventional 2-D Frangi vesselness measure into a pre-weighted neural network (“Frangi-Net”), and illustrate that the Frangi-Net is equivalent to the original Frangi filter. Furthermore, we show that, as a neural network, Frangi-Net is trainable. We evaluate the proposed method on a set of 45 high resolution fundus images. After fine-tuning, we observe both qualitative and quantitative improvements in the segmentation quality compared to the original Frangi measure, with an increase up to $17%$ in F1 score. |
| Tasks | |
| Published | 2017-11-09 |
| URL | http://arxiv.org/abs/1711.03345v1 |
| http://arxiv.org/pdf/1711.03345v1.pdf | |
| PWC | https://paperswithcode.com/paper/frangi-net-a-neural-network-approach-to |
| Repo | |
| Framework | |
Stream Graphs and Link Streams for the Modeling of Interactions over Time
| Title | Stream Graphs and Link Streams for the Modeling of Interactions over Time |
| Authors | Matthieu Latapy, Tiphaine Viard, Clémence Magnien |
| Abstract | Graph theory provides a language for studying the structure of relations, and it is often used to study interactions over time too. However, it poorly captures the both temporal and structural nature of interactions, that calls for a dedicated formalism. In this paper, we generalize graph concepts in order to cope with both aspects in a consistent way. We start with elementary concepts like density, clusters, or paths, and derive from them more advanced concepts like cliques, degrees, clustering coefficients, or connected components. We obtain a language to directly deal with interactions over time, similar to the language provided by graphs to deal with relations. This formalism is self-consistent: usual relations between different concepts are preserved. It is also consistent with graph theory: graph concepts are special cases of the ones we introduce. This makes it easy to generalize higher-level objects such as quotient graphs, line graphs, k-cores, and centralities. This paper also considers discrete versus continuous time assumptions, instantaneous links, and extensions to more complex cases. |
| Tasks | |
| Published | 2017-10-11 |
| URL | http://arxiv.org/abs/1710.04073v1 |
| http://arxiv.org/pdf/1710.04073v1.pdf | |
| PWC | https://paperswithcode.com/paper/stream-graphs-and-link-streams-for-the |
| Repo | |
| Framework | |
Flexible End-to-End Dialogue System for Knowledge Grounded Conversation
| Title | Flexible End-to-End Dialogue System for Knowledge Grounded Conversation |
| Authors | Wenya Zhu, Kaixiang Mo, Yu Zhang, Zhangbin Zhu, Xuezheng Peng, Qiang Yang |
| Abstract | In knowledge grounded conversation, domain knowledge plays an important role in a special domain such as Music. The response of knowledge grounded conversation might contain multiple answer entities or no entity at all. Although existing generative question answering (QA) systems can be applied to knowledge grounded conversation, they either have at most one entity in a response or cannot deal with out-of-vocabulary entities. We propose a fully data-driven generative dialogue system GenDS that is capable of generating responses based on input message and related knowledge base (KB). To generate arbitrary number of answer entities even when these entities never appear in the training set, we design a dynamic knowledge enquirer which selects different answer entities at different positions in a single response, according to different local context. It does not rely on the representations of entities, enabling our model deal with out-of-vocabulary entities. We collect a human-human conversation data (ConversMusic) with knowledge annotations. The proposed method is evaluated on CoversMusic and a public question answering dataset. Our proposed GenDS system outperforms baseline methods significantly in terms of the BLEU, entity accuracy, entity recall and human evaluation. Moreover,the experiments also demonstrate that GenDS works better even on small datasets. |
| Tasks | Question Answering |
| Published | 2017-09-13 |
| URL | http://arxiv.org/abs/1709.04264v1 |
| http://arxiv.org/pdf/1709.04264v1.pdf | |
| PWC | https://paperswithcode.com/paper/flexible-end-to-end-dialogue-system-for |
| Repo | |
| Framework | |
Don’t Fear the Bit Flips: Optimized Coding Strategies for Binary Classification
| Title | Don’t Fear the Bit Flips: Optimized Coding Strategies for Binary Classification |
| Authors | Frederic Sala, Shahroze Kabir, Guy Van den Broeck, Lara Dolecek |
| Abstract | After being trained, classifiers must often operate on data that has been corrupted by noise. In this paper, we consider the impact of such noise on the features of binary classifiers. Inspired by tools for classifier robustness, we introduce the same classification probability (SCP) to measure the resulting distortion on the classifier outputs. We introduce a low-complexity estimate of the SCP based on quantization and polynomial multiplication. We also study channel coding techniques based on replication error-correcting codes. In contrast to the traditional channel coding approach, where error-correction is meant to preserve the data and is agnostic to the application, our schemes specifically aim to maximize the SCP (equivalently minimizing the distortion of the classifier output) for the same redundancy overhead. |
| Tasks | Quantization |
| Published | 2017-03-08 |
| URL | http://arxiv.org/abs/1703.02641v1 |
| http://arxiv.org/pdf/1703.02641v1.pdf | |
| PWC | https://paperswithcode.com/paper/dont-fear-the-bit-flips-optimized-coding |
| Repo | |
| Framework | |
End-to-End Supervised Product Quantization for Image Search and Retrieval
| Title | End-to-End Supervised Product Quantization for Image Search and Retrieval |
| Authors | Benjamin Klein, Lior Wolf |
| Abstract | Product Quantization, a dictionary based hashing method, is one of the leading unsupervised hashing techniques. While it ignores the labels, it harnesses the features to construct look up tables that can approximate the feature space. In recent years, several works have achieved state of the art results on hashing benchmarks by learning binary representations in a supervised manner. This work presents Deep Product Quantization (DPQ), a technique that leads to more accurate retrieval and classification than the latest state of the art methods, while having similar computational complexity and memory footprint as the Product Quantization method. To our knowledge, this is the first work to introduce a dictionary-based representation that is inspired by Product Quantization and which is learned end-to-end, and thus benefits from the supervised signal. DPQ explicitly learns soft and hard representations to enable an efficient and accurate asymmetric search, by using a straight-through estimator. Our method obtains state of the art results on an extensive array of retrieval and classification experiments. |
| Tasks | Image Retrieval, Quantization |
| Published | 2017-11-23 |
| URL | https://arxiv.org/abs/1711.08589v2 |
| https://arxiv.org/pdf/1711.08589v2.pdf | |
| PWC | https://paperswithcode.com/paper/in-defense-of-product-quantization |
| Repo | |
| Framework | |
A Deep Learning Approach for Expert Identification in Question Answering Communities
| Title | A Deep Learning Approach for Expert Identification in Question Answering Communities |
| Authors | Chen Zheng, Shuangfei Zhai, Zhongfei Zhang |
| Abstract | In this paper, we describe an effective convolutional neural network framework for identifying the expert in question answering community. This approach uses the convolutional neural network and combines user feature representations with question feature representations to compute scores that the user who gets the highest score is the expert on this question. Unlike prior work, this method does not measure expert based on measure answer content quality to identify the expert but only require question sentence and user embedding feature to identify the expert. Remarkably, Our model can be applied to different languages and different domains. The proposed framework is trained on two datasets, The first dataset is Stack Overflow and the second one is Zhihu. The Top-1 accuracy results of our experiments show that our framework outperforms the best baseline framework for expert identification. |
| Tasks | Question Answering |
| Published | 2017-11-14 |
| URL | http://arxiv.org/abs/1711.05350v1 |
| http://arxiv.org/pdf/1711.05350v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-deep-learning-approach-for-expert |
| Repo | |
| Framework | |
Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry
| Title | Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry |
| Authors | Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa |
| Abstract | We provide a comprehensive study of the convergence of forward-backward algorithm under suitable geometric conditions leading to fast rates. We present several new results and collect in a unified view a variety of results scattered in the literature, often providing simplified proofs. Novel contributions include the analysis of infinite dimensional convex minimization problems, allowing the case where minimizers might not exist. Further, we analyze the relation between different geometric conditions, and discuss novel connections with a priori conditions in linear inverse problems, including source conditions, restricted isometry properties and partial smoothness. |
| Tasks | |
| Published | 2017-03-28 |
| URL | http://arxiv.org/abs/1703.09477v3 |
| http://arxiv.org/pdf/1703.09477v3.pdf | |
| PWC | https://paperswithcode.com/paper/convergence-of-the-forward-backward-algorithm |
| Repo | |
| Framework | |
Fast initial guess estimation for digital image correlation
| Title | Fast initial guess estimation for digital image correlation |
| Authors | Peihan Tu |
| Abstract | Digital image correlation (DIC) is a widely used optical metrology for quantitative deformation measurement due to its non-contact, low-cost, highly precise feature. DIC relies on nonlinear optimization algorithm. Thus it is quite important to efficiently obtain a reliable initial guess. The most widely used method for obtaining initial guess is reliability-guided digital image correlation (RG-DIC) method, which is reliable but path-dependent. This path-dependent method limits the further improvement of computation speed of DIC using parallel computing technology, and error of calculation may be spread out along the calculation path. Therefore, a reliable and path-independent algorithm which is able to provide reliable initial guess is desirable to reach full potential of the ability of parallel computing. In this paper, an algorithm used for initial guess estimation is proposed. Numerical and real experiments show that the proposed algorithm, adaptive incremental dissimilarity approximations algorithm (A-IDA), has the following characteristics: 1) Compared with inverse compositional Gauss-Newton (IC-GN) sub-pixel registration algorithm, the computational time required by A-IDA algorithm is negligible, especially when subset size is relatively large; 2) the efficiency of A-IDA algorithm is less influenced by search range; 3) the efficiency is less influenced by subset size; 4) it is easy to select the threshold for the proposed algorithm. |
| Tasks | |
| Published | 2017-10-12 |
| URL | https://arxiv.org/abs/1710.04359v2 |
| https://arxiv.org/pdf/1710.04359v2.pdf | |
| PWC | https://paperswithcode.com/paper/fast-initial-guess-estimation-for-digital |
| Repo | |
| Framework | |
Model-Based Clustering of Time-Evolving Networks through Temporal Exponential-Family Random Graph Models
| Title | Model-Based Clustering of Time-Evolving Networks through Temporal Exponential-Family Random Graph Models |
| Authors | Kevin H. Lee, Lingzhou Xue, David R. Hunter |
| Abstract | Dynamic networks are a general language for describing time-evolving complex systems, and discrete time network models provide an emerging statistical technique for various applications. It is a fundamental research question to detect the community structure in time-evolving networks. However, due to significant computational challenges and difficulties in modeling communities of time-evolving networks, there is little progress in the current literature to effectively find communities in time-evolving networks. In this work, we propose a novel model-based clustering framework for time-evolving networks based on discrete time exponential-family random graph models. To choose the number of communities, we use conditional likelihood to construct an effective model selection criterion. Furthermore, we propose an efficient variational expectation-maximization (EM) algorithm to find approximate maximum likelihood estimates of network parameters and mixing proportions. By using variational methods and minorization-maximization (MM) techniques, our method has appealing scalability for large-scale time-evolving networks. The power of our method is demonstrated in simulation studies and empirical applications to international trade networks and the collaboration networks of a large American research university. |
| Tasks | Model Selection |
| Published | 2017-12-20 |
| URL | http://arxiv.org/abs/1712.07325v1 |
| http://arxiv.org/pdf/1712.07325v1.pdf | |
| PWC | https://paperswithcode.com/paper/model-based-clustering-of-time-evolving |
| Repo | |
| Framework | |
