Paper Group ANR 458
Regression-aware decompositions. Learning Detection with Diverse Proposals. Unbiasing Truncated Backpropagation Through Time. Causality Refined Diagnostic Prediction. Planecell: Representing the 3D Space with Planes. Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models. Hemingway: Modeling Distributed Optimization Algorithms …
Regression-aware decompositions
| Title | Regression-aware decompositions |
| Authors | Mark Tygert |
| Abstract | Linear least-squares regression with a “design” matrix A approximates a given matrix B via minimization of the spectral- or Frobenius-norm discrepancy AX-B over every conformingly sized matrix X. Another popular approximation is low-rank approximation via principal component analysis (PCA) – which is essentially singular value decomposition (SVD) – or interpolative decomposition (ID). Classically, PCA/SVD and ID operate solely with the matrix B being approximated, not supervised by any auxiliary matrix A. However, linear least-squares regression models can inform the ID, yielding regression-aware ID. As a bonus, this provides an interpretation as regression-aware PCA for a kind of canonical correlation analysis between A and B. The regression-aware decompositions effectively enable supervision to inform classical dimensionality reduction, which classically has been totally unsupervised. The regression-aware decompositions reveal the structure inherent in B that is relevant to regression against A. |
| Tasks | Dimensionality Reduction |
| Published | 2017-10-11 |
| URL | http://arxiv.org/abs/1710.04238v2 |
| http://arxiv.org/pdf/1710.04238v2.pdf | |
| PWC | https://paperswithcode.com/paper/regression-aware-decompositions |
| Repo | |
| Framework | |
Learning Detection with Diverse Proposals
| Title | Learning Detection with Diverse Proposals |
| Authors | Samaneh Azadi, Jiashi Feng, Trevor Darrell |
| Abstract | To predict a set of diverse and informative proposals with enriched representations, this paper introduces a differentiable Determinantal Point Process (DPP) layer that is able to augment the object detection architectures. Most modern object detection architectures, such as Faster R-CNN, learn to localize objects by minimizing deviations from the ground-truth but ignore correlation between multiple proposals and object categories. Non-Maximum Suppression (NMS) as a widely used proposal pruning scheme ignores label- and instance-level relations between object candidates resulting in multi-labeled detections. In the multi-class case, NMS selects boxes with the largest prediction scores ignoring the semantic relation between categories of potential election. In contrast, our trainable DPP layer, allowing for Learning Detection with Diverse Proposals (LDDP), considers both label-level contextual information and spatial layout relationships between proposals without increasing the number of parameters of the network, and thus improves location and category specifications of final detected bounding boxes substantially during both training and inference schemes. Furthermore, we show that LDDP keeps it superiority over Faster R-CNN even if the number of proposals generated by LDPP is only ~30% as many as those for Faster R-CNN. |
| Tasks | Object Detection |
| Published | 2017-04-11 |
| URL | http://arxiv.org/abs/1704.03533v1 |
| http://arxiv.org/pdf/1704.03533v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-detection-with-diverse-proposals |
| Repo | |
| Framework | |
Unbiasing Truncated Backpropagation Through Time
| Title | Unbiasing Truncated Backpropagation Through Time |
| Authors | Corentin Tallec, Yann Ollivier |
| Abstract | Truncated Backpropagation Through Time (truncated BPTT) is a widespread method for learning recurrent computational graphs. Truncated BPTT keeps the computational benefits of Backpropagation Through Time (BPTT) while relieving the need for a complete backtrack through the whole data sequence at every step. However, truncation favors short-term dependencies: the gradient estimate of truncated BPTT is biased, so that it does not benefit from the convergence guarantees from stochastic gradient theory. We introduce Anticipated Reweighted Truncated Backpropagation (ARTBP), an algorithm that keeps the computational benefits of truncated BPTT, while providing unbiasedness. ARTBP works by using variable truncation lengths together with carefully chosen compensation factors in the backpropagation equation. We check the viability of ARTBP on two tasks. First, a simple synthetic task where careful balancing of temporal dependencies at different scales is needed: truncated BPTT displays unreliable performance, and in worst case scenarios, divergence, while ARTBP converges reliably. Second, on Penn Treebank character-level language modelling, ARTBP slightly outperforms truncated BPTT. |
| Tasks | Language Modelling |
| Published | 2017-05-23 |
| URL | http://arxiv.org/abs/1705.08209v1 |
| http://arxiv.org/pdf/1705.08209v1.pdf | |
| PWC | https://paperswithcode.com/paper/unbiasing-truncated-backpropagation-through |
| Repo | |
| Framework | |
Causality Refined Diagnostic Prediction
| Title | Causality Refined Diagnostic Prediction |
| Authors | Marcus Klasson, Kun Zhang, Bo C. Bertilson, Cheng Zhang, Hedvig Kjellström |
| Abstract | Applying machine learning in the health care domain has shown promising results in recent years. Interpretable outputs from learning algorithms are desirable for decision making by health care personnel. In this work, we explore the possibility of utilizing causal relationships to refine diagnostic prediction. We focus on the task of diagnostic prediction using discomfort drawings, and explore two ways to employ causal identification to improve the diagnostic results. Firstly, we use causal identification to infer the causal relationships among diagnostic labels which, by itself, provides interpretable results to aid the decision making and training of health care personnel. Secondly, we suggest a post-processing approach where the inferred causal relationships are used to refine the prediction accuracy of a multi-view probabilistic model. Experimental results show firstly that causal identification is capable of detecting the causal relationships among diagnostic labels correctly, and secondly that there is potential for improving pain diagnostics prediction accuracy using the causal relationships. |
| Tasks | Causal Identification, Decision Making |
| Published | 2017-11-29 |
| URL | http://arxiv.org/abs/1711.10915v1 |
| http://arxiv.org/pdf/1711.10915v1.pdf | |
| PWC | https://paperswithcode.com/paper/causality-refined-diagnostic-prediction |
| Repo | |
| Framework | |
Planecell: Representing the 3D Space with Planes
| Title | Planecell: Representing the 3D Space with Planes |
| Authors | Lei Fan, Ziyu Pan, Long Chen, Kai Huang |
| Abstract | Reconstruction based on the stereo camera has received considerable attention recently, but two particular challenges still remain. The first concerns the need to aggregate similar pixels in an effective approach, and the second is to maintain as much of the available information as possible while ensuring sufficient accuracy. To overcome these issues, we propose a new 3D representation method, namely, planecell, that extracts planarity from the depth-assisted image segmentation and then projects these depth planes into the 3D world. An energy function formulated from Conditional Random Field that generalizes the planar relationships is maximized to merge coplanar segments. We evaluate our method with a variety of reconstruction baselines on both KITTI and Middlebury datasets, and the results indicate the superiorities compared to other 3D space representation methods in accuracy, memory requirements and further applications. |
| Tasks | Semantic Segmentation |
| Published | 2017-03-30 |
| URL | http://arxiv.org/abs/1703.10304v1 |
| http://arxiv.org/pdf/1703.10304v1.pdf | |
| PWC | https://paperswithcode.com/paper/planecell-representing-the-3d-space-with |
| Repo | |
| Framework | |
Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models
| Title | Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models |
| Authors | Yanzhang He, Rohit Prabhavalkar, Kanishka Rao, Wei Li, Anton Bakhtin, Ian McGraw |
| Abstract | We develop streaming keyword spotting systems using a recurrent neural network transducer (RNN-T) model: an all-neural, end-to-end trained, sequence-to-sequence model which jointly learns acoustic and language model components. Our models are trained to predict either phonemes or graphemes as subword units, thus allowing us to detect arbitrary keyword phrases, without any out-of-vocabulary words. In order to adapt the models to the requirements of keyword spotting, we propose a novel technique which biases the RNN-T system towards a specific keyword of interest. Our systems are compared against a strong sequence-trained, connectionist temporal classification (CTC) based “keyword-filler” baseline, which is augmented with a separate phoneme language model. Overall, our RNN-T system with the proposed biasing technique significantly improves performance over the baseline system. |
| Tasks | Keyword Spotting, Language Modelling, Small-Footprint Keyword Spotting |
| Published | 2017-10-26 |
| URL | http://arxiv.org/abs/1710.09617v1 |
| http://arxiv.org/pdf/1710.09617v1.pdf | |
| PWC | https://paperswithcode.com/paper/streaming-small-footprint-keyword-spotting |
| Repo | |
| Framework | |
Hemingway: Modeling Distributed Optimization Algorithms
| Title | Hemingway: Modeling Distributed Optimization Algorithms |
| Authors | Xinghao Pan, Shivaram Venkataraman, Zizheng Tai, Joseph Gonzalez |
| Abstract | Distributed optimization algorithms are widely used in many industrial machine learning applications. However choosing the appropriate algorithm and cluster size is often difficult for users as the performance and convergence rate of optimization algorithms vary with the size of the cluster. In this paper we make the case for an ML-optimizer that can select the appropriate algorithm and cluster size to use for a given problem. To do this we propose building two models: one that captures the system level characteristics of how computation, communication change as we increase cluster sizes and another that captures how convergence rates change with cluster sizes. We present preliminary results from our prototype implementation called Hemingway and discuss some of the challenges involved in developing such a system. |
| Tasks | Distributed Optimization |
| Published | 2017-02-20 |
| URL | http://arxiv.org/abs/1702.05865v1 |
| http://arxiv.org/pdf/1702.05865v1.pdf | |
| PWC | https://paperswithcode.com/paper/hemingway-modeling-distributed-optimization |
| Repo | |
| Framework | |
Exploring the Use of Text Classification in the Legal Domain
| Title | Exploring the Use of Text Classification in the Legal Domain |
| Authors | Octavia-Maria Sulea, Marcos Zampieri, Shervin Malmasi, Mihaela Vela, Liviu P. Dinu, Josef van Genabith |
| Abstract | In this paper, we investigate the application of text classification methods to support law professionals. We present several experiments applying machine learning techniques to predict with high accuracy the ruling of the French Supreme Court and the law area to which a case belongs to. We also investigate the influence of the time period in which a ruling was made on the form of the case description and the extent to which we need to mask information in a full case ruling to automatically obtain training and test data that resembles case descriptions. We developed a mean probability ensemble system combining the output of multiple SVM classifiers. We report results of 98% average F1 score in predicting a case ruling, 96% F1 score for predicting the law area of a case, and 87.07% F1 score on estimating the date of a ruling. |
| Tasks | Text Classification |
| Published | 2017-10-25 |
| URL | http://arxiv.org/abs/1710.09306v1 |
| http://arxiv.org/pdf/1710.09306v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-the-use-of-text-classification-in |
| Repo | |
| Framework | |
Sales Forecast in E-commerce using Convolutional Neural Network
| Title | Sales Forecast in E-commerce using Convolutional Neural Network |
| Authors | Kui Zhao, Can Wang |
| Abstract | Sales forecast is an essential task in E-commerce and has a crucial impact on making informed business decisions. It can help us to manage the workforce, cash flow and resources such as optimizing the supply chain of manufacturers etc. Sales forecast is a challenging problem in that sales is affected by many factors including promotion activities, price changes, and user preferences etc. Traditional sales forecast techniques mainly rely on historical sales data to predict future sales and their accuracies are limited. Some more recent learning-based methods capture more information in the model to improve the forecast accuracy. However, these methods require case-by-case manual feature engineering for specific commercial scenarios, which is usually a difficult, time-consuming task and requires expert knowledge. To overcome the limitations of existing methods, we propose a novel approach in this paper to learn effective features automatically from the structured data using the Convolutional Neural Network (CNN). When fed with raw log data, our approach can automatically extract effective features from that and then forecast sales using those extracted features. We test our method on a large real-world dataset from CaiNiao.com and the experimental results validate the effectiveness of our method. |
| Tasks | Feature Engineering |
| Published | 2017-08-26 |
| URL | http://arxiv.org/abs/1708.07946v1 |
| http://arxiv.org/pdf/1708.07946v1.pdf | |
| PWC | https://paperswithcode.com/paper/sales-forecast-in-e-commerce-using |
| Repo | |
| Framework | |
Fundamental Limits of Weak Recovery with Applications to Phase Retrieval
| Title | Fundamental Limits of Weak Recovery with Applications to Phase Retrieval |
| Authors | Marco Mondelli, Andrea Montanari |
| Abstract | In phase retrieval we want to recover an unknown signal $\boldsymbol x\in\mathbb C^d$ from $n$ quadratic measurements of the form $y_i = \langle{\boldsymbol a}_i,{\boldsymbol x}\rangle^2+w_i$ where $\boldsymbol a_i\in \mathbb C^d$ are known sensing vectors and $w_i$ is measurement noise. We ask the following weak recovery question: what is the minimum number of measurements $n$ needed to produce an estimator $\hat{\boldsymbol x}(\boldsymbol y)$ that is positively correlated with the signal $\boldsymbol x$? We consider the case of Gaussian vectors $\boldsymbol a_i$. We prove that - in the high-dimensional limit - a sharp phase transition takes place, and we locate the threshold in the regime of vanishingly small noise. For $n\le d-o(d)$ no estimator can do significantly better than random and achieve a strictly positive correlation. For $n\ge d+o(d)$ a simple spectral estimator achieves a positive correlation. Surprisingly, numerical simulations with the same spectral estimator demonstrate promising performance with realistic sensing matrices. Spectral methods are used to initialize non-convex optimization algorithms in phase retrieval, and our approach can boost the performance in this setting as well. Our impossibility result is based on classical information-theory arguments. The spectral algorithm computes the leading eigenvector of a weighted empirical covariance matrix. We obtain a sharp characterization of the spectral properties of this random matrix using tools from free probability and generalizing a recent result by Lu and Li. Both the upper and lower bound generalize beyond phase retrieval to measurements $y_i$ produced according to a generalized linear model. As a byproduct of our analysis, we compare the threshold of the proposed spectral method with that of a message passing algorithm. |
| Tasks | |
| Published | 2017-08-20 |
| URL | http://arxiv.org/abs/1708.05932v3 |
| http://arxiv.org/pdf/1708.05932v3.pdf | |
| PWC | https://paperswithcode.com/paper/fundamental-limits-of-weak-recovery-with |
| Repo | |
| Framework | |
Modular Representation of Layered Neural Networks
| Title | Modular Representation of Layered Neural Networks |
| Authors | Chihiro Watanabe, Kaoru Hiramatsu, Kunio Kashino |
| Abstract | Layered neural networks have greatly improved the performance of various applications including image processing, speech recognition, natural language processing, and bioinformatics. However, it is still difficult to discover or interpret knowledge from the inference provided by a layered neural network, since its internal representation has many nonlinear and complex parameters embedded in hierarchical layers. Therefore, it becomes important to establish a new methodology by which layered neural networks can be understood. In this paper, we propose a new method for extracting a global and simplified structure from a layered neural network. Based on network analysis, the proposed method detects communities or clusters of units with similar connection patterns. We show its effectiveness by applying it to three use cases. (1) Network decomposition: it can decompose a trained neural network into multiple small independent networks thus dividing the problem and reducing the computation time. (2) Training assessment: the appropriateness of a trained result with a given hyperparameter or randomly chosen initial parameters can be evaluated by using a modularity index. And (3) data analysis: in practical data it reveals the community structure in the input, hidden, and output layers, which serves as a clue for discovering knowledge from a trained neural network. |
| Tasks | Speech Recognition |
| Published | 2017-03-01 |
| URL | http://arxiv.org/abs/1703.00168v2 |
| http://arxiv.org/pdf/1703.00168v2.pdf | |
| PWC | https://paperswithcode.com/paper/modular-representation-of-layered-neural |
| Repo | |
| Framework | |
Historic Emergence of Diversity in Painting: Heterogeneity in Chromatic Distance in Images and Characterization of Massive Painting Data Set
| Title | Historic Emergence of Diversity in Painting: Heterogeneity in Chromatic Distance in Images and Characterization of Massive Painting Data Set |
| Authors | Byunghwee Lee, Daniel Kim, Seunghye Sun, Hawoong Jeong, Juyong Park |
| Abstract | Painting is an art form that has long functioned as a major channel for the creative expression and communication of humans, its evolution taking place under an interplay with the science, technology, and social environments of the times. Therefore, understanding the process based on comprehensive data could shed light on how humans acted and manifested creatively under changing conditions. Yet, there exist few systematic frameworks that characterize the process for painting, which would require robust statistical methods for defining painting characteristics and identifying human’s creative developments, and data of high quality and sufficient quantity. Here we propose that the color contrast of a painting image signifying the heterogeneity in inter-pixel chromatic distance can be a useful representation of its style, integrating both the color and geometry. From the color contrasts of paintings from a large-scale, comprehensive archive of 179,853 high-quality images spanning several centuries we characterize the temporal evolutionary patterns of paintings, and present a deep study of an extraordinary expansion in creative diversity and individuality that came to define the modern era. |
| Tasks | |
| Published | 2017-01-25 |
| URL | http://arxiv.org/abs/1701.07164v2 |
| http://arxiv.org/pdf/1701.07164v2.pdf | |
| PWC | https://paperswithcode.com/paper/historic-emergence-of-diversity-in-painting |
| Repo | |
| Framework | |
Embodied Evolution in Collective Robotics: A Review
| Title | Embodied Evolution in Collective Robotics: A Review |
| Authors | Nicolas Bredeche, Evert Haasdijk, Abraham Prieto |
| Abstract | This paper provides an overview of evolutionary robotics techniques applied to on-line distributed evolution for robot collectives – namely, embodied evolution. It provides a definition of embodied evolution as well as a thorough description of the underlying concepts and mechanisms. The paper also presents a comprehensive summary of research published in the field since its inception (1999-2017), providing various perspectives to identify the major trends. In particular, we identify a shift from considering embodied evolution as a parallel search method within small robot collectives (fewer than 10 robots) to embodied evolution as an on-line distributed learning method for designing collective behaviours in swarm-like collectives. The paper concludes with a discussion of applications and open questions, providing a milestone for past and an inspiration for future research. |
| Tasks | |
| Published | 2017-09-26 |
| URL | http://arxiv.org/abs/1709.08992v2 |
| http://arxiv.org/pdf/1709.08992v2.pdf | |
| PWC | https://paperswithcode.com/paper/embodied-evolution-in-collective-robotics-a |
| Repo | |
| Framework | |
Bridge Simulation and Metric Estimation on Landmark Manifolds
| Title | Bridge Simulation and Metric Estimation on Landmark Manifolds |
| Authors | Stefan Sommer, Alexis Arnaudon, Line Kuhnel, Sarang Joshi |
| Abstract | We present an inference algorithm and connected Monte Carlo based estimation procedures for metric estimation from landmark configurations distributed according to the transition distribution of a Riemannian Brownian motion arising from the Large Deformation Diffeomorphic Metric Mapping (LDDMM) metric. The distribution possesses properties similar to the regular Euclidean normal distribution but its transition density is governed by a high-dimensional PDE with no closed-form solution in the nonlinear case. We show how the density can be numerically approximated by Monte Carlo sampling of conditioned Brownian bridges, and we use this to estimate parameters of the LDDMM kernel and thus the metric structure by maximum likelihood. |
| Tasks | |
| Published | 2017-05-31 |
| URL | http://arxiv.org/abs/1705.10943v1 |
| http://arxiv.org/pdf/1705.10943v1.pdf | |
| PWC | https://paperswithcode.com/paper/bridge-simulation-and-metric-estimation-on |
| Repo | |
| Framework | |
Joint RNN Model for Argument Component Boundary Detection
| Title | Joint RNN Model for Argument Component Boundary Detection |
| Authors | Minglan Li, Yang Gao, Hui Wen, Yang Du, Haijing Liu, Hao Wang |
| Abstract | Argument Component Boundary Detection (ACBD) is an important sub-task in argumentation mining; it aims at identifying the word sequences that constitute argument components, and is usually considered as the first sub-task in the argumentation mining pipeline. Existing ACBD methods heavily depend on task-specific knowledge, and require considerable human efforts on feature-engineering. To tackle these problems, in this work, we formulate ACBD as a sequence labeling problem and propose a variety of Recurrent Neural Network (RNN) based methods, which do not use domain specific or handcrafted features beyond the relative position of the sentence in the document. In particular, we propose a novel joint RNN model that can predict whether sentences are argumentative or not, and use the predicted results to more precisely detect the argument component boundaries. We evaluate our techniques on two corpora from two different genres; results suggest that our joint RNN model obtain the state-of-the-art performance on both datasets. |
| Tasks | Boundary Detection, Feature Engineering |
| Published | 2017-05-05 |
| URL | http://arxiv.org/abs/1705.02131v1 |
| http://arxiv.org/pdf/1705.02131v1.pdf | |
| PWC | https://paperswithcode.com/paper/joint-rnn-model-for-argument-component |
| Repo | |
| Framework | |
