January 29, 2020

3464 words 17 mins read

Paper Group ANR 589

CLN2INV: Learning Loop Invariants with Continuous Logic Networks. Outlier Detection for Improved Data Quality and Diversity in Dialog Systems. Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model. New Potential-Based Bounds for the Geometric-Stopping Version of Prediction with Expert Advice. Learning to design from humans …

CLN2INV: Learning Loop Invariants with Continuous Logic Networks


Title	CLN2INV: Learning Loop Invariants with Continuous Logic Networks
Authors	Gabriel Ryan, Justin Wong, Jianan Yao, Ronghui Gu, Suman Jana
Abstract	Program verification offers a framework for ensuring program correctness and therefore systematically eliminating different classes of bugs. Inferring loop invariants is one of the main challenges behind automated verification of real-world programs which often contain many loops. In this paper, we present Continuous Logic Network (CLN), a novel neural architecture for automatically learning loop invariants directly from program execution traces. Unlike existing neural networks, CLNs can learn precise and explicit representations of formulas in Satisfiability Modulo Theories (SMT) for loop invariants from program execution traces. We develop a new sound and complete semantic mapping for assigning SMT formulas to continuous truth values that allows CLNs to be trained efficiently. We use CLNs to implement a new inference system for loop invariants, CLN2INV, that significantly outperforms existing approaches on the popular Code2Inv dataset. CLN2INV is the first tool to solve all 124 theoretically solvable problems in the Code2Inv dataset. Moreover, CLN2INV takes only 1.1 seconds on average for each problem, which is 40 times faster than existing approaches. We further demonstrate that CLN2INV can even learn 12 significantly more complex loop invariants than the ones required for the Code2Inv dataset.
Tasks
Published	2019-09-25
URL	https://arxiv.org/abs/1909.11542v3
PDF	https://arxiv.org/pdf/1909.11542v3.pdf
PWC	https://paperswithcode.com/paper/cln2inv-learning-loop-invariants-with
Repo
Framework

Outlier Detection for Improved Data Quality and Diversity in Dialog Systems


Title	Outlier Detection for Improved Data Quality and Diversity in Dialog Systems
Authors	Stefan Larson, Anish Mahendran, Andrew Lee, Jonathan K. Kummerfeld, Parker Hill, Michael A. Laurenzano, Johann Hauswald, Lingjia Tang, Jason Mars
Abstract	In a corpus of data, outliers are either errors: mistakes in the data that are counterproductive, or are unique: informative samples that improve model robustness. Identifying outliers can lead to better datasets by (1) removing noise in datasets and (2) guiding collection of additional data to fill gaps. However, the problem of detecting both outlier types has received relatively little attention in NLP, particularly for dialog systems. We introduce a simple and effective technique for detecting both erroneous and unique samples in a corpus of short texts using neural sentence embeddings combined with distance-based outlier detection. We also present a novel data collection pipeline built atop our detection technique to automatically and iteratively mine unique data samples while discarding erroneous samples. Experiments show that our outlier detection technique is effective at finding errors while our data collection pipeline yields highly diverse corpora that in turn produce more robust intent classification and slot-filling models.
Tasks	Intent Classification, Outlier Detection, Sentence Embeddings, Slot Filling
Published	2019-04-05
URL	http://arxiv.org/abs/1904.03122v1
PDF	http://arxiv.org/pdf/1904.03122v1.pdf
PWC	https://paperswithcode.com/paper/outlier-detection-for-improved-data-quality
Repo
Framework

Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model


Title	Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
Authors	Wei Li, Jingjing Xu, Yancheng He, Shengli Yan, Yunfang Wu, Xu sun
Abstract	Automatic article commenting is helpful in encouraging user engagement and interaction on online news platforms. However, the news documents are usually too long for traditional encoder-decoder based models, which often results in general and irrelevant comments. In this paper, we propose to generate comments with a graph-to-sequence model that models the input news as a topic interaction graph. By organizing the article into graph structure, our model can better understand the internal structure of the article and the connection between topics, which makes it better able to understand the story. We collect and release a large scale news-comment corpus from a popular Chinese online news platform Tencent Kuaibao. Extensive experiment results show that our model can generate much more coherent and informative comments compared with several strong baseline models.
Tasks	Graph-to-Sequence
Published	2019-06-04
URL	https://arxiv.org/abs/1906.01231v1
PDF	https://arxiv.org/pdf/1906.01231v1.pdf
PWC	https://paperswithcode.com/paper/coherent-comment-generation-for-chinese
Repo
Framework

New Potential-Based Bounds for the Geometric-Stopping Version of Prediction with Expert Advice


Title	New Potential-Based Bounds for the Geometric-Stopping Version of Prediction with Expert Advice
Authors	Vladimir A. Kobzar, Robert V. Kohn, Zhilei Wang
Abstract	This work addresses the classic machine learning problem of online prediction with expert advice. A potential-based framework for the fixed horizon version of this problem was previously developed using verification arguments from optimal control theory (Kobzar, Kohn and Wang, New Potential-Based Bounds for Prediction with Expert Advice (2019)). This paper extends this framework to the random (geometric) stopping version. Taking advantage of these ideas, we construct potentials for the geometric version of prediction with expert advice from potentials used for the fixed horizon version. This construction leads to new explicit lower and upper bounds associated with specific adversary and player strategies for the geometric problem. We identify regimes where these bounds are state of the art.
Tasks
Published	2019-12-05
URL	https://arxiv.org/abs/1912.03132v1
PDF	https://arxiv.org/pdf/1912.03132v1.pdf
PWC	https://paperswithcode.com/paper/new-potential-based-bounds-for-the-geometric
Repo
Framework

Learning to design from humans: Imitating human designers through deep learning


Title	Learning to design from humans: Imitating human designers through deep learning
Authors	Ayush Raina, Christopher McComb, Jonathan Cagan
Abstract	Humans as designers have quite versatile problem-solving strategies. Computer agents on the other hand can access large scale computational resources to solve certain design problems. Hence, if agents can learn from human behavior, a synergetic human-agent problem solving team can be created. This paper presents an approach to extract human design strategies and implicit rules, purely from historical human data, and use that for design generation. A two-step framework that learns to imitate human design strategies from observation is proposed and implemented. This framework makes use of deep learning constructs to learn to generate designs without any explicit information about objective and performance metrics. The framework is designed to interact with the problem through a visual interface as humans did when solving the problem. It is trained to imitate a set of human designers by observing their design state sequences without inducing problem-specific modelling bias or extra information about the problem. Furthermore, an end-to-end agent is developed that uses this deep learning framework as its core in conjunction with image processing to map pixel-to-design moves as a mechanism to generate designs. Finally, the designs generated by a computational team of these agents are then compared to actual human data for teams solving a truss design problem. Results demonstrates that these agents are able to create feasible and efficient truss designs without guidance, showing that this methodology allows agents to learn effective design strategies.
Tasks
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11813v2
PDF	https://arxiv.org/pdf/1907.11813v2.pdf
PWC	https://paperswithcode.com/paper/learning-to-design-from-humans-imitating
Repo
Framework

StarNet: Targeted Computation for Object Detection in Point Clouds


Title	StarNet: Targeted Computation for Object Detection in Point Clouds
Authors	Jiquan Ngiam, Benjamin Caine, Wei Han, Brandon Yang, Yuning Chai, Pei Sun, Yin Zhou, Xi Yi, Ouais Alsharif, Patrick Nguyen, Zhifeng Chen, Jonathon Shlens, Vijay Vasudevan
Abstract	Detecting objects from LiDAR point clouds is an important component of self-driving car technology as LiDAR provides high resolution spatial information. Previous work on point-cloud 3D object detection has re-purposed convolutional approaches from traditional camera imagery. In this work, we present an object detection system called StarNet designed specifically to take advantage of the sparse and 3D nature of point cloud data. StarNet is entirely point-based, uses no global information, has data dependent anchors, and uses sampling instead of learned region proposals. We demonstrate how this design leads to competitive or superior performance on the large Waymo Open Dataset and the KITTI detection dataset, as compared to convolutional baselines. In particular, we show how our detector can outperform a competitive baseline on Pedestrian detection on the Waymo Open Dataset by more than 7 absolute mAP while being more computationally efficient. We show how our redesign—namely using only local information and using sampling instead of learned proposals—leads to a significantly more flexible and adaptable system: we demonstrate how we can vary the computational cost of a single trained StarNet without retraining, and how we can target proposals towards areas of interest with priors and heuristics. Finally, we show how our design allows for incorporating temporal context by using detections from previous frames to target computation of the detector, which leads to further improvements in performance without additional computational cost.
Tasks	3D Object Detection, Object Detection, Pedestrian Detection, Self-Driving Cars
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11069v3
PDF	https://arxiv.org/pdf/1908.11069v3.pdf
PWC	https://paperswithcode.com/paper/starnet-targeted-computation-for-object
Repo
Framework

Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation


Title	Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation
Authors	Clebeson Canuto dos Santos, Jorge Leonid Aching Samatelo, Raquel Frizera Vassallo
Abstract	Due to the advance of technologies, machines are increasingly present in people’s daily lives. Thus, there has been more and more effort to develop interfaces, such as dynamic gestures, that provide an intuitive way of interaction. Currently, the most common trend is to use multimodal data, as depth and skeleton information, to enable dynamic gesture recognition. However, using only color information would be more interesting, since RGB cameras are usually available in almost every public place, and could be used for gesture recognition without the need of installing other equipment. The main problem with such approach is the difficulty of representing spatio-temporal information using just color. With this in mind, we propose a technique capable of condensing a dynamic gesture, shown in a video, in just one RGB image. We call this technique star RGB. This image is then passed to a classifier formed by two Resnet CNNs, a soft-attention ensemble, and a fully connected layer, which indicates the class of the gesture present in the input video. Experiments were carried out using both Montalbano and GRIT datasets. For Montalbano dataset, the proposed approach achieved an accuracy of 94.58%. Such result reaches the state-of-the-art when considering this dataset and only color information. Regarding the GRIT dataset, our proposal achieves more than 98% of accuracy, recall, precision, and F1-score, outperforming the reference approach by more than 6%.
Tasks	Gesture Recognition
Published	2019-04-10
URL	https://arxiv.org/abs/1904.08505v2
PDF	https://arxiv.org/pdf/1904.08505v2.pdf
PWC	https://paperswithcode.com/paper/190408505
Repo
Framework

Stochastic data-driven model predictive control using Gaussian processes


Title	Stochastic data-driven model predictive control using Gaussian processes
Authors	Eric Bradford, Lars Imsland, Dongda Zhang, Ehecatl Antonio del Rio Chanona
Abstract	Nonlinear model predictive control (NMPC) is one of the few control methods that can handle multivariable nonlinear control systems with constraints. Gaussian processes (GPs) present a powerful tool to identify the required plant model and quantify the residual uncertainty of the plant-model mismatch given its probabilistic nature . It is crucial to account for this uncertainty, since it may lead to worse control performance and constraint violations. In this paper we propose a new method to design a GP-based NMPC algorithm for finite horizon control problems. The method generates Monte Carlo samples of the GP offline for constraint tightening using back-offs. The tightened constraints then guarantee the satisfaction of joint chance constraints online. Advantages of our proposed approach over existing methods include fast online evaluation time, consideration of closed-loop behaviour, and the possibility to alleviate conservativeness by accounting for both online learning and state dependency of the uncertainty. The algorithm is verified on a challenging semi-batch bioprocess case study, with its high performance thoroughly demonstrated.
Tasks	Gaussian Processes
Published	2019-08-05
URL	https://arxiv.org/abs/1908.01786v1
PDF	https://arxiv.org/pdf/1908.01786v1.pdf
PWC	https://paperswithcode.com/paper/stochastic-data-driven-model-predictive
Repo
Framework

Interventional Experiment Design for Causal Structure Learning


Title	Interventional Experiment Design for Causal Structure Learning
Authors	AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash
Abstract	It is known that from purely observational data, a causal DAG is identifiable only up to its Markov equivalence class, and for many ground truth DAGs, the direction of a large portion of the edges will be remained unidentified. The golden standard for learning the causal DAG beyond Markov equivalence is to perform a sequence of interventions in the system and use the data gathered from the interventional distributions. We consider a setup in which given a budget $k$, we design $k$ interventions non-adaptively. We cast the problem of finding the best intervention target set as an optimization problem which aims to maximize the number of edges whose directions are identified due to the performed interventions. First, we consider the case that the underlying causal structure is a tree. For this case, we propose an efficient exact algorithm for the worst-case gain setup, as well as an approximate algorithm for the average gain setup. We then show that the proposed approach for the average gain setup can be extended to the case of general causal structures. In this case, besides the design of interventions, calculating the objective function is also challenging. We propose an efficient exact calculator as well as two estimators for this task. We evaluate the proposed methods using synthetic as well as real data.
Tasks
Published	2019-10-12
URL	https://arxiv.org/abs/1910.05651v1
PDF	https://arxiv.org/pdf/1910.05651v1.pdf
PWC	https://paperswithcode.com/paper/interventional-experiment-design-for-causal
Repo
Framework

A Stereo Algorithm for Thin Obstacles and Reflective Objects


Title	A Stereo Algorithm for Thin Obstacles and Reflective Objects
Authors	John Keller, Sebastian Scherer
Abstract	Stereo cameras are a popular choice for obstacle avoidance for outdoor lighweight, low-cost robotics applications. However, they are unable to sense thin and reflective objects well. Currently, many algorithms are tuned to perform well on indoor scenes like the Middlebury dataset. When navigating outdoors, reflective objects, like windows and glass, and thin obstacles, like wires, are not well handled by most stereo disparity algorithms. Reflections, repeating patterns and objects parallel to the cameras’ baseline causes mismatches between image pairs which leads to bad disparity estimates. Thin obstacles are difficult for many sliding window based disparity methods to detect because they do not take up large portions of the pixels in the sliding window. We use a trinocular camera setup and micropolarizer camera capable of detecting reflective objects to overcome these issues. We present a hierarchical disparity algorithm that reduces noise, separately identify wires using semantic object triangulation in three images, and use information about the polarization of light to estimate the disparity of reflective objects. We evaluate our approach on outdoor data that we collected. Our method contained an average of 9.27% of bad pixels compared to a typical stereo algorithm’s 18.4% of bad pixels in scenes containing reflective objects. Our trinocular and semantic wire disparity methods detected 53% of wire pixels, whereas a typical two camera stereo algorithm detected 5%.
Tasks
Published	2019-10-03
URL	https://arxiv.org/abs/1910.04874v1
PDF	https://arxiv.org/pdf/1910.04874v1.pdf
PWC	https://paperswithcode.com/paper/a-stereo-algorithm-for-thin-obstacles-and
Repo
Framework

Integrated Optimization of Ascent Trajectory and SRM Design of Multistage Launch Vehicles


Title	Integrated Optimization of Ascent Trajectory and SRM Design of Multistage Launch Vehicles
Authors	Lorenzo Federici, Alessandro Zavoli, Guido Colasurdo, Lucandrea Mancini, Agostino Neri
Abstract	This paper presents a methodology for the concurrent first-stage preliminary design and ascent trajectory optimization, with application to a Vega-derived Light Launch Vehicle. The reuse as first stage of an existing upper-stage (Zefiro 40) requires a propellant grain geometry redesign, in order to account for the mutated operating conditions. An optimization code based on the parallel running of several Differential Evolution algorithms is used to find the optimal internal pressure law during Z40 operation, together with the optimal thrust direction and other relevant flight parameters of the entire ascent trajectory. Payload injected into a target orbit is maximized, while respecting multiple design constraints, either involving the alone solid rocket motor or dependent on the actual flight trajectory. Numerical results for SSO injection are presented.
Tasks
Published	2019-10-08
URL	https://arxiv.org/abs/1910.03268v1
PDF	https://arxiv.org/pdf/1910.03268v1.pdf
PWC	https://paperswithcode.com/paper/integrated-optimization-of-ascent-trajectory
Repo
Framework

Neural Packet Classification


Title	Neural Packet Classification
Authors	Eric Liang, Hang Zhu, Xin Jin, Ion Stoica
Abstract	Packet classification is a fundamental problem in computer networking. This problem exposes a hard tradeoff between the computation and state complexity, which makes it particularly challenging. To navigate this tradeoff, existing solutions rely on complex hand-tuned heuristics, which are brittle and hard to optimize. In this paper, we propose a deep reinforcement learning (RL) approach to solve the packet classification problem. There are several characteristics that make this problem a good fit for Deep RL. First, many of the existing solutions are iteratively building a decision tree by splitting nodes in the tree. Second, the effects of these actions (e.g., splitting nodes) can only be evaluated once we are done with building the tree. These two characteristics are naturally captured by the ability of RL to take actions that have sparse and delayed rewards. Third, it is computationally efficient to generate data traces and evaluate decision trees, which alleviate the notoriously high sample complexity problem of Deep RL algorithms. Our solution, NeuroCuts, uses succinct representations to encode state and action space, and efficiently explore candidate decision trees to optimize for a global objective. It produces compact decision trees optimized for a specific set of rules and a given performance metric, such as classification time, memory footprint, or a combination of the two. Evaluation on ClassBench shows that NeuroCuts outperforms existing hand-crafted algorithms in classification time by 18% at the median, and reduces both time and memory footprint by up to 3x.
Tasks
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10319v1
PDF	http://arxiv.org/pdf/1902.10319v1.pdf
PWC	https://paperswithcode.com/paper/neural-packet-classification
Repo
Framework

Moving Target Defense for Deep Visual Sensing against Adversarial Examples


Title	Moving Target Defense for Deep Visual Sensing against Adversarial Examples
Authors	Qun Song, Zhenyu Yan, Rui Tan
Abstract	Deep learning based visual sensing has achieved attractive accuracy but is shown vulnerable to adversarial example attacks. Specifically, once the attackers obtain the deep model, they can construct adversarial examples to mislead the model to yield wrong classification results. Deployable adversarial examples such as small stickers pasted on the road signs and lanes have been shown effective in misleading advanced driver-assistance systems. Many existing countermeasures against adversarial examples build their security on the attackers’ ignorance of the defense mechanisms. Thus, they fall short of following Kerckhoffs’s principle and can be subverted once the attackers know the details of the defense. This paper applies the strategy of moving target defense (MTD) to generate multiple new deep models after system deployment, that will collaboratively detect and thwart adversarial examples. Our MTD design is based on the adversarial examples’ minor transferability to models differing from the one (e.g., the factory-designed model) used for attack construction. The post-deployment quasi-secret deep models significantly increase the bar for the attackers to construct effective adversarial examples. We also apply the technique of serial data fusion with early stopping to reduce the inference time by a factor of up to 5 while maintaining the sensing and defense performance. Extensive evaluation based on three datasets including a road sign image database and a GPU-equipped Jetson embedded computing board shows the effectiveness of our approach.
Tasks
Published	2019-05-11
URL	https://arxiv.org/abs/1905.13148v1
PDF	https://arxiv.org/pdf/1905.13148v1.pdf
PWC	https://paperswithcode.com/paper/190513148
Repo
Framework

Recovering the Lowest Layer of Deep Networks with High Threshold Activations


Title	Recovering the Lowest Layer of Deep Networks with High Threshold Activations
Authors	Surbhi Goel, Rina Panigrahy
Abstract	Giving provable guarantees for learning neural networks is a core challenge of machine learning theory. Most prior work gives parameter recovery guarantees for one hidden layer networks, however, the networks used in practice have multiple non-linear layers. In this work, we show how we can strengthen such results to deeper networks – we address the problem of uncovering the lowest layer in a deep neural network under the assumption that the lowest layer uses a high threshold before applying the activation, the upper network can be modeled as a well-behaved polynomial and the input distribution is Gaussian.
Tasks
Published	2019-03-21
URL	https://arxiv.org/abs/1903.09231v2
PDF	https://arxiv.org/pdf/1903.09231v2.pdf
PWC	https://paperswithcode.com/paper/learning-two-layer-networks-with-multinomial
Repo
Framework

Efficient Convolutional Neural Network Training with Direct Feedback Alignment


Title	Efficient Convolutional Neural Network Training with Direct Feedback Alignment
Authors	Donghyeon Han, Hoi-jun Yoo
Abstract	There were many algorithms to substitute the back-propagation (BP) in the deep neural network (DNN) training. However, they could not become popular because their training accuracy and the computational efficiency were worse than BP. One of them was direct feedback alignment (DFA), but it showed low training performance especially for the convolutional neural network (CNN). In this paper, we overcome the limitation of the DFA algorithm by combining with the conventional BP during the CNN training. To improve the training stability, we also suggest the feedback weight initialization method by analyzing the patterns of the fixed random matrices in the DFA. Finally, we propose the new training algorithm, binary direct feedback alignment (BDFA) to minimize the computational cost while maintaining the training accuracy compared with the DFA. In our experiments, we use the CIFAR-10 and CIFAR-100 dataset to simulate the CNN learning from the scratch and apply the BDFA to the online learning based object tracking application to examine the training in the small dataset environment. Our proposed algorithms show better performance than conventional BP in both two different training tasks especially when the dataset is small.
Tasks	Object Tracking
Published	2019-01-06
URL	http://arxiv.org/abs/1901.01986v1
PDF	http://arxiv.org/pdf/1901.01986v1.pdf
PWC	https://paperswithcode.com/paper/efficient-convolutional-neural-network
Repo
Framework