July 29, 2019

3207 words 16 mins read

Paper Group ANR 156

Learning Graphical Models Using Multiplicative Weights. Character-Based Text Classification using Top Down Semantic Model for Sentence Representation. Object Recognition from very few Training Examples for Enhancing Bicycle Maps. Communications that Emerge through Reinforcement Learning Using a (Recurrent) Neural Network. Cyberattack Detection in M …

Learning Graphical Models Using Multiplicative Weights


Title	Learning Graphical Models Using Multiplicative Weights
Authors	Adam Klivans, Raghu Meka
Abstract	We give a simple, multiplicative-weight update algorithm for learning undirected graphical models or Markov random fields (MRFs). The approach is new, and for the well-studied case of Ising models or Boltzmann machines, we obtain an algorithm that uses a nearly optimal number of samples and has quadratic running time (up to logarithmic factors), subsuming and improving on all prior work. Additionally, we give the first efficient algorithm for learning Ising models over general alphabets. Our main application is an algorithm for learning the structure of t-wise MRFs with nearly-optimal sample complexity (up to polynomial losses in necessary terms that depend on the weights) and running time that is $n^{O(t)}$. In addition, given $n^{O(t)}$ samples, we can also learn the parameters of the model and generate a hypothesis that is close in statistical distance to the true MRF. All prior work runs in time $n^{\Omega(d)}$ for graphs of bounded degree d and does not generate a hypothesis close in statistical distance even for t=3. We observe that our runtime has the correct dependence on n and t assuming the hardness of learning sparse parities with noise. Our algorithm–the Sparsitron– is easy to implement (has only one parameter) and holds in the on-line setting. Its analysis applies a regret bound from Freund and Schapire’s classic Hedge algorithm. It also gives the first solution to the problem of learning sparse Generalized Linear Models (GLMs).
Tasks
Published	2017-06-20
URL	http://arxiv.org/abs/1706.06274v1
PDF	http://arxiv.org/pdf/1706.06274v1.pdf
PWC	https://paperswithcode.com/paper/learning-graphical-models-using
Repo
Framework

Character-Based Text Classification using Top Down Semantic Model for Sentence Representation


Title	Character-Based Text Classification using Top Down Semantic Model for Sentence Representation
Authors	Zhenzhou Wu, Xin Zheng, Daniel Dahlmeier
Abstract	Despite the success of deep learning on many fronts especially image and speech, its application in text classification often is still not as good as a simple linear SVM on n-gram TF-IDF representation especially for smaller datasets. Deep learning tends to emphasize on sentence level semantics when learning a representation with models like recurrent neural network or recursive neural network, however from the success of TF-IDF representation, it seems a bag-of-words type of representation has its strength. Taking advantage of both representions, we present a model known as TDSM (Top Down Semantic Model) for extracting a sentence representation that considers both the word-level semantics by linearly combining the words with attention weights and the sentence-level semantics with BiLSTM and use it on text classification. We apply the model on characters and our results show that our model is better than all the other character-based and word-based convolutional neural network models by \cite{zhang15} across seven different datasets with only 1% of their parameters. We also demonstrate that this model beats traditional linear models on TF-IDF vectors on small and polished datasets like news article in which typically deep learning models surrender.
Tasks	Text Classification
Published	2017-05-29
URL	http://arxiv.org/abs/1705.10586v1
PDF	http://arxiv.org/pdf/1705.10586v1.pdf
PWC	https://paperswithcode.com/paper/character-based-text-classification-using-top
Repo
Framework

Object Recognition from very few Training Examples for Enhancing Bicycle Maps


Title	Object Recognition from very few Training Examples for Enhancing Bicycle Maps
Authors	Christoph Reinders, Hanno Ackermann, Michael Ying Yang, Bodo Rosenhahn
Abstract	In recent years, data-driven methods have shown great success for extracting information about the infrastructure in urban areas. These algorithms are usually trained on large datasets consisting of thousands or millions of labeled training examples. While large datasets have been published regarding cars, for cyclists very few labeled data is available although appearance, point of view, and positioning of even relevant objects differ. Unfortunately, labeling data is costly and requires a huge amount of work. In this paper, we thus address the problem of learning with very few labels. The aim is to recognize particular traffic signs in crowdsourced data to collect information which is of interest to cyclists. We propose a system for object recognition that is trained with only 15 examples per class on average. To achieve this, we combine the advantages of convolutional neural networks and random forests to learn a patch-wise classifier. In the next step, we map the random forest to a neural network and transform the classifier to a fully convolutional network. Thereby, the processing of full images is significantly accelerated and bounding boxes can be predicted. Finally, we integrate data of the Global Positioning System (GPS) to localize the predictions on the map. In comparison to Faster R-CNN and other networks for object recognition or algorithms for transfer learning, we considerably reduce the required amount of labeled data. We demonstrate good performance on the recognition of traffic signs for cyclists as well as their localization in maps.
Tasks	Object Recognition, Transfer Learning
Published	2017-09-18
URL	http://arxiv.org/abs/1709.05910v4
PDF	http://arxiv.org/pdf/1709.05910v4.pdf
PWC	https://paperswithcode.com/paper/object-recognition-from-very-few-training
Repo
Framework

Communications that Emerge through Reinforcement Learning Using a (Recurrent) Neural Network


Title	Communications that Emerge through Reinforcement Learning Using a (Recurrent) Neural Network
Authors	Katsunari Shibata
Abstract	Communication is not only an action of choosing a signal, but needs to consider the context and sensor signals. It also needs to decide what information is communicated and how it is represented in or understood from signals. Therefore, communication should be realized comprehensively together with its purpose and other functions. The recent successful results in end-to-end reinforcement learning (RL) show the importance of comprehensive learning and the usefulness of end-to-end RL. Although little is known, we have shown that a variety of communications emerge through RL using a (recurrent) neural network (NN). Here, three of them are introduced. In the 1st one, negotiation to avoid conflicts among 4 randomly-picked agents was learned. Each agent generates a binary signal from the output of its recurrent NN (RNN), and receives 4 signals from the agents three times. After learning, each agent made an appropriate final decision after negotiation for any combination of 4 agents. Differentiation of individuality among the agents also could be seen. The 2nd one focused on discretization of communication signal. A sender agent perceives the receiver’s location and generates a continuous signal twice by its RNN. A receiver agent receives them sequentially, and moves according to its RNN’s output to reach the sender’s location. When noises were added to the signal, it was binarized through learning and 2-bit communication was established. The 3rd one focused on end-to-end comprehensive communication. A sender receives 1,785-pixel real camera image on which a real robot can be seen, and sends two sounds whose frequencies are computed by its NN. A receiver receives them, and two motion commands for the robot are generated by its NN. After learning, though some preliminary learning was necessary for the sender, the robot could reach the goal from any initial location.
Tasks
Published	2017-03-10
URL	http://arxiv.org/abs/1703.03543v2
PDF	http://arxiv.org/pdf/1703.03543v2.pdf
PWC	https://paperswithcode.com/paper/communications-that-emerge-through
Repo
Framework

Cyberattack Detection in Mobile Cloud Computing: A Deep Learning Approach


Title	Cyberattack Detection in Mobile Cloud Computing: A Deep Learning Approach
Authors	Khoi Khac Nguyen, Dinh Thai Hoang, Dusit Niyato, Ping Wang, Diep Nguyen, Eryk Dutkiewicz
Abstract	With the rapid growth of mobile applications and cloud computing, mobile cloud computing has attracted great interest from both academia and industry. However, mobile cloud applications are facing security issues such as data integrity, users’ confidentiality, and service availability. A preventive approach to such problems is to detect and isolate cyber threats before they can cause serious impacts to the mobile cloud computing system. In this paper, we propose a novel framework that leverages a deep learning approach to detect cyberattacks in mobile cloud environment. Through experimental results, we show that our proposed framework not only recognizes diverse cyberattacks, but also achieves a high accuracy (up to 97.11%) in detecting the attacks. Furthermore, we present the comparisons with current machine learning-based approaches to demonstrate the effectiveness of our proposed solution.
Tasks
Published	2017-12-16
URL	http://arxiv.org/abs/1712.05914v1
PDF	http://arxiv.org/pdf/1712.05914v1.pdf
PWC	https://paperswithcode.com/paper/cyberattack-detection-in-mobile-cloud
Repo
Framework

Automatic Question-Answering Using A Deep Similarity Neural Network


Title	Automatic Question-Answering Using A Deep Similarity Neural Network
Authors	Shervin Minaee, Zhu Liu
Abstract	Automatic question-answering is a classical problem in natural language processing, which aims at designing systems that can automatically answer a question, in the same way as human does. In this work, we propose a deep learning based model for automatic question-answering. First the questions and answers are embedded using neural probabilistic modeling. Then a deep similarity neural network is trained to find the similarity score of a pair of answer and question. Then for each question, the best answer is found as the one with the highest similarity score. We first train this model on a large-scale public question-answering database, and then fine-tune it to transfer to the customer-care chat data. We have also tested our framework on a public question-answering database and achieved very good performance.
Tasks	Question Answering
Published	2017-08-05
URL	http://arxiv.org/abs/1708.01713v1
PDF	http://arxiv.org/pdf/1708.01713v1.pdf
PWC	https://paperswithcode.com/paper/automatic-question-answering-using-a-deep
Repo
Framework

Homotopy Parametric Simplex Method for Sparse Learning


Title	Homotopy Parametric Simplex Method for Sparse Learning
Authors	Haotian Pang, Robert Vanderbei, Han Liu, Tuo Zhao
Abstract	High dimensional sparse learning has imposed a great computational challenge to large scale data analysis. In this paper, we are interested in a broad class of sparse learning approaches formulated as linear programs parametrized by a {\em regularization factor}, and solve them by the parametric simplex method (PSM). Our parametric simplex method offers significant advantages over other competing methods: (1) PSM naturally obtains the complete solution path for all values of the regularization parameter; (2) PSM provides a high precision dual certificate stopping criterion; (3) PSM yields sparse solutions through very few iterations, and the solution sparsity significantly reduces the computational cost per iteration. Particularly, we demonstrate the superiority of PSM over various sparse learning approaches, including Dantzig selector for sparse linear regression, LAD-Lasso for sparse robust linear regression, CLIME for sparse precision matrix estimation, sparse differential network estimation, and sparse Linear Programming Discriminant (LPD) analysis. We then provide sufficient conditions under which PSM always outputs sparse solutions such that its computational performance can be significantly boosted. Thorough numerical experiments are provided to demonstrate the outstanding performance of the PSM method.
Tasks	Sparse Learning
Published	2017-04-04
URL	http://arxiv.org/abs/1704.01079v2
PDF	http://arxiv.org/pdf/1704.01079v2.pdf
PWC	https://paperswithcode.com/paper/homotopy-parametric-simplex-method-for-sparse
Repo
Framework

Improving Speaker-Independent Lipreading with Domain-Adversarial Training


Title	Improving Speaker-Independent Lipreading with Domain-Adversarial Training
Authors	Michael Wand, Juergen Schmidhuber
Abstract	We present a Lipreading system, i.e. a speech recognition system using only visual features, which uses domain-adversarial training for speaker independence. Domain-adversarial training is integrated into the optimization of a lipreader based on a stack of feedforward and LSTM (Long Short-Term Memory) recurrent neural networks, yielding an end-to-end trainable system which only requires a very small number of frames of untranscribed target data to substantially improve the recognition accuracy on the target speaker. On pairs of different source and target speakers, we achieve a relative accuracy improvement of around 40% with only 15 to 20 seconds of untranscribed target speech data. On multi-speaker training setups, the accuracy improvements are smaller but still substantial.
Tasks	Lipreading, Speech Recognition
Published	2017-08-04
URL	http://arxiv.org/abs/1708.01565v1
PDF	http://arxiv.org/pdf/1708.01565v1.pdf
PWC	https://paperswithcode.com/paper/improving-speaker-independent-lipreading-with
Repo
Framework

Computationally efficient cardiac views projection using 3D Convolutional Neural Networks


Title	Computationally efficient cardiac views projection using 3D Convolutional Neural Networks
Authors	Matthieu Le, Jesse Lieman-Sifry, Felix Lau, Sean Sall, Albert Hsiao, Daniel Golden
Abstract	4D Flow is an MRI sequence which allows acquisition of 3D images of the heart. The data is typically acquired volumetrically, so it must be reformatted to generate cardiac long axis and short axis views for diagnostic interpretation. These views may be generated by placing 6 landmarks: the left and right ventricle apex, and the aortic, mitral, pulmonary, and tricuspid valves. In this paper, we propose an automatic method to localize landmarks in order to compute the cardiac views. Our approach consists of first calculating a bounding box that tightly crops the heart, followed by a landmark localization step within this bounded region. Both steps are based on a 3D extension of the recently introduced ENet. We demonstrate that the long and short axis projections computed with our automated method are of equivalent quality to projections created with landmarks placed by an experienced cardiac radiologist, based on a blinded test administered to a different cardiac radiologist.
Tasks
Published	2017-11-03
URL	http://arxiv.org/abs/1711.01345v1
PDF	http://arxiv.org/pdf/1711.01345v1.pdf
PWC	https://paperswithcode.com/paper/computationally-efficient-cardiac-views
Repo
Framework

TF Boosted Trees: A scalable TensorFlow based framework for gradient boosting


Title	TF Boosted Trees: A scalable TensorFlow based framework for gradient boosting
Authors	Natalia Ponomareva, Soroush Radpour, Gilbert Hendry, Salem Haykal, Thomas Colthurst, Petr Mitrichev, Alexander Grushetsky
Abstract	TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, principled multi-class handling, and a number of regularization techniques to prevent overfitting.
Tasks
Published	2017-10-31
URL	http://arxiv.org/abs/1710.11555v1
PDF	http://arxiv.org/pdf/1710.11555v1.pdf
PWC	https://paperswithcode.com/paper/tf-boosted-trees-a-scalable-tensorflow-based
Repo
Framework


Title	Google Map Aided Visual Navigation for UAVs in GPS-denied Environment
Authors	Mo Shan, Fei Wang, Feng Lin, Zhi Gao, Ya Z. Tang, Ben M. Chen
Abstract	We propose a framework for Google Map aided UAV navigation in GPS-denied environment. Geo-referenced navigation provides drift-free localization and does not require loop closures. The UAV position is initialized via correlation, which is simple and efficient. We then use optical flow to predict its position in subsequent frames. During pose tracking, we obtain inter-frame translation either by motion field or homography decomposition, and we use HOG features for registration on Google Map. We employ particle filter to conduct a coarse to fine search to localize the UAV. Offline test using aerial images collected by our quadrotor platform shows promising results as our approach eliminates the drift in dead-reckoning, and the small localization error indicates the superiority of our approach as a supplement to GPS.
Tasks	Optical Flow Estimation, Pose Tracking, Visual Navigation
Published	2017-03-29
URL	http://arxiv.org/abs/1703.10125v1
PDF	http://arxiv.org/pdf/1703.10125v1.pdf
PWC	https://paperswithcode.com/paper/google-map-aided-visual-navigation-for-uavs
Repo
Framework

Institutionally Distributed Deep Learning Networks


Title	Institutionally Distributed Deep Learning Networks
Authors	Ken Chang, Niranjan Balachandar, Carson K Lam, Darvin Yi, James M Brown, Andrew Beers, Bruce R Rosen, Daniel L Rubin, Jayashree Kalpathy-Cramer
Abstract	Deep learning has become a promising approach for automated medical diagnoses. When medical data samples are limited, collaboration among multiple institutions is necessary to achieve high algorithm performance. However, sharing patient data often has limitations due to technical, legal, or ethical concerns. In such cases, sharing a deep learning model is a more attractive alternative. The best method of performing such a task is unclear, however. In this study, we simulate the dissemination of learning deep learning network models across four institutions using various heuristics and compare the results with a deep learning model trained on centrally hosted patient data. The heuristics investigated include ensembling single institution models, single weight transfer, and cyclical weight transfer. We evaluated these approaches for image classification in three independent image collections (retinal fundus photos, mammography, and ImageNet). We find that cyclical weight transfer resulted in a performance (testing accuracy = 77.3%) that was closest to that of centrally hosted patient data (testing accuracy = 78.7%). We also found that there is an improvement in the performance of cyclical weight transfer heuristic with high frequency of weight transfer.
Tasks	Image Classification
Published	2017-09-10
URL	http://arxiv.org/abs/1709.05929v1
PDF	http://arxiv.org/pdf/1709.05929v1.pdf
PWC	https://paperswithcode.com/paper/institutionally-distributed-deep-learning
Repo
Framework

Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning


Title	Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning
Authors	Qi Dou, Hao Chen, Yueming Jin, Huangjing Lin, Jing Qin, Pheng-Ann Heng
Abstract	In this paper, we propose a novel framework with 3D convolutional networks (ConvNets) for automated detection of pulmonary nodules from low-dose CT scans, which is a challenging yet crucial task for lung cancer early diagnosis and treatment. Different from previous standard ConvNets, we try to tackle the severe hard/easy sample imbalance problem in medical datasets and explore the benefits of localized annotations to regularize the learning, and hence boost the performance of ConvNets to achieve more accurate detections. Our proposed framework consists of two stages: 1) candidate screening, and 2) false positive reduction. In the first stage, we establish a 3D fully convolutional network, effectively trained with an online sample filtering scheme, to sensitively and rapidly screen the nodule candidates. In the second stage, we design a hybrid-loss residual network which harnesses the location and size information as important cues to guide the nodule recognition procedure. Experimental results on the public large-scale LUNA16 dataset demonstrate superior performance of our proposed method compared with state-of-the-art approaches for the pulmonary nodule detection task.
Tasks
Published	2017-08-13
URL	http://arxiv.org/abs/1708.03867v1
PDF	http://arxiv.org/pdf/1708.03867v1.pdf
PWC	https://paperswithcode.com/paper/automated-pulmonary-nodule-detection-via-3d
Repo
Framework

Neural networks and rational functions


Title	Neural networks and rational functions
Authors	Matus Telgarsky
Abstract	Neural networks and rational functions efficiently approximate each other. In more detail, it is shown here that for any ReLU network, there exists a rational function of degree $O(\text{polylog}(1/\epsilon))$ which is $\epsilon$-close, and similarly for any rational function there exists a ReLU network of size $O(\text{polylog}(1/\epsilon))$ which is $\epsilon$-close. By contrast, polynomials need degree $\Omega(\text{poly}(1/\epsilon))$ to approximate even a single ReLU. When converting a ReLU network to a rational function as above, the hidden constants depend exponentially on the number of layers, which is shown to be tight; in other words, a compositional representation can be beneficial even for rational functions.
Tasks
Published	2017-06-11
URL	http://arxiv.org/abs/1706.03301v1
PDF	http://arxiv.org/pdf/1706.03301v1.pdf
PWC	https://paperswithcode.com/paper/neural-networks-and-rational-functions
Repo
Framework

Progress Estimation and Phase Detection for Sequential Processes


Title	Progress Estimation and Phase Detection for Sequential Processes
Authors	Xinyu Li, Yanyi Zhang, Jianyu Zhang, Yueyang Chen, Shuhong Chen, Yue Gu, Moliang Zhou, Richard A. Farneth, Ivan Marsic, Randall S. Burd
Abstract	Process modeling and understanding are fundamental for advanced human-computer interfaces and automation systems. Most recent research has focused on activity recognition, but little has been done on sensor-based detection of process progress. We introduce a real-time, sensor-based system for modeling, recognizing and estimating the progress of a work process. We implemented a multimodal deep learning structure to extract the relevant spatio-temporal features from multiple sensory inputs and used a novel deep regression structure for overall completeness estimation. Using process completeness estimation with a Gaussian mixture model, our system can predict the phase for sequential processes. The performance speed, calculated using completeness estimation, allows online estimation of the remaining time. To train our system, we introduced a novel rectified hyperbolic tangent (rtanh) activation function and conditional loss. Our system was tested on data obtained from the medical process (trauma resuscitation) and sports events (Olympic swimming competition). Our system outperformed the existing trauma-resuscitation phase detectors with a phase detection accuracy of over 86%, an F1-score of 0.67, a completeness estimation error of under 12.6%, and a remaining-time estimation error of less than 7.5 minutes. For the Olympic swimming dataset, our system achieved an accuracy of 88%, an F1-score of 0.58, a completeness estimation error of 6.3% and a remaining-time estimation error of 2.9 minutes.
Tasks	Activity Recognition
Published	2017-02-28
URL	http://arxiv.org/abs/1702.08623v3
PDF	http://arxiv.org/pdf/1702.08623v3.pdf
PWC	https://paperswithcode.com/paper/progress-estimation-and-phase-detection-for
Repo
Framework