October 18, 2019

3217 words 16 mins read

Paper Group ANR 533

Computational Intelligence in Sports: A Systematic Literature Review. Pumpout: A Meta Approach to Robust Deep Learning with Noisy Labels. Parameter-free Sentence Embedding via Orthogonal Basis. Challenges of Convex Quadratic Bi-objective Benchmark Problems. Semantic Parsing Natural Language into SPARQL: Improving Target Language Representation with …

Computational Intelligence in Sports: A Systematic Literature Review


Title	Computational Intelligence in Sports: A Systematic Literature Review
Authors	Robson P. Bonidia, Luiz A. L. Rodrigues, Anderson P. Avila-Santos, Danilo S. Sanches, Jacques D. Brancher
Abstract	Recently, data mining studies are being successfully conducted to estimate several parameters in a variety of domains. Data mining techniques have attracted the attention of the information industry and society as a whole, due to a large amount of data and the imminent need to turn it into useful knowledge. However, the effective use of data in some areas is still under development, as is the case in sports, which in recent years, has presented a slight growth; consequently, many sports organizations have begun to see that there is a wealth of unexplored knowledge in the data extracted by them. Therefore, this article presents a systematic review of sports data mining. Regarding years 2010 to 2018, 31 types of research were found in this topic. Based on these studies, we present the current panorama, themes, the database used, proposals, algorithms, and research opportunities. Our findings provide a better understanding of the sports data mining potentials, besides motivating the scientific community to explore this timely and interesting topic.
Tasks
Published	2018-10-30
URL	http://arxiv.org/abs/1810.12850v1
PDF	http://arxiv.org/pdf/1810.12850v1.pdf
PWC	https://paperswithcode.com/paper/computational-intelligence-in-sports-a
Repo
Framework

Pumpout: A Meta Approach to Robust Deep Learning with Noisy Labels


Title	Pumpout: A Meta Approach to Robust Deep Learning with Noisy Labels
Authors	Bo Han, Gang Niu, Jiangchao Yao, Xingrui Yu, Miao Xu, Ivor Tsang, Masashi Sugiyama
Abstract	Recent studies reveal that deep neural networks gradually memorize individual data while fitting distributions of data. Hence, when facing noisy labels, all existing methods inevitably suffer from generalization degeneration and have to be early stopped. In this paper, we propose Pumpout as a meta approach to learning with noisy labels and an alternative to early stopping. Pumpout comes from sample selection and goes beyond: in every mini-batch, it uses gradient decent on good data, while it uses scaled gradient ascent on bad data rather than drops those data, where the goodness and badness are w.r.t. a base learning method. It is advantageous over early stopping, since it can continue to fit distributions of data and it has the ability of actively forgetting individual data that is memorized by mistakes. We demonstrate via experiments that Pumpout robustifies two representative base learning methods, and the performance boost is often significant.
Tasks
Published	2018-09-28
URL	http://arxiv.org/abs/1809.11008v2
PDF	http://arxiv.org/pdf/1809.11008v2.pdf
PWC	https://paperswithcode.com/paper/pumpout-a-meta-approach-to-robust-deep
Repo
Framework

Parameter-free Sentence Embedding via Orthogonal Basis


Title	Parameter-free Sentence Embedding via Orthogonal Basis
Authors	Ziyi Yang, Chenguang Zhu, Weizhu Chen
Abstract	We propose a simple and robust non-parameterized approach for building sentence representations. Inspired by the Gram-Schmidt Process in geometric theory, we build an orthogonal basis of the subspace spanned by a word and its surrounding context in a sentence. We model the semantic meaning of a word in a sentence based on two aspects. One is its relatedness to the word vector subspace already spanned by its contextual words. The other is the word’s novel semantic meaning which shall be introduced as a new basis vector perpendicular to this existing subspace. Following this motivation, we develop an innovative method based on orthogonal basis to combine pre-trained word embeddings into sentence representations. This approach requires zero parameters, along with efficient inference performance. We evaluate our approach on 11 downstream NLP tasks. Our model shows superior performance compared with non-parameterized alternatives and it is competitive to other approaches relying on either large amounts of labelled data or prolonged training time.
Tasks	Sentence Embedding, Word Embeddings
Published	2018-09-30
URL	https://arxiv.org/abs/1810.00438v2
PDF	https://arxiv.org/pdf/1810.00438v2.pdf
PWC	https://paperswithcode.com/paper/zero-training-sentence-embedding-via
Repo
Framework

Challenges of Convex Quadratic Bi-objective Benchmark Problems


Title	Challenges of Convex Quadratic Bi-objective Benchmark Problems
Authors	Tobias Glasmachers
Abstract	Convex quadratic objective functions are an important base case in state-of-the-art benchmark collections for single-objective optimization on continuous domains. Although often considered rather simple, they represent the highly relevant challenges of non-separability and ill-conditioning. In the multi-objective case, quadratic benchmark problems are under-represented. In this paper we analyze the specific challenges that can be posed by quadratic functions in the bi-objective case. Our construction yields a full factorial design of 54 different problem classes. We perform experiments with well-established algorithms to demonstrate the insights that can be supported by this function class. We find huge performance differences, which can be clearly attributed to two root causes: non-separability and alignment of the Pareto set with the coordinate system.
Tasks
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09690v4
PDF	http://arxiv.org/pdf/1810.09690v4.pdf
PWC	https://paperswithcode.com/paper/challenges-of-convex-quadratic-bi-objective
Repo
Framework

Semantic Parsing Natural Language into SPARQL: Improving Target Language Representation with Neural Attention


Title	Semantic Parsing Natural Language into SPARQL: Improving Target Language Representation with Neural Attention
Authors	Fabiano Ferreira Luz, Marcelo Finger
Abstract	Semantic parsing is the process of mapping a natural language sentence into a formal representation of its meaning. In this work we use the neural network approach to transform natural language sentence into a query to an ontology database in the SPARQL language. This method does not rely on handcraft-rules, high-quality lexicons, manually-built templates or other handmade complex structures. Our approach is based on vector space model and neural networks. The proposed model is based in two learning steps. The first step generates a vector representation for the sentence in natural language and SPARQL query. The second step uses this vector representation as input to a neural network (LSTM with attention mechanism) to generate a model able to encode natural language and decode SPARQL.
Tasks	Semantic Parsing
Published	2018-03-12
URL	http://arxiv.org/abs/1803.04329v1
PDF	http://arxiv.org/pdf/1803.04329v1.pdf
PWC	https://paperswithcode.com/paper/semantic-parsing-natural-language-into-sparql
Repo
Framework

TuSeRACT: Turn-Sample-Based Real-Time Traffic Signal Control


Title	TuSeRACT: Turn-Sample-Based Real-Time Traffic Signal Control
Authors	Srishti Dhamija, Pradeep Varakantham
Abstract	Real-time traffic signal control is a challenging problem owing to constantly changing traffic demand patterns, limited planning time and various sources of uncertainty (e.g., turn movements, vehicle detection) in the real world. SURTRAC (Scalable URban TRAffic Control) is a recently developed traffic signal control approach which computes delay-minimizing and coordinated (across neighbouring traffic lights) schedules of oncoming vehicle clusters in real time. To ensure real-time responsiveness in the presence of turn-induced uncertainty, SURTRAC computes schedules which minimize the delay for the expected turn movements as opposed to minimizing the expected delay under turn-induced uncertainty. This approximation ensures real-time tractability, but degrades solution quality in the presence of turn-induced uncertainty. To address this limitation, we introduce TuSeRACT (Turn Sample based Real-time trAffic signal ConTrol), a distributed sample-based scheduling approach to traffic signal control. Unlike SURTRAC, TuSeRACT computes schedules that minimize expected delay over sampled turn movements of observed traffic, and communicates samples of traffic outflows to neighbouring intersections. We formulate this sample-based scheduling problem as a constraint program and empirically evaluate our approach on synthetic traffic networks. Our approach provides substantially lower mean vehicular waiting times relative to SURTRAC.
Tasks
Published	2018-12-13
URL	http://arxiv.org/abs/1812.05591v4
PDF	http://arxiv.org/pdf/1812.05591v4.pdf
PWC	https://paperswithcode.com/paper/tuseract-turn-sample-based-real-time-traffic
Repo
Framework

High Order Recurrent Neural Networks for Acoustic Modelling


Title	High Order Recurrent Neural Networks for Acoustic Modelling
Authors	Chao Zhang, Philip Woodland
Abstract	Vanishing long-term gradients are a major issue in training standard recurrent neural networks (RNNs), which can be alleviated by long short-term memory (LSTM) models with memory cells. However, the extra parameters associated with the memory cells mean an LSTM layer has four times as many parameters as an RNN with the same hidden vector size. This paper addresses the vanishing gradient problem using a high order RNN (HORNN) which has additional connections from multiple previous time steps. Speech recognition experiments using British English multi-genre broadcast (MGB3) data showed that the proposed HORNN architectures for rectified linear unit and sigmoid activation functions reduced word error rates (WER) by 4.2% and 6.3% over the corresponding RNNs, and gave similar WERs to a (projected) LSTM while using only 20%–50% of the recurrent layer parameters and computation.
Tasks	Acoustic Modelling, Speech Recognition
Published	2018-02-22
URL	http://arxiv.org/abs/1802.08314v1
PDF	http://arxiv.org/pdf/1802.08314v1.pdf
PWC	https://paperswithcode.com/paper/high-order-recurrent-neural-networks-for
Repo
Framework

Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network


Title	Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network
Authors	Liping Zhang, Zongqing Lu, Qingmin Liao
Abstract	The convolutional neural network model for optical flow estimation usually outputs a low-resolution(LR) optical flow field. To obtain the corresponding full image resolution,interpolation and variational approach are the most common options, which do not effectively improve the results. With the motivation of various convolutional neural network(CNN) structures succeeded in single image super-resolution(SISR) task, an end-to-end convolutional neural network is proposed to reconstruct the high resolution(HR) optical flow field from initial LR optical flow with the guidence of the first frame used in optical flow estimation. Our optical flow super-resolution(OFSR) problem differs from the general SISR problem in two main aspects. Firstly, the optical flow includes less texture information than image so that the SISR CNN structures can’t be directly used in our OFSR problem. Secondly, the initial LR optical flow data contains estimation error, while the LR image data for SISR is generally a bicubic downsampled, blurred, and noisy version of HR ground truth. We evaluate the proposed approach on two different optical flow estimation mehods and show that it can not only obtain the full image resolution, but generate more accurate optical flow field (Accuracy improve 15% on FlyingChairs, 13% on MPI Sintel) with sharper edges than the estimation result of original method.
Tasks	Image Super-Resolution, Optical Flow Estimation, Super-Resolution
Published	2018-09-03
URL	http://arxiv.org/abs/1809.00588v1
PDF	http://arxiv.org/pdf/1809.00588v1.pdf
PWC	https://paperswithcode.com/paper/optical-flow-super-resolution-based-on-image
Repo
Framework

Batch Size Influence on Performance of Graphic and Tensor Processing Units during Training and Inference Phases


Title	Batch Size Influence on Performance of Graphic and Tensor Processing Units during Training and Inference Phases
Authors	Yuriy Kochura, Yuri Gordienko, Vlad Taran, Nikita Gordienko, Alexandr Rokovyi, Oleg Alienin, Sergii Stirenko
Abstract	The impact of the maximally possible batch size (for the better runtime) on performance of graphic processing units (GPU) and tensor processing units (TPU) during training and inference phases is investigated. The numerous runs of the selected deep neural network (DNN) were performed on the standard MNIST and Fashion-MNIST datasets. The significant speedup was obtained even for extremely low-scale usage of Google TPUv2 units (8 cores only) in comparison to the quite powerful GPU NVIDIA Tesla K80 card with the speedup up to 10x for training stage (without taking into account the overheads) and speedup up to 2x for prediction stage (with and without taking into account overheads). The precise speedup values depend on the utilization level of TPUv2 units and increase with the increase of the data volume under processing, but for the datasets used in this work (MNIST and Fashion-MNIST with images of sizes 28x28) the speedup was observed for batch sizes >512 images for training phase and >40 000 images for prediction phase. It should be noted that these results were obtained without detriment to the prediction accuracy and loss that were equal for both GPU and TPU runs up to the 3rd significant digit for MNIST dataset, and up to the 2nd significant digit for Fashion-MNIST dataset.
Tasks
Published	2018-12-31
URL	http://arxiv.org/abs/1812.11731v1
PDF	http://arxiv.org/pdf/1812.11731v1.pdf
PWC	https://paperswithcode.com/paper/batch-size-influence-on-performance-of
Repo
Framework

Closed-form solution to cooperative visual-inertial structure from motion


Title	Closed-form solution to cooperative visual-inertial structure from motion
Authors	Agostino Martinelli
Abstract	This paper considers the problem of visual-inertial sensor fusion in the cooperative case and it provides new theoretical contributions, which regard its observability and its resolvability in closed form. The case of two agents is investigated. Each agent is equipped with inertial sensors (accelerometer and gyroscope) and with a monocular camera. By using the monocular camera, each agent can observe the other agent. No additional camera observations (e.g., of external point features in the environment) are considered. All the inertial sensors are assumed to be affected by a bias. First, the entire observable state is analytically derived. This state includes the absolute scale, the relative velocity between the two agents, the three Euler angles that express the rotation between the two agent frames and all the accelerometer and gyroscope biases. Second, the paper provides the extension of the closed-form solution given in [19] (which holds for a single agent) to the aforementioned cooperative case. The impact of the presence of the bias on the performance of this closed-form solution is investigated. As in the case of a single agent, this performance is significantly sensitive to the presence of a bias on the gyroscope, while, the presence of a bias on the accelerometer is negligible. Finally, a simple and effective method to obtain the gyroscope bias is proposed. Extensive simulations clearly show that the proposed method is successful. It is amazing that, it is possible to automatically retrieve the absolute scale and simultaneously calibrate the gyroscopes not only without any prior knowledge (as in [13]), but also without external point features in the environment.
Tasks	Sensor Fusion
Published	2018-02-23
URL	http://arxiv.org/abs/1802.08515v1
PDF	http://arxiv.org/pdf/1802.08515v1.pdf
PWC	https://paperswithcode.com/paper/closed-form-solution-to-cooperative-visual
Repo
Framework

Dynamic Spatio-temporal Graph-based CNNs for Traffic Prediction


Title	Dynamic Spatio-temporal Graph-based CNNs for Traffic Prediction
Authors	Ken Chen, Fei Chen, Baisheng Lai, Zhongming Jin, Yong Liu, Kai Li, Long Wei, Pengfei Wang, Yandong Tang, Jianqiang Huang, Xian-Sheng Hua
Abstract	Forecasting future traffic flows from previous ones is a challenging problem because of their complex and dynamic nature of spatio-temporal structures. Most existing graph-based CNNs attempt to capture the static relations while largely neglecting the dynamics underlying sequential data. In this paper, we present dynamic spatio-temporal graph-based CNNs (DST-GCNNs) by learning expressive features to represent spatio-temporal structures and predict future traffic flows from surveillance video data. In particular, DST-GCNN is a two stream network. In the flow prediction stream, we present a novel graph-based spatio-temporal convolutional layer to extract features from a graph representation of traffic flows. Then several such layers are stacked together to predict future flows over time. Meanwhile, the relations between traffic flows in the graph are often time variant as the traffic condition changes over time. To capture the graph dynamics, we use the graph prediction stream to predict the dynamic graph structures, and the predicted structures are fed into the flow prediction stream. Experiments on real datasets demonstrate that the proposed model achieves competitive performances compared with the other state-of-the-art methods.
Tasks	Traffic Prediction
Published	2018-12-05
URL	https://arxiv.org/abs/1812.02019v4
PDF	https://arxiv.org/pdf/1812.02019v4.pdf
PWC	https://paperswithcode.com/paper/dynamic-spatio-temporal-graph-based-cnns-for
Repo
Framework

A torus model for optical flow


Title	A torus model for optical flow
Authors	Henry Adams, Johnathan Bush, Brittany Carr, Lara Kassab, Joshua Mirth
Abstract	We propose a torus model for high-contrast patches of optical flow. Our model is derived from a database of ground-truth optical flow from the computer-generated video \emph{Sintel}, collected by Butler et al.\ in \emph{A naturalistic open source movie for optical flow evaluation}. Using persistent homology and zigzag persistence, popular tools from the field of computational topology, we show that the high-contrast $3\times 3$ patches from this video are well-modeled by a \emph{torus}, a nonlinear 2-dimensional manifold. Furthermore, we show that the optical flow torus model is naturally equipped with the structure of a fiber bundle, related to the statistics of range image patches.
Tasks	Optical Flow Estimation
Published	2018-11-09
URL	https://arxiv.org/abs/1812.00875v2
PDF	https://arxiv.org/pdf/1812.00875v2.pdf
PWC	https://paperswithcode.com/paper/a-torus-model-for-optical-flow
Repo
Framework

Feature Prioritization and Regularization Improve Standard Accuracy and Adversarial Robustness


Title	Feature Prioritization and Regularization Improve Standard Accuracy and Adversarial Robustness
Authors	Chihuang Liu, Joseph JaJa
Abstract	Adversarial training has been successfully applied to build robust models at a certain cost. While the robustness of a model increases, the standard classification accuracy declines. This phenomenon is suggested to be an inherent trade-off. We propose a model that employs feature prioritization by a nonlinear attention module and $L_2$ feature regularization to improve the adversarial robustness and the standard accuracy relative to adversarial training. The attention module encourages the model to rely heavily on robust features by assigning larger weights to them while suppressing non-robust features. The regularizer encourages the model to extract similar features for the natural and adversarial images, effectively ignoring the added perturbation. In addition to evaluating the robustness of our model, we provide justification for the attention module and propose a novel experimental strategy that quantitatively demonstrates that our model is almost ideally aligned with salient data characteristics. Additional experimental results illustrate the power of our model relative to the state of the art methods.
Tasks	Denoising
Published	2018-10-04
URL	https://arxiv.org/abs/1810.02424v3
PDF	https://arxiv.org/pdf/1810.02424v3.pdf
PWC	https://paperswithcode.com/paper/feature-prioritization-and-regularization
Repo
Framework

Out of the Black Box: Properties of deep neural networks and their applications


Title	Out of the Black Box: Properties of deep neural networks and their applications
Authors	Nizar Ouarti, David Carmona
Abstract	Deep neural networks are powerful machine learning approaches that have exhibited excellent results on many classification tasks. However, they are considered as black boxes and some of their properties remain to be formalized. In the context of image recognition, it is still an arduous task to understand why an image is recognized or not. In this study, we formalize some properties shared by eight state-of-the-art deep neural networks in order to grasp the principles allowing a given deep neural network to classify an image. Our results, tested on these eight networks, show that an image can be sub-divided into several regions (patches) responding at different degrees of probability (local property). With the same patch, some locations in the image can answer two (or three) orders of magnitude higher than other locations (spatial property). Some locations are activators and others inhibitors (activation-inhibition property). The repetition of the same patch can increase (or decrease) the probability of recognition of an object (cumulative property). Furthermore, we propose a new approach called Deepception that exploits these properties to deceive a deep neural network. We obtain for the VGG-VDD-19 neural network a fooling ratio of 88%. Thanks to our “Psychophysics” approach, no prior knowledge on the networks architectures is required.
Tasks
Published	2018-08-10
URL	http://arxiv.org/abs/1808.04433v1
PDF	http://arxiv.org/pdf/1808.04433v1.pdf
PWC	https://paperswithcode.com/paper/out-of-the-black-box-properties-of-deep
Repo
Framework

Detecting Concrete Abnormality Using Time-series Thermal Imaging and Supervised Learning


Title	Detecting Concrete Abnormality Using Time-series Thermal Imaging and Supervised Learning
Authors	Chongsheng Cheng, Zhigang Shen
Abstract	Nondestructive detecting defects (NDD) in concrete structures have been explored for decades. Although limited successes were reported, major limitations still exist. The major limitations are the high noises to signal ratio created from the environmental factors, such as cloud, shadow, water, surface texture etc. and the decision making still relies on the engineering judgment of interpretation of image content. Time-series approach, such as principle component thermography approach has been experimented with some improved results. Recent progress in image processing using machine learning approach made it possible for detecting defects thermal features in more quantitative ways. In this paper, we provide a procedure to represent the thermal feature in the time domain by principal component analysis and regress the prediction of detection by two schemes of supervised learning models. Three independent experiments were conducted in a similar laboratory setup but varied in conditions to illustrate the performance and generalization of models. Results showed the effectiveness for the detection purpose with appropriate tuning for parameters. Future studies will focus on implementing more sophisticated structured models to handle more realistic cases under natural conditions.
Tasks	Decision Making, Time Series
Published	2018-04-15
URL	http://arxiv.org/abs/1804.05406v2
PDF	http://arxiv.org/pdf/1804.05406v2.pdf
PWC	https://paperswithcode.com/paper/detecting-concrete-abnormality-using-time
Repo
Framework