April 2, 2020

3255 words 16 mins read

Paper Group ANR 131

FlexiBO: Cost-Aware Multi-Objective Optimization of Deep Neural Networks. Joint Event Extraction along Shortest Dependency Paths using Graph Convolutional Networks. Optimizing Revenue while showing Relevant Assortments at Scale. Classification of Hyperspectral and LiDAR Data Using Coupled CNNs. Differential Dynamic Programming Neural Optimizer. Ill …

FlexiBO: Cost-Aware Multi-Objective Optimization of Deep Neural Networks


Title	FlexiBO: Cost-Aware Multi-Objective Optimization of Deep Neural Networks
Authors	Md Shahriar Iqbal, Jianhai Su, Lars Kotthoff, Pooyan Jamshidi
Abstract	One of the key challenges in designing machine learning systems is to determine the right balance amongst several objectives, which also oftentimes are incommensurable and conflicting. For example, when designing deep neural networks (DNNs), one often has to trade-off between multiple objectives, such as accuracy, energy consumption, and inference time. Typically, there is no single configuration that performs equally well for all objectives. Consequently, one is interested in identifying Pareto-optimal designs. Although different multi-objective optimization algorithms have been developed to identify Pareto-optimal configurations, state-of-the-art multi-objective optimization methods do not consider the different evaluation costs attending the objectives under consideration. This is particularly important for optimizing DNNs: the cost arising on account of assessing the accuracy of DNNs is orders of magnitude higher than that of measuring the energy consumption of pre-trained DNNs. We propose FlexiBO, a flexible Bayesian optimization method, to address this issue. We formulate a new acquisition function based on the improvement of the Pareto hyper-volume weighted by the measurement cost of each objective. Our acquisition function selects the next sample and objective that provides maximum information gain per unit of cost. We evaluated FlexiBO on 7 state-of-the-art DNNs for object detection, natural language processing, and speech recognition. Our results indicate that, when compared to other state-of-the-art methods across the 7 architectures we tested, the Pareto front obtained using FlexiBO has, on average, a 28.44% higher contribution to the true Pareto front and achieves 25.64% better diversity.
Tasks	Object Detection, Speech Recognition
Published	2020-01-18
URL	https://arxiv.org/abs/2001.06588v1
PDF	https://arxiv.org/pdf/2001.06588v1.pdf
PWC	https://paperswithcode.com/paper/flexibo-cost-aware-multi-objective
Repo
Framework

Joint Event Extraction along Shortest Dependency Paths using Graph Convolutional Networks


Title	Joint Event Extraction along Shortest Dependency Paths using Graph Convolutional Networks
Authors	Ali Balali, Masoud Asadpour, Ricardo Campos, Adam Jatowt
Abstract	Event extraction (EE) is one of the core information extraction tasks, whose purpose is to automatically identify and extract information about incidents and their actors from texts. This may be beneficial to several domains such as knowledge bases, question answering, information retrieval and summarization tasks, to name a few. The problem of extracting event information from texts is longstanding and usually relies on elaborately designed lexical and syntactic features, which, however, take a large amount of human effort and lack generalization. More recently, deep neural network approaches have been adopted as a means to learn underlying features automatically. However, existing networks do not make full use of syntactic features, which play a fundamental role in capturing very long-range dependencies. Also, most approaches extract each argument of an event separately without considering associations between arguments which ultimately leads to low efficiency, especially in sentences with multiple events. To address the two above-referred problems, we propose a novel joint event extraction framework that aims to extract multiple event triggers and arguments simultaneously by introducing shortest dependency path (SDP) in the dependency graph. We do this by eliminating irrelevant words in the sentence, thus capturing long-range dependencies. Also, an attention-based graph convolutional network is proposed, to carry syntactically related information along the shortest paths between argument candidates that captures and aggregates the latent associations between arguments; a problem that has been overlooked by most of the literature. Our results show a substantial improvement over state-of-the-art methods.
Tasks	Information Retrieval, Question Answering
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08615v1
PDF	https://arxiv.org/pdf/2003.08615v1.pdf
PWC	https://paperswithcode.com/paper/joint-event-extraction-along-shortest
Repo
Framework

Optimizing Revenue while showing Relevant Assortments at Scale


Title	Optimizing Revenue while showing Relevant Assortments at Scale
Authors	Theja Tulabandhula, Deeksha Sinha
Abstract	Scalable real-time assortment optimization has become essential in e-commerce operations due to the need for personalization and the availability of a large variety of items. While this can be done when there are simplistic assortment choices to be made, imposing constraints on the collection of feasible assortments gives more flexibility to incorporate insights of store-managers and historically well-performing assortments. We design fast and flexible algorithms based on variations of binary search that find the revenue of the (approximately) optimal assortment. In particular, we revisit the problem of large-scale assortment optimization under the multinomial logit choice model without any assumptions on the structure of the feasible assortments. We speed up the comparisons steps using novel vector space embeddings, based on advances in the fields of information retrieval and machine learning. For an arbitrary collection of assortments, our algorithms can find a solution in time that is sub-linear in the number of assortments and for the simpler case of cardinality constraints - linear in the number of items (existing methods are quadratic or worse). Empirical validations using the Billion Prices dataset and several retail transaction datasets show that our algorithms are competitive even when the number of items is $\sim 10^5$ ($100$x larger instances than previously studied).
Tasks	Information Retrieval
Published	2020-03-06
URL	https://arxiv.org/abs/2003.04736v1
PDF	https://arxiv.org/pdf/2003.04736v1.pdf
PWC	https://paperswithcode.com/paper/optimizing-revenue-while-showing-relevant
Repo
Framework

Classification of Hyperspectral and LiDAR Data Using Coupled CNNs


Title	Classification of Hyperspectral and LiDAR Data Using Coupled CNNs
Authors	Renlong Hang, Zhu Li, Pedram Ghamisi, Danfeng Hong, Guiyu Xia, Qingshan Liu
Abstract	In this paper, we propose an efficient and effective framework to fuse hyperspectral and Light Detection And Ranging (LiDAR) data using two coupled convolutional neural networks (CNNs). One CNN is designed to learn spectral-spatial features from hyperspectral data, and the other one is used to capture the elevation information from LiDAR data. Both of them consist of three convolutional layers, and the last two convolutional layers are coupled together via a parameter sharing strategy. In the fusion phase, feature-level and decision-level fusion methods are simultaneously used to integrate these heterogeneous features sufficiently. For the feature-level fusion, three different fusion strategies are evaluated, including the concatenation strategy, the maximization strategy, and the summation strategy. For the decision-level fusion, a weighted summation strategy is adopted, where the weights are determined by the classification accuracy of each output. The proposed model is evaluated on an urban data set acquired over Houston, USA, and a rural one captured over Trento, Italy. On the Houston data, our model can achieve a new record overall accuracy of 96.03%. On the Trento data, it achieves an overall accuracy of 99.12%. These results sufficiently certify the effectiveness of our proposed model.
Tasks
Published	2020-02-04
URL	https://arxiv.org/abs/2002.01144v1
PDF	https://arxiv.org/pdf/2002.01144v1.pdf
PWC	https://paperswithcode.com/paper/classification-of-hyperspectral-and-lidar
Repo
Framework

Differential Dynamic Programming Neural Optimizer


Title	Differential Dynamic Programming Neural Optimizer
Authors	Guan-Horng Liu, Tianrong Chen, Evangelos A. Theodorou
Abstract	Interpretation of Deep Neural Networks (DNNs) training as an optimal control problem with nonlinear dynamical systems has received considerable attention recently, yet the algorithmic development remains relatively limited. In this work, we make an attempt along this line by reformulating the training procedure from the trajectory optimization perspective. We first show that most widely-used algorithms for training DNNs can be linked to the Differential Dynamic Programming (DDP), a celebrated second-order trajectory optimization algorithm rooted in the Approximate Dynamic Programming. In this vein, we propose a new variant of DDP that can accept batch optimization for training feedforward networks, while integrating naturally with the recent progress in curvature approximation. The resulting algorithm features layer-wise feedback policies which improve convergence rate and reduce sensitivity to hyper-parameter over existing methods. We show that the algorithm is competitive against state-ofthe-art first and second order methods. Our work opens up new avenues for principled algorithmic design built upon the optimal control theory.
Tasks
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08809v1
PDF	https://arxiv.org/pdf/2002.08809v1.pdf
PWC	https://paperswithcode.com/paper/differential-dynamic-programming-neural
Repo
Framework

Illumination adaptive person reid based on teacher-student model and adversarial training


Title	Illumination adaptive person reid based on teacher-student model and adversarial training
Authors	Ziyue Zhang, Richard YD Xu, Shuai Jiang, Yang Li, Congzhentao Huang, Chen Deng
Abstract	Most existing works in Person Re-identification (ReID) focus on settings where illumination either is kept the same or has very little fluctuation. However, the changes in the illumination degree may affect the robustness of a ReID algorithm significantly. To address this problem, we proposed a Two-Stream Network that can separate ReID features from lighting features to enhance ReID performance. Its innovations are threefold: (1) A discriminative entropy loss to ensure the ReID features contain no lighting information. (2) A ReID Teacher model trained by images under “neutral” lighting conditions to guide ReID classification. (3) An illumination Teacher model trained by the differences between the illumination-adjusted and original images to guide illumination classification. We construct two augmented datasets by synthetically changing a set of predefined lighting conditions in two of the most popular ReID benchmarks: Market1501 and DukeMTMC-ReID. Experiments demonstrate that our algorithm outperforms other state-of-the-art works and particularly potent in handling images under extremely low light.
Tasks	Person Re-Identification
Published	2020-02-05
URL	https://arxiv.org/abs/2002.01625v2
PDF	https://arxiv.org/pdf/2002.01625v2.pdf
PWC	https://paperswithcode.com/paper/illumination-adaptive-person-reid-based-on
Repo
Framework

Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settings


Title	Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settings
Authors	C. Shi, S. Zhang, W. Lu, R. Song
Abstract	Reinforcement learning is a general technique that allows an agent to learn an optimal policy and interact with an environment in sequential decision making problems. The goodness of a policy is measured by its value function starting from some initial state. The focus of this paper is to construct confidence intervals (CIs) for a policy’s value in infinite horizon settings where the number of decision points diverges to infinity. We propose to model the action-value state function (Q-function) associated with a policy based on series/sieve method to derive its confidence interval. When the target policy depends on the observed data as well, we propose a SequentiAl Value Evaluation (SAVE) method to recursively update the estimated policy and its value estimator. As long as either the number of trajectories or the number of decision points diverges to infinity, we show that the proposed CI achieves nominal coverage even in cases where the optimal policy is not unique. Simulation studies are conducted to back up our theoretical findings. We apply the proposed method to a dataset from mobile health studies and find that reinforcement learning algorithms could help improve patient’s health status.
Tasks	Decision Making
Published	2020-01-13
URL	https://arxiv.org/abs/2001.04515v1
PDF	https://arxiv.org/pdf/2001.04515v1.pdf
PWC	https://paperswithcode.com/paper/statistical-inference-of-the-value-function
Repo
Framework

Quantifying daseinisation using Shannon entropy


Title	Quantifying daseinisation using Shannon entropy
Authors	Roman Zapatrin
Abstract	Topos formalism for quantum mechanics is interpreted in a broader, information retrieval perspective. Contexts, its basic components, are treated as sources of information. Their interplay, called daseinisation, defined in purely logical terms, is reformulated in terms of two relations: exclusion and preclusion of queries. Then, broadening these options, daseinisation becomes a characteristic of proximity of contexts; to quantify it numerically, Shannon entropy is used.
Tasks	Information Retrieval
Published	2020-02-26
URL	https://arxiv.org/abs/2002.12456v1
PDF	https://arxiv.org/pdf/2002.12456v1.pdf
PWC	https://paperswithcode.com/paper/quantifying-daseinisation-using-shannon
Repo
Framework

Regret Minimization in Stochastic Contextual Dueling Bandits


Title	Regret Minimization in Stochastic Contextual Dueling Bandits
Authors	Aadirupa Saha, Aditya Gopalan
Abstract	We consider the problem of stochastic $K$-armed dueling bandit in the contextual setting, where at each round the learner is presented with a context set of $K$ items, each represented by a $d$-dimensional feature vector, and the goal of the learner is to identify the best arm of each context sets. However, unlike the classical contextual bandit setup, our framework only allows the learner to receive item feedback in terms of their (noisy) pariwise preferences–famously studied as dueling bandits which is practical interests in various online decision making scenarios, e.g. recommender systems, information retrieval, tournament ranking, where it is easier to elicit the relative strength of the items instead of their absolute scores. However, to the best of our knowledge this work is the first to consider the problem of regret minimization of contextual dueling bandits for potentially infinite decision spaces and gives provably optimal algorithms along with a matching lower bound analysis. We present two algorithms for the setup with respective regret guarantees $\tilde O(d\sqrt{T})$ and $\tilde O(\sqrt{dT \log K})$. Subsequently we also show that $\Omega(\sqrt {dT})$ is actually the fundamental performance limit for this problem, implying the optimality of our second algorithm. However the analysis of our first algorithm is comparatively simpler, and it is often shown to outperform the former empirically. Finally, we corroborate all the theoretical results with suitable experiments.
Tasks	Decision Making, Information Retrieval, Recommendation Systems
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08583v1
PDF	https://arxiv.org/pdf/2002.08583v1.pdf
PWC	https://paperswithcode.com/paper/regret-minimization-in-stochastic-contextual
Repo
Framework

Regularizing Reasons for Outfit Evaluation with Gradient Penalty


Title	Regularizing Reasons for Outfit Evaluation with Gradient Penalty
Authors	Xingxing Zou, Zhizhong Li, Ke Bai, Dahua Lin, Waikeung Wong
Abstract	In this paper, we build an outfit evaluation system which provides feedbacks consisting of a judgment with a convincing explanation. The system is trained in a supervised manner which faithfully follows the domain knowledge in fashion. We create the EVALUATION3 dataset which is annotated with judgment, the decisive reason for the judgment, and all corresponding attributes (e.g. print, silhouette, and material \etc.). In the training process, features of all attributes in an outfit are first extracted and then concatenated as the input for the intra-factor compatibility net. Then, the inter-factor compatibility net is used to compute the loss for judgment. We penalize the gradient of judgment loss of so that our Grad-CAM-like reason is regularized to be consistent with the labeled reason. In inference, according to the obtained information of judgment, reason, and attributes, a user-friendly explanation sentence is generated by the pre-defined templates. The experimental results show that the obtained network combines the advantages of high precision and good interpretation.
Tasks
Published	2020-02-02
URL	https://arxiv.org/abs/2002.00460v1
PDF	https://arxiv.org/pdf/2002.00460v1.pdf
PWC	https://paperswithcode.com/paper/regularizing-reasons-for-outfit-evaluation
Repo
Framework

Partially Observed Dynamic Tensor Response Regression


Title	Partially Observed Dynamic Tensor Response Regression
Authors	Jie Zhou, Will Wei Sun, Jingfei Zhang, Lexin Li
Abstract	In modern data science, dynamic tensor data is prevailing in numerous applications. An important task is to characterize the relationship between such dynamic tensor and external covariates. However, the tensor data is often only partially observed, rendering many existing methods inapplicable. In this article, we develop a regression model with partially observed dynamic tensor as the response and external covariates as the predictor. We introduce the low-rank, sparsity and fusion structures on the regression coefficient tensor, and consider a loss function projected over the observed entries. We develop an efficient non-convex alternating updating algorithm, and derive the finite-sample error bound of the actual estimator from each step of our optimization algorithm. Unobserved entries in tensor response have imposed serious challenges. As a result, our proposal differs considerably in terms of estimation algorithm, regularity conditions, as well as theoretical properties, compared to the existing tensor completion or tensor response regression solutions. We illustrate the efficacy of our proposed method using simulations, and two real applications, a neuroimaging dementia study and a digital advertising study.
Tasks
Published	2020-02-22
URL	https://arxiv.org/abs/2002.09735v2
PDF	https://arxiv.org/pdf/2002.09735v2.pdf
PWC	https://paperswithcode.com/paper/partially-observed-dynamic-tensor-response
Repo
Framework

NeCPD: An Online Tensor Decomposition with Optimal Stochastic Gradient Descent


Title	NeCPD: An Online Tensor Decomposition with Optimal Stochastic Gradient Descent
Authors	Ali Anaissi, Basem Suleiman, Seid Miad Zandavi
Abstract	Multi-way data analysis has become an essential tool for capturing underlying structures in higher-order datasets stored in tensor $\mathcal{X} \in \mathbb{R} ^{I_1 \times \dots \times I_N} $. $CANDECOMP/PARAFAC$ (CP) decomposition has been extensively studied and applied to approximate $\mathcal{X}$ by $N$ loading matrices $A^{(1)}, \dots, A^{(N)}$ where $N$ represents the order of the tensor. We propose a new efficient CP decomposition solver named NeCPD for non-convex problem in multi-way online data based on stochastic gradient descent (SGD) algorithm. SGD is very useful in online setting since it allows us to update $\mathcal{X}^{(t+1)}$ in one single step. In terms of global convergence, it is well known that SGD stuck in many saddle points when it deals with non-convex problems. We study the Hessian matrix to identify theses saddle points, and then try to escape them using the perturbation approach which adds little noise to the gradient update step. We further apply Nesterov’s Accelerated Gradient (NAG) method in SGD algorithm to optimally accelerate the convergence rate and compensate Hessian computational delay time per epoch. Experimental evaluation in the field of structural health monitoring using laboratory-based and real-life structural datasets show that our method provides more accurate results compared with existing online tensor analysis methods.
Tasks
Published	2020-03-18
URL	https://arxiv.org/abs/2003.08844v1
PDF	https://arxiv.org/pdf/2003.08844v1.pdf
PWC	https://paperswithcode.com/paper/necpd-an-online-tensor-decomposition-with
Repo
Framework

The Differentially Private Lottery Ticket Mechanism


Title	The Differentially Private Lottery Ticket Mechanism
Authors	Lovedeep Gondara, Ke Wang, Ricardo Silva Carvalho
Abstract	We propose the differentially private lottery ticket mechanism (DPLTM). An end-to-end differentially private training paradigm based on the lottery ticket hypothesis. Using “high-quality winners”, selected via our custom score function, DPLTM significantly improves the privacy-utility trade-off over the state-of-the-art. We show that DPLTM converges faster, allowing for early stopping with reduced privacy budget consumption. We further show that the tickets from DPLTM are transferable across datasets, domains, and architectures. Our extensive evaluation on several public datasets provides evidence to our claims.
Tasks
Published	2020-02-16
URL	https://arxiv.org/abs/2002.11613v1
PDF	https://arxiv.org/pdf/2002.11613v1.pdf
PWC	https://paperswithcode.com/paper/the-differentially-private-lottery-ticket
Repo
Framework

On-Device Information Extraction from SMS using Hybrid Hierarchical Classification


Title	On-Device Information Extraction from SMS using Hybrid Hierarchical Classification
Authors	Shubham Vatsal, Naresh Purre, Sukumar Moharana, Gopi Ramena, Debi Prasanna Mohanty
Abstract	Cluttering of SMS inbox is one of the serious problems that users today face in the digital world where every online login, transaction, along with promotions generate multiple SMS. This problem not only prevents users from searching and navigating messages efficiently but often results in users missing out the relevant information associated with the corresponding SMS like offer codes, payment reminders etc. In this paper, we propose a unique architecture to organize and extract the appropriate information from SMS and further display it in an intuitive template. In the proposed architecture, we use a Hybrid Hierarchical Long Short Term Memory (LSTM)-Convolutional Neural Network (CNN) to categorize SMS into multiple classes followed by a set of entity parsers used to extract the relevant information from the classified message. The architecture using its preprocessing techniques not only takes into account the enormous variations observed in SMS data but also makes it efficient for its on-device (mobile phone) functionalities in terms of inference timing and size.
Tasks
Published	2020-02-03
URL	https://arxiv.org/abs/2002.02755v1
PDF	https://arxiv.org/pdf/2002.02755v1.pdf
PWC	https://paperswithcode.com/paper/on-device-information-extraction-from-sms
Repo
Framework

Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks


Title	Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks
Authors	Joseph Suarez, Yilun Du, Igor Mordach, Phillip Isola
Abstract	Progress in multiagent intelligence research is fundamentally limited by the number and quality of environments available for study. In recent years, simulated games have become a dominant research platform within reinforcement learning, in part due to their accessibility and interpretability. Previous works have targeted and demonstrated success on arcade, first person shooter (FPS), real-time strategy (RTS), and massive online battle arena (MOBA) games. Our work considers massively multiplayer online role-playing games (MMORPGs or MMOs), which capture several complexities of real-world learning that are not well modeled by any other game genre. We present Neural MMO, a massively multiagent game environment inspired by MMOs and discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO. We further demonstrate that standard policy gradient methods and simple baseline models can learn interesting emergent exploration and specialization behaviors in this setting.
Tasks	Policy Gradient Methods
Published	2020-01-31
URL	https://arxiv.org/abs/2001.12004v1
PDF	https://arxiv.org/pdf/2001.12004v1.pdf
PWC	https://paperswithcode.com/paper/neural-mmo-v13-a-massively-multiagent-game
Repo
Framework