Paper Group ANR 5
Rotation Averaging and Strong Duality. ZigZag: A new approach to adaptive online learning. A Learning Based Optimal Human Robot Collaboration with Linear Temporal Logic Constraints. Customer Lifetime Value Prediction Using Embeddings. A New Type of Neurons for Machine Learning. Quality-Efficiency Trade-offs in Machine Learning for Text Processing. …
Rotation Averaging and Strong Duality
| Title | Rotation Averaging and Strong Duality |
| Authors | Anders Eriksson, Carl Olsson, Fredrik Kahl, Tat-Jun Chin |
| Abstract | In this paper we explore the role of duality principles within the problem of rotation averaging, a fundamental task in a wide range of computer vision applications. In its conventional form, rotation averaging is stated as a minimization over multiple rotation constraints. As these constraints are non-convex, this problem is generally considered challenging to solve globally. We show how to circumvent this difficulty through the use of Lagrangian duality. While such an approach is well-known it is normally not guaranteed to provide a tight relaxation. Based on spectral graph theory, we analytically prove that in many cases there is no duality gap unless the noise levels are severe. This allows us to obtain certifiably global solutions to a class of important non-convex problems in polynomial time. We also propose an efficient, scalable algorithm that out-performs general purpose numerical solvers and is able to handle the large problem instances commonly occurring in structure from motion settings. The potential of this proposed method is demonstrated on a number of different problems, consisting of both synthetic and real-world data. |
| Tasks | |
| Published | 2017-05-03 |
| URL | http://arxiv.org/abs/1705.01362v2 |
| http://arxiv.org/pdf/1705.01362v2.pdf | |
| PWC | https://paperswithcode.com/paper/rotation-averaging-and-strong-duality |
| Repo | |
| Framework | |
ZigZag: A new approach to adaptive online learning
| Title | ZigZag: A new approach to adaptive online learning |
| Authors | Dylan J. Foster, Alexander Rakhlin, Karthik Sridharan |
| Abstract | We develop a novel family of algorithms for the online learning setting with regret against any data sequence bounded by the empirical Rademacher complexity of that sequence. To develop a general theory of when this type of adaptive regret bound is achievable we establish a connection to the theory of decoupling inequalities for martingales in Banach spaces. When the hypothesis class is a set of linear functions bounded in some norm, such a regret bound is achievable if and only if the norm satisfies certain decoupling inequalities for martingales. Donald Burkholder’s celebrated geometric characterization of decoupling inequalities (1984) states that such an inequality holds if and only if there exists a special function called a Burkholder function satisfying certain restricted concavity properties. Our online learning algorithms are efficient in terms of queries to this function. We realize our general theory by giving novel efficient algorithms for classes including lp norms, Schatten p-norms, group norms, and reproducing kernel Hilbert spaces. The empirical Rademacher complexity regret bound implies — when used in the i.i.d. setting — a data-dependent complexity bound for excess risk after online-to-batch conversion. To showcase the power of the empirical Rademacher complexity regret bound, we derive improved rates for a supervised learning generalization of the online learning with low rank experts task and for the online matrix prediction task. In addition to obtaining tight data-dependent regret bounds, our algorithms enjoy improved efficiency over previous techniques based on Rademacher complexity, automatically work in the infinite horizon setting, and are scale-free. To obtain such adaptive methods, we introduce novel machinery, and the resulting algorithms are not based on the standard tools of online convex optimization. |
| Tasks | |
| Published | 2017-04-13 |
| URL | http://arxiv.org/abs/1704.04010v1 |
| http://arxiv.org/pdf/1704.04010v1.pdf | |
| PWC | https://paperswithcode.com/paper/zigzag-a-new-approach-to-adaptive-online |
| Repo | |
| Framework | |
A Learning Based Optimal Human Robot Collaboration with Linear Temporal Logic Constraints
| Title | A Learning Based Optimal Human Robot Collaboration with Linear Temporal Logic Constraints |
| Authors | Bo Wu, Bin Hu, Hai Lin |
| Abstract | This paper considers an optimal task allocation problem for human robot collaboration in human robot systems with persistent tasks. Such human robot systems consist of human operators and intelligent robots collaborating with each other to accomplish complex tasks that cannot be done by either part alone. The system objective is to maximize the probability of successfully executing persistent tasks that are formulated as linear temporal logic specifications and minimize the average cost between consecutive visits of a particular proposition. This paper proposes to model the human robot collaboration under a framework with the composition of multiple Markov Decision Process (MDP) with possibly unknown transition probabilities, which characterizes how human cognitive states, such as human trust and fatigue, stochastically change with the robot performance. Under the unknown MDP models, an algorithm is developed to learn the model and obtain an optimal task allocation policy that minimizes the expected average cost for each task cycle and maximizes the probability of satisfying linear temporal logic constraints. Moreover, this paper shows that the difference between the optimal policy based on the learned model and that based on the underlying ground truth model can be bounded by arbitrarily small constant and large confidence level with sufficient samples. The case study of an assembly process demonstrates the effectiveness and benefits of our proposed learning based human robot collaboration. |
| Tasks | |
| Published | 2017-05-31 |
| URL | http://arxiv.org/abs/1706.00007v1 |
| http://arxiv.org/pdf/1706.00007v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-learning-based-optimal-human-robot |
| Repo | |
| Framework | |
Customer Lifetime Value Prediction Using Embeddings
| Title | Customer Lifetime Value Prediction Using Embeddings |
| Authors | Benjamin Paul Chamberlain, Angelo Cardoso, C. H. Bryan Liu, Roberto Pagliari, Marc Peter Deisenroth |
| Abstract | We describe the Customer LifeTime Value (CLTV) prediction system deployed at ASOS.com, a global online fashion retailer. CLTV prediction is an important problem in e-commerce where an accurate estimate of future value allows retailers to effectively allocate marketing spend, identify and nurture high value customers and mitigate exposure to losses. The system at ASOS provides daily estimates of the future value of every customer and is one of the cornerstones of the personalised shopping experience. The state of the art in this domain uses large numbers of handcrafted features and ensemble regressors to forecast value, predict churn and evaluate customer loyalty. Recently, domains including language, vision and speech have shown dramatic advances by replacing handcrafted features with features that are learned automatically from data. We detail the system deployed at ASOS and show that learning feature representations is a promising extension to the state of the art in CLTV modelling. We propose a novel way to generate embeddings of customers, which addresses the issue of the ever changing product catalogue and obtain a significant improvement over an exhaustive set of handcrafted features. |
| Tasks | |
| Published | 2017-03-07 |
| URL | http://arxiv.org/abs/1703.02596v3 |
| http://arxiv.org/pdf/1703.02596v3.pdf | |
| PWC | https://paperswithcode.com/paper/customer-lifetime-value-prediction-using |
| Repo | |
| Framework | |
A New Type of Neurons for Machine Learning
| Title | A New Type of Neurons for Machine Learning |
| Authors | Fenglei Fan, Wenxiang Cong, Ge Wang |
| Abstract | In machine learning, the use of an artificial neural network is the mainstream approach. Such a network consists of layers of neurons. These neurons are of the same type characterized by the two features: (1) an inner product of an input vector and a matching weighting vector of trainable parameters and (2) a nonlinear excitation function. Here we investigate the possibility of replacing the inner product with a quadratic function of the input vector, thereby upgrading the 1st order neuron to the 2nd order neuron, empowering individual neurons, and facilitating the optimization of neural networks. Also, numerical examples are provided to illustrate the feasibility and merits of the 2nd order neurons. Finally, further topics are discussed. |
| Tasks | |
| Published | 2017-04-26 |
| URL | http://arxiv.org/abs/1704.08362v1 |
| http://arxiv.org/pdf/1704.08362v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-new-type-of-neurons-for-machine-learning |
| Repo | |
| Framework | |
Quality-Efficiency Trade-offs in Machine Learning for Text Processing
| Title | Quality-Efficiency Trade-offs in Machine Learning for Text Processing |
| Authors | Ricardo Baeza-Yates, Zeinab Liaghat |
| Abstract | Data mining, machine learning, and natural language processing are powerful techniques that can be used together to extract information from large texts. Depending on the task or problem at hand, there are many different approaches that can be used. The methods available are continuously being optimized, but not all these methods have been tested and compared in a set of problems that can be solved using supervised machine learning algorithms. The question is what happens to the quality of the methods if we increase the training data size from, say, 100 MB to over 1 GB? Moreover, are quality gains worth it when the rate of data processing diminishes? Can we trade quality for time efficiency and recover the quality loss by just being able to process more data? We attempt to answer these questions in a general way for text processing tasks, considering the trade-offs involving training data size, learning time, and quality obtained. We propose a performance trade-off framework and apply it to three important text processing problems: Named Entity Recognition, Sentiment Analysis and Document Classification. These problems were also chosen because they have different levels of object granularity: words, paragraphs, and documents. For each problem, we selected several supervised machine learning algorithms and we evaluated the trade-offs of them on large publicly available data sets (news, reviews, patents). To explore these trade-offs, we use different data subsets of increasing size ranging from 50 MB to several GB. We also consider the impact of the data set and the evaluation technique. We find that the results do not change significantly and that most of the time the best algorithms is the fastest. However, we also show that the results for small data (say less than 100 MB) are different from the results for big data and in those cases the best algorithm is much harder to determine. |
| Tasks | Document Classification, Named Entity Recognition, Sentiment Analysis |
| Published | 2017-11-07 |
| URL | http://arxiv.org/abs/1711.02295v1 |
| http://arxiv.org/pdf/1711.02295v1.pdf | |
| PWC | https://paperswithcode.com/paper/quality-efficiency-trade-offs-in-machine |
| Repo | |
| Framework | |
Zero-order Reverse Filtering
| Title | Zero-order Reverse Filtering |
| Authors | Xin Tao, Chao Zhou, Xiaoyong Shen, Jue Wang, Jiaya Jia |
| Abstract | In this paper, we study an unconventional but practically meaningful reversibility problem of commonly used image filters. We broadly define filters as operations to smooth images or to produce layers via global or local algorithms. And we raise the intriguingly problem if they are reservable to the status before filtering. To answer it, we present a novel strategy to understand general filter via contraction mappings on a metric space. A very simple yet effective zero-order algorithm is proposed. It is able to practically reverse most filters with low computational cost. We present quite a few experiments in the paper and supplementary file to thoroughly verify its performance. This method can also be generalized to solve other inverse problems and enables new applications. |
| Tasks | |
| Published | 2017-04-13 |
| URL | http://arxiv.org/abs/1704.04037v1 |
| http://arxiv.org/pdf/1704.04037v1.pdf | |
| PWC | https://paperswithcode.com/paper/zero-order-reverse-filtering |
| Repo | |
| Framework | |
Knowledge Projection for Deep Neural Networks
| Title | Knowledge Projection for Deep Neural Networks |
| Authors | Zhi Zhang, Guanghan Ning, Zhihai He |
| Abstract | While deeper and wider neural networks are actively pushing the performance limits of various computer vision and machine learning tasks, they often require large sets of labeled data for effective training and suffer from extremely high computational complexity. In this paper, we will develop a new framework for training deep neural networks on datasets with limited labeled samples using cross-network knowledge projection which is able to improve the network performance while reducing the overall computational complexity significantly. Specifically, a large pre-trained teacher network is used to observe samples from the training data. A projection matrix is learned to project this teacher-level knowledge and its visual representations from an intermediate layer of the teacher network to an intermediate layer of a thinner and faster student network to guide and regulate its training process. Both the intermediate layers from the teacher network and the injection layers from the student network are adaptively selected during training by evaluating a joint loss function in an iterative manner. This knowledge projection framework allows us to use crucial knowledge learned by large networks to guide the training of thinner student networks, avoiding over-fitting, achieving better network performance, and significantly reducing the complexity. Extensive experimental results on benchmark datasets have demonstrated that our proposed knowledge projection approach outperforms existing methods, improving accuracy by up to 4% while reducing network complexity by 4 to 10 times, which is very attractive for practical applications of deep neural networks. |
| Tasks | |
| Published | 2017-10-26 |
| URL | http://arxiv.org/abs/1710.09505v1 |
| http://arxiv.org/pdf/1710.09505v1.pdf | |
| PWC | https://paperswithcode.com/paper/knowledge-projection-for-deep-neural-networks |
| Repo | |
| Framework | |
On the Taut String Interpretation of the One-dimensional Rudin-Osher-Fatemi Model: A New Proof, a Fundamental Estimate and Some Applications
| Title | On the Taut String Interpretation of the One-dimensional Rudin-Osher-Fatemi Model: A New Proof, a Fundamental Estimate and Some Applications |
| Authors | Niels Chr. Overgaard |
| Abstract | A new proof of the equivalence of the Taut String Algorithm and the one-dimensional Rudin-Osher-Fatemi model is presented. Based on duality and the projection theorem in Hilbert space, the proof is strictly elementary. Existence and uniqueness of solutions to both denoising models follow as by-products. The standard convergence properties of the denoised signal, as the regularizing parameter tends to zero, are recalled and efficient proofs provided. Moreover, a new and fundamental bound on the denoised signal is derived. This bound implies, among other things, the strong convergence (in the space of functions of bounded variation) of the denoised signal to the insignal as the regularization parameter vanishes. The methods developed in the paper can be modified to cover other interesting applications such as isotonic regression. |
| Tasks | Denoising |
| Published | 2017-10-27 |
| URL | http://arxiv.org/abs/1710.10985v1 |
| http://arxiv.org/pdf/1710.10985v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-taut-string-interpretation-of-the-one |
| Repo | |
| Framework | |
Using Complex Wavelet Transform and Bilateral Filtering for Image Denoising
| Title | Using Complex Wavelet Transform and Bilateral Filtering for Image Denoising |
| Authors | Seyede Mahya Hazavei, Hamid Reza Shahdoosti |
| Abstract | The bilateral filter is a useful nonlinear filter which without smoothing edges, it does spatial averaging. In the literature, the effectiveness of this method for image denoising is shown. In this paper, an extension of this method is proposed which is based on complex wavelet transform. In fact, the bilateral filtering is applied to the low-frequency (approximation) subbands of the decomposed image using complex wavelet transform, while the thresholding approach is applied to the high frequency subbands. Using the bilateral filter in the complex wavelet domain forms a new image denoising framework. Experimental results for real data are provided, by which one can see the effectiveness of the proposed method in eliminating noise. |
| Tasks | Denoising, Image Denoising |
| Published | 2017-02-04 |
| URL | http://arxiv.org/abs/1702.01276v1 |
| http://arxiv.org/pdf/1702.01276v1.pdf | |
| PWC | https://paperswithcode.com/paper/using-complex-wavelet-transform-and-bilateral |
| Repo | |
| Framework | |
Steering Output Style and Topic in Neural Response Generation
| Title | Steering Output Style and Topic in Neural Response Generation |
| Authors | Di Wang, Nebojsa Jojic, Chris Brockett, Eric Nyberg |
| Abstract | We propose simple and flexible training and decoding methods for influencing output style and topic in neural encoder-decoder based language generation. This capability is desirable in a variety of applications, including conversational systems, where successful agents need to produce language in a specific style and generate responses steered by a human puppeteer or external knowledge. We decompose the neural generation process into empirically easier sub-problems: a faithfulness model and a decoding method based on selective-sampling. We also describe training and sampling algorithms that bias the generation process with a specific language style restriction, or a topic restriction. Human evaluation results show that our proposed methods are able to restrict style and topic without degrading output quality in conversational tasks. |
| Tasks | Text Generation |
| Published | 2017-09-09 |
| URL | http://arxiv.org/abs/1709.03010v1 |
| http://arxiv.org/pdf/1709.03010v1.pdf | |
| PWC | https://paperswithcode.com/paper/steering-output-style-and-topic-in-neural |
| Repo | |
| Framework | |
Doubly-Attentive Decoder for Multi-modal Neural Machine Translation
| Title | Doubly-Attentive Decoder for Multi-modal Neural Machine Translation |
| Authors | Iacer Calixto, Qun Liu, Nick Campbell |
| Abstract | We introduce a Multi-modal Neural Machine Translation model in which a doubly-attentive decoder naturally incorporates spatial visual features obtained using pre-trained convolutional neural networks, bridging the gap between image description and translation. Our decoder learns to attend to source-language words and parts of an image independently by means of two separate attention mechanisms as it generates words in the target language. We find that our model can efficiently exploit not just back-translated in-domain multi-modal data but also large general-domain text-only MT corpora. We also report state-of-the-art results on the Multi30k data set. |
| Tasks | Machine Translation |
| Published | 2017-02-04 |
| URL | http://arxiv.org/abs/1702.01287v1 |
| http://arxiv.org/pdf/1702.01287v1.pdf | |
| PWC | https://paperswithcode.com/paper/doubly-attentive-decoder-for-multi-modal |
| Repo | |
| Framework | |
A PDE-based log-agnostic illumination correction algorithm
| Title | A PDE-based log-agnostic illumination correction algorithm |
| Authors | U. A. Nnolim |
| Abstract | This report presents the results of a partial differential equation (PDE)-based image enhancement algorithm, for dynamic range compression and illumination correction in the absence of the logarithmic function. The proposed algorithm combines forward and reverse flows in a PDE-based formulation. The experimental results are compared with algorithms from the literature and indicate comparable performance in most cases. |
| Tasks | Image Enhancement |
| Published | 2017-12-30 |
| URL | http://arxiv.org/abs/1801.00098v2 |
| http://arxiv.org/pdf/1801.00098v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-pde-based-log-agnostic-illumination |
| Repo | |
| Framework | |
Saliency-guided Adaptive Seeding for Supervoxel Segmentation
| Title | Saliency-guided Adaptive Seeding for Supervoxel Segmentation |
| Authors | Ge Gao, Mikko Lauri, Jianwei Zhang, Simone Frintrop |
| Abstract | We propose a new saliency-guided method for generating supervoxels in 3D space. Rather than using an evenly distributed spatial seeding procedure, our method uses visual saliency to guide the process of supervoxel generation. This results in densely distributed, small, and precise supervoxels in salient regions which often contain objects, and larger supervoxels in less salient regions that often correspond to background. Our approach largely improves the quality of the resulting supervoxel segmentation in terms of boundary recall and under-segmentation error on publicly available benchmarks. |
| Tasks | |
| Published | 2017-04-13 |
| URL | http://arxiv.org/abs/1704.04054v2 |
| http://arxiv.org/pdf/1704.04054v2.pdf | |
| PWC | https://paperswithcode.com/paper/saliency-guided-adaptive-seeding-for |
| Repo | |
| Framework | |
Uncertainty-Aware Reinforcement Learning for Collision Avoidance
| Title | Uncertainty-Aware Reinforcement Learning for Collision Avoidance |
| Authors | Gregory Kahn, Adam Villaflor, Vitchyr Pong, Pieter Abbeel, Sergey Levine |
| Abstract | Reinforcement learning can enable complex, adaptive behavior to be learned automatically for autonomous robotic platforms. However, practical deployment of reinforcement learning methods must contend with the fact that the training process itself can be unsafe for the robot. In this paper, we consider the specific case of a mobile robot learning to navigate an a priori unknown environment while avoiding collisions. In order to learn collision avoidance, the robot must experience collisions at training time. However, high-speed collisions, even at training time, could damage the robot. A successful learning method must therefore proceed cautiously, experiencing only low-speed collisions until it gains confidence. To this end, we present an uncertainty-aware model-based learning algorithm that estimates the probability of collision together with a statistical estimate of uncertainty. By formulating an uncertainty-dependent cost function, we show that the algorithm naturally chooses to proceed cautiously in unfamiliar environments, and increases the velocity of the robot in settings where it has high confidence. Our predictive model is based on bootstrapped neural networks using dropout, allowing it to process raw sensory inputs from high-bandwidth sensors such as cameras. Our experimental evaluation demonstrates that our method effectively minimizes dangerous collisions at training time in an obstacle avoidance task for a simulated and real-world quadrotor, and a real-world RC car. Videos of the experiments can be found at https://sites.google.com/site/probcoll. |
| Tasks | |
| Published | 2017-02-03 |
| URL | http://arxiv.org/abs/1702.01182v1 |
| http://arxiv.org/pdf/1702.01182v1.pdf | |
| PWC | https://paperswithcode.com/paper/uncertainty-aware-reinforcement-learning-for |
| Repo | |
| Framework | |
