Paper Group ANR 474
Anti-drift in electronic nose via dimensionality reduction: a discriminative subspace projection approach. Bottom-up Attention, Models of. View-volume Network for Semantic Scene Completion from a Single Depth Image. Federated Learning via Over-the-Air Computation. Approximate Method of Variational Bayesian Matrix Factorization/Completion with Spars …
Anti-drift in electronic nose via dimensionality reduction: a discriminative subspace projection approach
| Title | Anti-drift in electronic nose via dimensionality reduction: a discriminative subspace projection approach |
| Authors | Zhengkun Yi, Cheng Li |
| Abstract | Sensor drift is a well-known issue in the field of sensors and measurement and has plagued the sensor community for many years. In this paper, we propose a sensor drift correction method to deal with the sensor drift problem. Specifically, we propose a discriminative subspace projection approach for sensor drift reduction in electronic noses. The proposed method inherits the merits of the subspace projection method called domain regularized component analysis. Moreover, the proposed method takes the source data label information into consideration, which minimizes the within-class variance of the projected source samples and at the same time maximizes the between-class variance. The label information is exploited to avoid overlapping of samples with different labels in the subspace. Experiments on two sensor drift datasets have shown the effectiveness of the proposed approach. |
| Tasks | Dimensionality Reduction |
| Published | 2018-12-14 |
| URL | http://arxiv.org/abs/1901.02321v1 |
| http://arxiv.org/pdf/1901.02321v1.pdf | |
| PWC | https://paperswithcode.com/paper/anti-drift-in-electronic-nose-via |
| Repo | |
| Framework | |
Bottom-up Attention, Models of
| Title | Bottom-up Attention, Models of |
| Authors | Ali Borji, Hamed R. Tavakoli, Zoya Bylinskii |
| Abstract | In this review, we examine the recent progress in saliency prediction and proposed several avenues for future research. In spite of tremendous efforts and huge progress, there is still room for improvement in terms finer-grained analysis of deep saliency models, evaluation measures, datasets, annotation methods, cognitive studies, and new applications. This chapter will appear in Encyclopedia of Computational Neuroscience. |
| Tasks | Saliency Prediction |
| Published | 2018-10-11 |
| URL | http://arxiv.org/abs/1810.05680v3 |
| http://arxiv.org/pdf/1810.05680v3.pdf | |
| PWC | https://paperswithcode.com/paper/bottom-up-attention-models-of |
| Repo | |
| Framework | |
View-volume Network for Semantic Scene Completion from a Single Depth Image
| Title | View-volume Network for Semantic Scene Completion from a Single Depth Image |
| Authors | Yu-Xiao Guo, Xin Tong |
| Abstract | We introduce a View-Volume convolutional neural network (VVNet) for inferring the occupancy and semantic labels of a volumetric 3D scene from a single depth image. The VVNet concatenates a 2D view CNN and a 3D volume CNN with a differentiable projection layer. Given a single RGBD image, our method extracts the detailed geometric features from the input depth image with a 2D view CNN and then projects the features into a 3D volume according to the input depth map via a projection layer. After that, we learn the 3D context information of the scene with a 3D volume CNN for computing the result volumetric occupancy and semantic labels. With combined 2D and 3D representations, the VVNet efficiently reduces the computational cost, enables feature extraction from multi-channel high resolution inputs, and thus significantly improves the result accuracy. We validate our method and demonstrate its efficiency and effectiveness on both synthetic SUNCG and real NYU dataset. |
| Tasks | |
| Published | 2018-06-14 |
| URL | http://arxiv.org/abs/1806.05361v1 |
| http://arxiv.org/pdf/1806.05361v1.pdf | |
| PWC | https://paperswithcode.com/paper/view-volume-network-for-semantic-scene |
| Repo | |
| Framework | |
Federated Learning via Over-the-Air Computation
| Title | Federated Learning via Over-the-Air Computation |
| Authors | Kai Yang, Tao Jiang, Yuanming Shi, Zhi Ding |
| Abstract | The stringent requirements for low-latency and privacy of the emerging high-stake applications with intelligent devices such as drones and smart vehicles make the cloud computing inapplicable in these scenarios. Instead, edge machine learning becomes increasingly attractive for performing training and inference directly at network edges without sending data to a centralized data center. This stimulates a nascent field termed as federated learning for training a machine learning model on computation, storage, energy and bandwidth limited mobile devices in a distributed manner. To preserve data privacy and address the issues of unbalanced and non-IID data points across different devices, the federated averaging algorithm has been proposed for global model aggregation by computing the weighted average of locally updated model at each selected device. However, the limited communication bandwidth becomes the main bottleneck for aggregating the locally computed updates. We thus propose a novel over-the-air computation based approach for fast global model aggregation via exploring the superposition property of a wireless multiple-access channel. This is achieved by joint device selection and beamforming design, which is modeled as a sparse and low-rank optimization problem to support efficient algorithms design. To achieve this goal, we provide a difference-of-convex-functions (DC) representation for the sparse and low-rank function to enhance sparsity and accurately detect the fixed-rank constraint in the procedure of device selection. A DC algorithm is further developed to solve the resulting DC program with global convergence guarantees. The algorithmic advantages and admirable performance of the proposed methodologies are demonstrated through extensive numerical results. |
| Tasks | |
| Published | 2018-12-31 |
| URL | http://arxiv.org/abs/1812.11750v3 |
| http://arxiv.org/pdf/1812.11750v3.pdf | |
| PWC | https://paperswithcode.com/paper/federated-learning-via-over-the-air |
| Repo | |
| Framework | |
Approximate Method of Variational Bayesian Matrix Factorization/Completion with Sparse Prior
| Title | Approximate Method of Variational Bayesian Matrix Factorization/Completion with Sparse Prior |
| Authors | Ryota Kawasumi, Koujin Takeda |
| Abstract | We derive analytical expression of matrix factorization/completion solution by variational Bayes method, under the assumption that observed matrix is originally the product of low-rank dense and sparse matrices with additive noise. We assume the prior of sparse matrix is Laplace distribution by taking matrix sparsity into consideration. Then we use several approximations for derivation of matrix factorization/completion solution. By our solution, we also numerically evaluate the performance of sparse matrix reconstruction in matrix factorization, and completion of missing matrix element in matrix completion. |
| Tasks | Matrix Completion |
| Published | 2018-03-14 |
| URL | http://arxiv.org/abs/1803.06234v1 |
| http://arxiv.org/pdf/1803.06234v1.pdf | |
| PWC | https://paperswithcode.com/paper/approximate-method-of-variational-bayesian |
| Repo | |
| Framework | |
Identifying Generalization Properties in Neural Networks
| Title | Identifying Generalization Properties in Neural Networks |
| Authors | Huan Wang, Nitish Shirish Keskar, Caiming Xiong, Richard Socher |
| Abstract | While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We connect model generalization with the local property of a solution under the PAC-Bayes paradigm. In particular, we prove that model generalization ability is related to the Hessian, the higher-order “smoothness” terms characterized by the Lipschitz constant of the Hessian, and the scales of the parameters. Guided by the proof, we propose a metric to score the generalization capability of the model, as well as an algorithm that optimizes the perturbed model accordingly. |
| Tasks | |
| Published | 2018-09-19 |
| URL | http://arxiv.org/abs/1809.07402v1 |
| http://arxiv.org/pdf/1809.07402v1.pdf | |
| PWC | https://paperswithcode.com/paper/identifying-generalization-properties-in |
| Repo | |
| Framework | |
Training Compact Neural Networks with Binary Weights and Low Precision Activations
| Title | Training Compact Neural Networks with Binary Weights and Low Precision Activations |
| Authors | Bohan Zhuang, Chunhua Shen, Ian Reid |
| Abstract | In this paper, we propose to train a network with binary weights and low-bitwidth activations, designed especially for mobile devices with limited power consumption. Most previous works on quantizing CNNs uncritically assume the same architecture, though with reduced precision. However, we take the view that for best performance it is possible (and even likely) that a different architecture may be better suited to dealing with low precision weights and activations. Specifically, we propose a “network expansion” strategy in which we aggregate a set of homogeneous low-precision branches to implicitly reconstruct the full-precision intermediate feature maps. Moreover, we also propose a group-wise feature approximation strategy which is very flexible and highly accurate. Experiments on ImageNet classification tasks demonstrate the superior performance of the proposed model, named Group-Net, over various popular architectures. In particular, with binary weights and activations, we outperform the previous best binary neural network in terms of accuracy as well as saving more than 5 times computational complexity on ImageNet with ResNet-18 and ResNet-50. |
| Tasks | |
| Published | 2018-08-08 |
| URL | http://arxiv.org/abs/1808.02631v1 |
| http://arxiv.org/pdf/1808.02631v1.pdf | |
| PWC | https://paperswithcode.com/paper/training-compact-neural-networks-with-binary |
| Repo | |
| Framework | |
Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines
| Title | Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines |
| Authors | Fenggen Yu, Yan Zhang, Kai Xu, Ali Mahdavi-Amiri, Hao Zhang |
| Abstract | We present a semi-supervised co-analysis method for learning 3D shape styles from projected feature lines, achieving style patch localization with only weak supervision. Given a collection of 3D shapes spanning multiple object categories and styles, we perform style co-analysis over projected feature lines of each 3D shape and then backproject the learned style features onto the 3D shapes. Our core analysis pipeline starts with mid-level patch sampling and pre-selection of candidate style patches. Projective features are then encoded via patch convolution. Multi-view feature integration and style clustering are carried out under the framework of partially shared latent factor (PSLF) learning, a multi-view feature learning scheme. PSLF achieves effective multi-view feature fusion by distilling and exploiting consistent and complementary feature information from multiple views, while also selecting style patches from the candidates. Our style analysis approach supports both unsupervised and semi-supervised analysis. For the latter, our method accepts both user-specified shape labels and style-ranked triplets as clustering constraints.We demonstrate results from 3D shape style analysis and patch localization as well as improvements over state-of-the-art methods. We also present several applications enabled by our style analysis. |
| Tasks | |
| Published | 2018-04-18 |
| URL | http://arxiv.org/abs/1804.06579v1 |
| http://arxiv.org/pdf/1804.06579v1.pdf | |
| PWC | https://paperswithcode.com/paper/semi-supervised-co-analysis-of-3d-shape |
| Repo | |
| Framework | |
Regularisation of Neural Networks by Enforcing Lipschitz Continuity
| Title | Regularisation of Neural Networks by Enforcing Lipschitz Continuity |
| Authors | Henry Gouk, Eibe Frank, Bernhard Pfahringer, Michael Cree |
| Abstract | We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs. To this end, we provide a simple technique for computing an upper bound to the Lipschitz constant of a feed forward neural network composed of commonly used layer types and demonstrate inaccuracies in previous work on this topic. Our technique is then used to formulate training a neural network with a bounded Lipschitz constant as a constrained optimisation problem that can be solved using projected stochastic gradient methods. Our evaluation study shows that, in isolation, our method performs comparatively to state-of-the-art regularisation techniques. Moreover, when combined with existing approaches to regularising neural networks the performance gains are cumulative. We also provide evidence that the hyperparameters are intuitive to tune and demonstrate how the choice of norm for computing the Lipschitz constant impacts the resulting model. |
| Tasks | |
| Published | 2018-04-12 |
| URL | http://arxiv.org/abs/1804.04368v2 |
| http://arxiv.org/pdf/1804.04368v2.pdf | |
| PWC | https://paperswithcode.com/paper/regularisation-of-neural-networks-by |
| Repo | |
| Framework | |
Generalized Byzantine-tolerant SGD
| Title | Generalized Byzantine-tolerant SGD |
| Authors | Cong Xie, Oluwasanmi Koyejo, Indranil Gupta |
| Abstract | We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience properties of these aggregation rules. Empirical analysis shows that the proposed techniques outperform current approaches for realistic use cases and Byzantine attack scenarios. |
| Tasks | |
| Published | 2018-02-27 |
| URL | http://arxiv.org/abs/1802.10116v3 |
| http://arxiv.org/pdf/1802.10116v3.pdf | |
| PWC | https://paperswithcode.com/paper/generalized-byzantine-tolerant-sgd |
| Repo | |
| Framework | |
Planning with Pixels in (Almost) Real Time
| Title | Planning with Pixels in (Almost) Real Time |
| Authors | Wilmer Bandres, Blai Bonet, Hector Geffner |
| Abstract | Recently, width-based planning methods have been shown to yield state-of-the-art results in the Atari 2600 video games. For this, the states were associated with the (RAM) memory states of the simulator. In this work, we consider the same planning problem but using the screen instead. By using the same visual inputs, the planning results can be compared with those of humans and learning methods. We show that the planning approach, out of the box and without training, results in scores that compare well with those obtained by humans and learning methods, and moreover, by developing an episodic, rollout version of the IW(k) algorithm, we show that such scores can be obtained in almost real time. |
| Tasks | |
| Published | 2018-01-10 |
| URL | http://arxiv.org/abs/1801.03354v1 |
| http://arxiv.org/pdf/1801.03354v1.pdf | |
| PWC | https://paperswithcode.com/paper/planning-with-pixels-in-almost-real-time |
| Repo | |
| Framework | |
Toward Scalable Verification for Safety-Critical Deep Networks
| Title | Toward Scalable Verification for Safety-Critical Deep Networks |
| Authors | Lindsey Kuper, Guy Katz, Justin Gottschlich, Kyle Julian, Clark Barrett, Mykel Kochenderfer |
| Abstract | The increasing use of deep neural networks for safety-critical applications, such as autonomous driving and flight control, raises concerns about their safety and reliability. Formal verification can address these concerns by guaranteeing that a deep learning system operates as intended, but the state of the art is limited to small systems. In this work-in-progress report we give an overview of our work on mitigating this difficulty, by pursuing two complementary directions: devising scalable verification techniques, and identifying design choices that result in deep learning systems that are more amenable to verification. |
| Tasks | Autonomous Driving |
| Published | 2018-01-18 |
| URL | http://arxiv.org/abs/1801.05950v2 |
| http://arxiv.org/pdf/1801.05950v2.pdf | |
| PWC | https://paperswithcode.com/paper/toward-scalable-verification-for-safety |
| Repo | |
| Framework | |
Deep Learning to Detect Redundant Method Comments
| Title | Deep Learning to Detect Redundant Method Comments |
| Authors | Annie Louis, Santanu Kumar Dash, Earl T. Barr, Charles Sutton |
| Abstract | Comments in software are critical for maintenance and reuse. But apart from prescriptive advice, there is little practical support or quantitative understanding of what makes a comment useful. In this paper, we introduce the task of identifying comments which are uninformative about the code they are meant to document. To address this problem, we introduce the notion of comment entailment from code, high entailment indicating that a comment’s natural language semantics can be inferred directly from the code. Although not all entailed comments are low quality, comments that are too easily inferred, for example, comments that restate the code, are widely discouraged by authorities on software style. Based on this, we develop a tool called CRAIC which scores method-level comments for redundancy. Highly redundant comments can then be expanded or alternately removed by the developer. CRAIC uses deep language models to exploit large software corpora without requiring expensive manual annotations of entailment. We show that CRAIC can perform the comment entailment task with good agreement with human judgements. Our findings also have implications for documentation tools. For example, we find that common tags in Javadoc are at least two times more predictable from code than non-Javadoc sentences, suggesting that Javadoc tags are less informative than more free-form comments |
| Tasks | |
| Published | 2018-06-12 |
| URL | http://arxiv.org/abs/1806.04616v1 |
| http://arxiv.org/pdf/1806.04616v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-to-detect-redundant-method |
| Repo | |
| Framework | |
Improvements on Hindsight Learning
| Title | Improvements on Hindsight Learning |
| Authors | Ameet Deshpande, Srikanth Sarma, Ashutosh Jha, Balaraman Ravindran |
| Abstract | Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising way to train an agent to perform goal-directed tasks is to use Hindsight Learning approaches. In these approaches, even when an agent fails to reach the desired goal, the agent learns to reach the goal it achieved instead. Doing this over multiple trajectories while generalizing the policy learned from the achieved goals, the agent learns a goal conditioned policy to reach any goal. One such approach is Hindsight Experience replay which uses an off-policy Reinforcement Learning algorithm to learn a goal conditioned policy. In this approach, a replay of the past transitions happens in a uniformly random fashion. Another approach is to use a Hindsight version of the policy gradients to directly learn a policy. In this work, we discuss different ways to replay past transitions to improve learning in hindsight experience replay focusing on prioritized variants in particular. Also, we implement the Hindsight Policy gradient methods to robotic tasks. |
| Tasks | Policy Gradient Methods |
| Published | 2018-09-16 |
| URL | http://arxiv.org/abs/1809.06719v2 |
| http://arxiv.org/pdf/1809.06719v2.pdf | |
| PWC | https://paperswithcode.com/paper/improvements-on-hindsight-learning |
| Repo | |
| Framework | |
Learning Vision-based Cohesive Flight in Drone Swarms
| Title | Learning Vision-based Cohesive Flight in Drone Swarms |
| Authors | Fabian Schilling, Julien Lecoeur, Fabrizio Schiano, Dario Floreano |
| Abstract | This paper presents a data-driven approach to learning vision-based collective behavior from a simple flocking algorithm. We simulate a swarm of quadrotor drones and formulate the controller as a regression problem in which we generate 3D velocity commands directly from raw camera images. The dataset is created by simultaneously acquiring omnidirectional images and computing the corresponding control command from the flocking algorithm. We show that a convolutional neural network trained on the visual inputs of the drone can learn not only robust collision avoidance but also coherence of the flock in a sample-efficient manner. The neural controller effectively learns to localize other agents in the visual input, which we show by visualizing the regions with the most influence on the motion of an agent. This weakly supervised saliency map can be computed efficiently and may be used as a prior for subsequent detection and relative localization of other agents. We remove the dependence on sharing positions among flock members by taking only local visual information into account for control. Our work can therefore be seen as the first step towards a fully decentralized, vision-based flock without the need for communication or visual markers to aid detection of other agents. |
| Tasks | |
| Published | 2018-09-03 |
| URL | http://arxiv.org/abs/1809.00543v1 |
| http://arxiv.org/pdf/1809.00543v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-vision-based-cohesive-flight-in |
| Repo | |
| Framework | |
