Paper Group ANR 544
Scalable Meta-Learning for Bayesian Optimization. Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learning. Learning Representations of Spatial Displacement through Sensorimotor Prediction. Modeling Global Dynamics from Local Snapshots with Deep Generative Neural Networks. Lidar Cloud Detection with Fully …
Scalable Meta-Learning for Bayesian Optimization
| Title | Scalable Meta-Learning for Bayesian Optimization |
| Authors | Matthias Feurer, Benjamin Letham, Eytan Bakshy |
| Abstract | Bayesian optimization has become a standard technique for hyperparameter optimization, including data-intensive models such as deep neural networks that may take days or weeks to train. We consider the setting where previous optimization runs are available, and we wish to use their results to warm-start a new optimization run. We develop an ensemble model that can incorporate the results of past optimization runs, while avoiding the poor scaling that comes with putting all results into a single Gaussian process model. The ensemble combines models from past runs according to estimates of their generalization performance on the current optimization. Results from a large collection of hyperparameter optimization benchmark problems and from optimization of a production computer vision platform at Facebook show that the ensemble can substantially reduce the time it takes to obtain near-optimal configurations, and is useful for warm-starting expensive searches or running quick re-optimizations. |
| Tasks | Hyperparameter Optimization, Meta-Learning |
| Published | 2018-02-06 |
| URL | http://arxiv.org/abs/1802.02219v1 |
| http://arxiv.org/pdf/1802.02219v1.pdf | |
| PWC | https://paperswithcode.com/paper/scalable-meta-learning-for-bayesian |
| Repo | |
| Framework | |
Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learning
| Title | Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learning |
| Authors | Jiazhuo Wang, Jason Xu, Xuejun Wang |
| Abstract | Deep learning has achieved impressive results on many problems. However, it requires high degree of expertise or a lot of experience to tune well the hyperparameters, and such manual tuning process is likely to be biased. Moreover, it is not practical to try out as many different hyperparameter configurations in deep learning as in other machine learning scenarios, because evaluating each single hyperparameter configuration in deep learning would mean training a deep neural network, which usually takes quite long time. Hyperband algorithm achieves state-of-the-art performance on various hyperparameter optimization problems in the field of deep learning. However, Hyperband algorithm does not utilize history information of previous explored hyperparameter configurations, thus the solution found is suboptimal. We propose to combine Hyperband algorithm with Bayesian optimization (which does not ignore history when sampling next trial configuration). Experimental results show that our combination approach is superior to other hyperparameter optimization approaches including Hyperband algorithm. |
| Tasks | Hyperparameter Optimization |
| Published | 2018-01-05 |
| URL | http://arxiv.org/abs/1801.01596v1 |
| http://arxiv.org/pdf/1801.01596v1.pdf | |
| PWC | https://paperswithcode.com/paper/combination-of-hyperband-and-bayesian |
| Repo | |
| Framework | |
Learning Representations of Spatial Displacement through Sensorimotor Prediction
| Title | Learning Representations of Spatial Displacement through Sensorimotor Prediction |
| Authors | Michael Garcia Ortiz, Alban Laflaquière |
| Abstract | Robots act in their environment through sequences of continuous motor commands. Because of the dimensionality of the motor space, as well as the infinite possible combinations of successive motor commands, agents need compact representations that capture the structure of the resulting displacements. In the case of an autonomous agent with no a priori knowledge about its sensorimotor apparatus, this compression has to be learned. We propose to use Recurrent Neural Networks to encode motor sequences into a compact representation, which is used to predict the consequence of motor sequences in term of sensory changes. We show that sensory prediction can successfully guide the compression of motor sequences into representations that are organized topologically in term of spatial displacement. |
| Tasks | |
| Published | 2018-05-16 |
| URL | http://arxiv.org/abs/1805.06250v1 |
| http://arxiv.org/pdf/1805.06250v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-representations-of-spatial |
| Repo | |
| Framework | |
Modeling Global Dynamics from Local Snapshots with Deep Generative Neural Networks
| Title | Modeling Global Dynamics from Local Snapshots with Deep Generative Neural Networks |
| Authors | Scott Gigante, David van Dijk, Kevin Moon, Alexander Strzalkowski, Guy Wolf, Smita Krishnaswamy |
| Abstract | Complex high dimensional stochastic dynamic systems arise in many applications in the natural sciences and especially biology. However, while these systems are difficult to describe analytically, “snapshot” measurements that sample the output of the system are often available. In order to model the dynamics of such systems given snapshot data, or local transitions, we present a deep neural network framework we call Dynamics Modeling Network or DyMoN. DyMoN is a neural network framework trained as a deep generative Markov model whose next state is a probability distribution based on the current state. DyMoN is trained using samples of current and next-state pairs, and thus does not require longitudinal measurements. We show the advantage of DyMoN over shallow models such as Kalman filters and hidden Markov models, and other deep models such as recurrent neural networks in its ability to embody the dynamics (which can be studied via perturbation of the neural network) and generate longitudinal hypothetical trajectories. We perform three case studies in which we apply DyMoN to different types of biological systems and extract features of the dynamics in each case by examining the learned model. |
| Tasks | Dimensionality Reduction |
| Published | 2018-02-10 |
| URL | https://arxiv.org/abs/1802.03497v5 |
| https://arxiv.org/pdf/1802.03497v5.pdf | |
| PWC | https://paperswithcode.com/paper/modeling-dynamics-of-biological-systems-with |
| Repo | |
| Framework | |
Lidar Cloud Detection with Fully Convolutional Networks
| Title | Lidar Cloud Detection with Fully Convolutional Networks |
| Authors | Erol Cromwell, Donna Flynn |
| Abstract | In this contribution, we present a novel approach for segmenting laser radar (lidar) imagery into geometric time-height cloud locations with a fully convolutional network (FCN). We describe a semi-supervised learning method to train the FCN by: pre-training the classification layers of the FCN with image-level annotations, pre-training the entire FCN with the cloud locations of the MPLCMASK cloud mask algorithm, and fully supervised learning with hand-labeled cloud locations. We show the model achieves higher levels of cloud identification compared to the cloud mask algorithm implementation. |
| Tasks | Cloud Detection |
| Published | 2018-05-02 |
| URL | http://arxiv.org/abs/1805.00928v2 |
| http://arxiv.org/pdf/1805.00928v2.pdf | |
| PWC | https://paperswithcode.com/paper/lidar-cloud-detection-with-fully |
| Repo | |
| Framework | |
Demystifying Deep Learning: A Geometric Approach to Iterative Projections
| Title | Demystifying Deep Learning: A Geometric Approach to Iterative Projections |
| Authors | Ashkan Panahi, Hamid Krim, Liyi Dai |
| Abstract | Parametric approaches to Learning, such as deep learning (DL), are highly popular in nonlinear regression, in spite of their extremely difficult training with their increasing complexity (e.g. number of layers in DL). In this paper, we present an alternative semi-parametric framework which foregoes the ordinarily required feedback, by introducing the novel idea of geometric regularization. We show that certain deep learning techniques such as residual network (ResNet) architecture are closely related to our approach. Hence, our technique can be used to analyze these types of deep learning. Moreover, we present preliminary results which confirm that our approach can be easily trained to obtain complex structures. |
| Tasks | |
| Published | 2018-03-22 |
| URL | http://arxiv.org/abs/1803.08416v1 |
| http://arxiv.org/pdf/1803.08416v1.pdf | |
| PWC | https://paperswithcode.com/paper/demystifying-deep-learning-a-geometric |
| Repo | |
| Framework | |
Improvement of K Mean Clustering Algorithm Based on Density
| Title | Improvement of K Mean Clustering Algorithm Based on Density |
| Authors | Su Chang, Xu Zhenzong, Gao Xuan |
| Abstract | The purpose of this paper is to improve the traditional K-means algorithm. In the traditional K mean clustering algorithm, the initial clustering centers are generated randomly in the data set. It is easy to fall into the local minimum solution when the initial cluster centers are randomly generated. The initial clustering center selected by K-means clustering algorithm which based on density is more representative. The experimental results show that the improved K clustering algorithm can eliminate the dependence on the initial cluster, and the accuracy of clustering is improved. |
| Tasks | |
| Published | 2018-10-09 |
| URL | http://arxiv.org/abs/1810.04559v1 |
| http://arxiv.org/pdf/1810.04559v1.pdf | |
| PWC | https://paperswithcode.com/paper/improvement-of-k-mean-clustering-algorithm |
| Repo | |
| Framework | |
Mobile Sound Recognition for the Deaf and Hard of Hearing
| Title | Mobile Sound Recognition for the Deaf and Hard of Hearing |
| Authors | Leonardo A. Fanzeres, Adriana S. Vivacqua, Luiz W. P. Biscainho |
| Abstract | Human perception of surrounding events is strongly dependent on audio cues. Thus, acoustic insulation can seriously impact situational awareness. We present an exploratory study in the domain of assistive computing, eliciting requirements and presenting solutions to problems found in the development of an environmental sound recognition system, which aims to assist deaf and hard of hearing people in the perception of sounds. To take advantage of smartphones computational ubiquity, we propose a system that executes all processing on the device itself, from audio features extraction to recognition and visual presentation of results. Our application also presents the confidence level of the classification to the user. A test of the system conducted with deaf users provided important and inspiring feedback from participants. |
| Tasks | |
| Published | 2018-10-19 |
| URL | http://arxiv.org/abs/1810.08707v1 |
| http://arxiv.org/pdf/1810.08707v1.pdf | |
| PWC | https://paperswithcode.com/paper/mobile-sound-recognition-for-the-deaf-and |
| Repo | |
| Framework | |
Reconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach
| Title | Reconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach |
| Authors | Ran Wang, Yao Wang, Adeen Flinker |
| Abstract | The superior temporal gyrus (STG) region of cortex critically contributes to speech recognition. In this work, we show that a proposed WaveNet, with limited available data, is able to reconstruct speech stimuli from STG intracranial recordings. We further investigate the impulse response of the fitted model for each recording electrode and observe phoneme level temporospectral tuning properties for the recorded area of cortex. This discovery is consistent with previous studies implicating the posterior STG (pSTG) in a phonetic representation of speech and provides detailed acoustic features that certain electrode sites possibly extract during speech recognition. |
| Tasks | Speech Recognition |
| Published | 2018-11-06 |
| URL | http://arxiv.org/abs/1811.02694v2 |
| http://arxiv.org/pdf/1811.02694v2.pdf | |
| PWC | https://paperswithcode.com/paper/reconstructing-speech-stimuli-from-human |
| Repo | |
| Framework | |
Unsupervised Dual-Cascade Learning with Pseudo-Feedback Distillation for Query-based Extractive Summarization
| Title | Unsupervised Dual-Cascade Learning with Pseudo-Feedback Distillation for Query-based Extractive Summarization |
| Authors | Haggai Roitman, Guy Feigenblat, David Konopnicki, Doron Cohen, Odellia Boni |
| Abstract | We propose Dual-CES – a novel unsupervised, query-focused, multi-document extractive summarizer. Dual-CES is designed to better handle the tradeoff between saliency and focus in summarization. To this end, Dual-CES employs a two-step dual-cascade optimization approach with saliency-based pseudo-feedback distillation. Overall, Dual-CES significantly outperforms all other state-of-the-art unsupervised alternatives. Dual-CES is even shown to be able to outperform strong supervised summarizers. |
| Tasks | Query-Based Extractive Summarization |
| Published | 2018-11-01 |
| URL | http://arxiv.org/abs/1811.00436v1 |
| http://arxiv.org/pdf/1811.00436v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-dual-cascade-learning-with |
| Repo | |
| Framework | |
Real-Time End-to-End Action Detection with Two-Stream Networks
| Title | Real-Time End-to-End Action Detection with Two-Stream Networks |
| Authors | Alaaeldin El-Nouby, Graham W. Taylor |
| Abstract | Two-stream networks have been very successful for solving the problem of action detection. However, prior work using two-stream networks train both streams separately, which prevents the network from exploiting regularities between the two streams. Moreover, unlike the visual stream, the dominant forms of optical flow computation typically do not maximally exploit GPU parallelism. We present a real-time end-to-end trainable two-stream network for action detection. First, we integrate the optical flow computation in our framework by using Flownet2. Second, we apply early fusion for the two streams and train the whole pipeline jointly end-to-end. Finally, for better network initialization, we transfer from the task of action recognition to action detection by pre-training our framework using the recently released large-scale Kinetics dataset. Our experimental results show that training the pipeline jointly end-to-end with fine-tuning the optical flow for the objective of action detection improves detection performance significantly. Additionally, we observe an improvement when initializing with parameters pre-trained using Kinetics. Last, we show that by integrating the optical flow computation, our framework is more efficient, running at real-time speeds (up to 31 fps). |
| Tasks | Action Detection, Optical Flow Estimation, Temporal Action Localization |
| Published | 2018-02-23 |
| URL | http://arxiv.org/abs/1802.08362v1 |
| http://arxiv.org/pdf/1802.08362v1.pdf | |
| PWC | https://paperswithcode.com/paper/real-time-end-to-end-action-detection-with |
| Repo | |
| Framework | |
Ensemble of Deep Learned Features for Melanoma Classification
| Title | Ensemble of Deep Learned Features for Melanoma Classification |
| Authors | Loris Nanni, Alessandra Lumini, Stefano Ghidoni |
| Abstract | The aim of this work is to propose an ensemble of descriptors for Melanoma Classification, whose performance has been evaluated on validation and test datasets of the melanoma challenge 2018. The system proposed here achieves a strong discriminative power thanks to the combination of multiple descriptors. The proposed system represents a very simple yet effective way of boosting the performance of trained CNNs by composing multiple CNNs into an ensemble and combining scores by sum rule. Several types of ensembles are considered, with different CNN architectures along with different learning parameter sets. Moreover CNN are used as feature extractors: an input image is processed by a trained CNN and the response of a particular layer (usually the classification layer, but also internal layers can be employed) is treated as a descriptor for the image and used for training a set of Support Vector Machines (SVM). |
| Tasks | |
| Published | 2018-07-20 |
| URL | http://arxiv.org/abs/1807.08008v1 |
| http://arxiv.org/pdf/1807.08008v1.pdf | |
| PWC | https://paperswithcode.com/paper/ensemble-of-deep-learned-features-for |
| Repo | |
| Framework | |
Building Efficient CNN Architecture for Offline Handwritten Chinese Character Recognition
| Title | Building Efficient CNN Architecture for Offline Handwritten Chinese Character Recognition |
| Authors | Zhiyuan Li, Nanjun Teng, Min Jin, Huaxiang Lu |
| Abstract | Deep convolutional networks based methods have brought great breakthrough in images classification, which provides an end-to-end solution for handwritten Chinese character recognition(HCCR) problem through learning discriminative features automatically. Nevertheless, state-of-the-art CNNs appear to incur huge computation cost, and require the storage of a large number of parameters especially in fully connected layers, which is difficult to deploy such networks into alternative hardware device with the limit of computation amount. To solve the storage problem, we propose a novel technique called Global Weighted Arverage Pooling for reducing the parameters in fully connected layer without loss in accuracy. Besides, we implement a cascaded model in single CNN by adding mid output layer to complete recognition as early as possible, which reduces average inference time significantly. Experiments were performed on the ICDAR-2013 offline HCCR dataset, and it is found that the proposed approach only needs 6.9ms for classfying a chracter image on average, and achieves the state-of-the-art accuracy of 97.1% while requiring only 3.3MB for storage. |
| Tasks | Offline Handwritten Chinese Character Recognition |
| Published | 2018-04-04 |
| URL | http://arxiv.org/abs/1804.01259v2 |
| http://arxiv.org/pdf/1804.01259v2.pdf | |
| PWC | https://paperswithcode.com/paper/building-efficient-cnn-architecture-for |
| Repo | |
| Framework | |
Accelerating Message Passing for MAP with Benders Decomposition
| Title | Accelerating Message Passing for MAP with Benders Decomposition |
| Authors | Julian Yarkony, Shaofei Wang |
| Abstract | We introduce a novel mechanism to tighten the local polytope relaxation for MAP inference in Markov random fields with low state space variables. We consider a surjection of the variables to a set of hyper-variables and apply the local polytope relaxation over these hyper-variables. The state space of each individual hyper-variable is constructed to be enumerable while the vector product of pairs is not easily enumerable making message passing inference intractable. To circumvent the difficulty of enumerating the vector product of state spaces of hyper-variables we introduce a novel Benders decomposition approach. This produces an upper envelope describing the message constructed from affine functions of the individual variables that compose the hyper-variable receiving the message. The envelope is tight at the minimizers which are shared by the true message. Benders rows are constructed to be Pareto optimal and are generated using an efficient procedure targeted for binary problems. |
| Tasks | |
| Published | 2018-05-13 |
| URL | http://arxiv.org/abs/1805.04958v1 |
| http://arxiv.org/pdf/1805.04958v1.pdf | |
| PWC | https://paperswithcode.com/paper/accelerating-message-passing-for-map-with |
| Repo | |
| Framework | |
Towards a General-Purpose Linguistic Annotation Backend
| Title | Towards a General-Purpose Linguistic Annotation Backend |
| Authors | Graham Neubig, Patrick Littell, Chian-Yu Chen, Jean Lee, Zirui Li, Yu-Hsiang Lin, Yuyan Zhang |
| Abstract | Language documentation is inherently a time-intensive process; transcription, glossing, and corpus management consume a significant portion of documentary linguists’ work. Advances in natural language processing can help to accelerate this work, using the linguists’ past decisions as training material, but questions remain about how to prioritize human involvement. In this extended abstract, we describe the beginnings of a new project that will attempt to ease this language documentation process through the use of natural language processing (NLP) technology. It is based on (1) methods to adapt NLP tools to new languages, based on recent advances in massively multilingual neural networks, and (2) backend APIs and interfaces that allow linguists to upload their data. We then describe our current progress on two fronts: automatic phoneme transcription, and glossing. Finally, we briefly describe our future directions. |
| Tasks | |
| Published | 2018-12-13 |
| URL | http://arxiv.org/abs/1812.05272v1 |
| http://arxiv.org/pdf/1812.05272v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-a-general-purpose-linguistic |
| Repo | |
| Framework | |
