Paper Group ANR 1253
Efficient Bimanual Manipulation Using Learned Task Schemas. Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks. Tensor Completion via Gaussian Process Based Initialization. J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction. Federated machine learning with Anonymous Random Hybridization …
Efficient Bimanual Manipulation Using Learned Task Schemas
| Title | Efficient Bimanual Manipulation Using Learned Task Schemas |
| Authors | Rohan Chitnis, Shubham Tulsiani, Saurabh Gupta, Abhinav Gupta |
| Abstract | We address the problem of effectively composing skills to solve sparse-reward tasks in the real world. Given a set of parameterized skills (such as exerting a force or doing a top grasp at a location), our goal is to learn policies that invoke these skills to efficiently solve such tasks. Our insight is that for many tasks, the learning process can be decomposed into learning a state-independent task schema (a sequence of skills to execute) and a policy to choose the parameterizations of the skills in a state-dependent manner. For such tasks, we show that explicitly modeling the schema’s state-independence can yield significant improvements in sample efficiency for model-free reinforcement learning algorithms. Furthermore, these schemas can be transferred to solve related tasks, by simply re-learning the parameterizations with which the skills are invoked. We find that doing so enables learning to solve sparse-reward tasks on real-world robotic systems very efficiently. We validate our approach experimentally over a suite of robotic bimanual manipulation tasks, both in simulation and on real hardware. See videos at http://tinyurl.com/chitnis-schema. |
| Tasks | |
| Published | 2019-09-30 |
| URL | https://arxiv.org/abs/1909.13874v2 |
| https://arxiv.org/pdf/1909.13874v2.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-bimanual-manipulation-using-learned |
| Repo | |
| Framework | |
Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks
| Title | Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks |
| Authors | Yunxiang Zhang, Chenglong Zhao, Bingbing Ni, Jian Zhang, Haoran Deng |
| Abstract | To address the limitations of existing magnitude-based pruning algorithms in cases where model weights or activations are of large and similar magnitude, we propose a novel perspective to discover parameter redundancy among channels and accelerate deep CNNs via channel pruning. Precisely, we argue that channels revealing similar feature information have functional overlap and that most channels within each such similarity group can be removed without compromising model’s representational power. After deriving an effective metric for evaluating channel similarity through probabilistic modeling, we introduce a pruning algorithm via hierarchical clustering of channels. In particular, the proposed algorithm does not rely on sparsity training techniques or complex data-driven optimization and can be directly applied to pre-trained models. Extensive experiments on benchmark datasets strongly demonstrate the superior acceleration performance of our approach over prior arts. On ImageNet, our pruned ResNet-50 with 30% FLOPs reduced outperforms the baseline model. |
| Tasks | |
| Published | 2019-08-06 |
| URL | https://arxiv.org/abs/1908.02620v1 |
| https://arxiv.org/pdf/1908.02620v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploiting-channel-similarity-for |
| Repo | |
| Framework | |
Tensor Completion via Gaussian Process Based Initialization
| Title | Tensor Completion via Gaussian Process Based Initialization |
| Authors | Yermek Kapushev, Ivan Oseledets, Evgeny Burnaev |
| Abstract | In this paper, we consider the tensor completion problem representing the solution in the tensor train (TT) format. It is assumed that tensor is high-dimensional, and tensor values are generated by an unknown smooth function. The assumption allows us to develop an efficient initialization scheme based on Gaussian Process Regression and TT-cross approximation technique. The proposed approach can be used in conjunction with any optimization algorithm that is usually utilized in tensor completion problems. We empirically justify that in this case the reconstruction error improves compared to the tensor completion with random initialization. As an additional benefit, our technique automatically selects rank thanks to using the TT-cross approximation technique. |
| Tasks | |
| Published | 2019-12-11 |
| URL | https://arxiv.org/abs/1912.05179v1 |
| https://arxiv.org/pdf/1912.05179v1.pdf | |
| PWC | https://paperswithcode.com/paper/tensor-completion-via-gaussian-process-based |
| Repo | |
| Framework | |
J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction
| Title | J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction |
| Authors | Hemant Kumar Aggarwal, Mathews Jacob |
| Abstract | Modern MRI schemes, which rely on compressed sensing or deep learning algorithms to recover MRI data from undersampled multichannel Fourier measurements, are widely used to reduce scan time. The image quality of these approaches is heavily dependent on the sampling pattern. We introduce a continuous strategy to jointly optimize the sampling pattern and the parameters of the reconstruction algorithm. We propose to use a model-based deep learning (MoDL) image reconstruction algorithm, which alternates between a data consistency module and a convolutional neural network (CNN). We use a multi-channel forward model, consisting of a non-uniform Fourier transform with continuously defined sampling locations, to realize the data consistency block. This approach facilitates the joint and continuous optimization of the sampling pattern and the CNN parameters. We observe that the joint optimization of the sampling patterns and the reconstruction module significantly improves the performance, compared to current deep learning methods that use variable density sampling patterns. Our experiments show that the improved decoupling of the CNN parameters from the sampling scheme offered by the MoDL scheme translates to improved optimization and performance compared to a similar scheme using a direct-inversion based reconstruction algorithm. The experiments also show that the proposed scheme offers good convergence and reduces the dependence on initialization. |
| Tasks | Image Reconstruction |
| Published | 2019-11-06 |
| URL | https://arxiv.org/abs/1911.02945v2 |
| https://arxiv.org/pdf/1911.02945v2.pdf | |
| PWC | https://paperswithcode.com/paper/joint-optimization-of-sampling-patterns-and |
| Repo | |
| Framework | |
Federated machine learning with Anonymous Random Hybridization (FeARH) on medical records
| Title | Federated machine learning with Anonymous Random Hybridization (FeARH) on medical records |
| Authors | Jianfei Cui, Dianbo Liu |
| Abstract | Sometimes electrical medical records are restricted and difficult to centralize for machine learning, which could only be trained in distributed manner that involved many institutions in the process. However, sometimes some institutions are likely to figure out the private data used for training certain models based on the parameters they obtained, which is a violation of privacy and certain regulations. Under those circumstances, we develop an algorithm, called ‘federated machine learning with anonymous random hybridization’(abbreviated as ‘FeARH’), using mainly hybridization algorithm to eliminate connections between medical record data and models’ parameters, which avoid untrustworthy institutions from stealing patients’ private medical records. Based on our experiment, our new algorithm has similar AUCROC and AUCPR result compared with machine learning in centralized manner and original federated machine learning, at the same time, our algorithm can greatly reduce data transfer size in comparison with original federated machine learning. |
| Tasks | |
| Published | 2019-12-25 |
| URL | https://arxiv.org/abs/2001.09751v1 |
| https://arxiv.org/pdf/2001.09751v1.pdf | |
| PWC | https://paperswithcode.com/paper/federated-machine-learning-with-anonymous |
| Repo | |
| Framework | |
Using Monolingual Data in Neural Machine Translation: a Systematic Study
| Title | Using Monolingual Data in Neural Machine Translation: a Systematic Study |
| Authors | Franck Burlot, François Yvon |
| Abstract | Neural Machine Translation (MT) has radically changed the way systems are developed. A major difference with the previous generation (Phrase-Based MT) is the way monolingual target data, which often abounds, is used in these two paradigms. While Phrase-Based MT can seamlessly integrate very large language models trained on billions of sentences, the best option for Neural MT developers seems to be the generation of artificial parallel data through \textsl{back-translation} - a technique that fails to fully take advantage of existing datasets. In this paper, we conduct a systematic study of back-translation, comparing alternative uses of monolingual data, as well as multiple data generation procedures. Our findings confirm that back-translation is very effective and give new explanations as to why this is the case. We also introduce new data simulation techniques that are almost as effective, yet much cheaper to implement. |
| Tasks | Machine Translation |
| Published | 2019-03-27 |
| URL | http://arxiv.org/abs/1903.11437v1 |
| http://arxiv.org/pdf/1903.11437v1.pdf | |
| PWC | https://paperswithcode.com/paper/using-monolingual-data-in-neural-machine-1 |
| Repo | |
| Framework | |
Gray Level Image Threshold Using Neutrosophic Shannon Entropy
| Title | Gray Level Image Threshold Using Neutrosophic Shannon Entropy |
| Authors | Vasile Patrascu |
| Abstract | This article presents a new method of segmenting grayscale images by minimizing Shannon’s neutrosophic entropy. For the proposed segmentation method, the neutrosophic information components, i.e., the degree of truth, the degree of neutrality and the degree of falsity are defined taking into account the belonging to the segmented regions and at the same time to the separation threshold area. The principle of the method is simple and easy to understand and can lead to multiple thresholds. The efficacy of the method is illustrated using some test gray level images. The experimental results show that the proposed method has good performance for segmentation with optimal gray level thresholds. |
| Tasks | |
| Published | 2019-06-26 |
| URL | https://arxiv.org/abs/1906.12167v1 |
| https://arxiv.org/pdf/1906.12167v1.pdf | |
| PWC | https://paperswithcode.com/paper/gray-level-image-threshold-using-neutrosophic |
| Repo | |
| Framework | |
Data Selection with Cluster-Based Language Difference Models and Cynical Selection
| Title | Data Selection with Cluster-Based Language Difference Models and Cynical Selection |
| Authors | Lucía Santamaría, Amittai Axelrod |
| Abstract | We present and apply two methods for addressing the problem of selecting relevant training data out of a general pool for use in tasks such as machine translation. Building on existing work on class-based language difference models, we first introduce a cluster-based method that uses Brown clusters to condense the vocabulary of the corpora. Secondly, we implement the cynical data selection method, which incrementally constructs a training corpus to efficiently model the task corpus. Both the cluster-based and the cynical data selection approaches are used for the first time within a machine translation system, and we perform a head-to-head comparison. Our intrinsic evaluations show that both new methods outperform the standard Moore-Lewis approach (cross-entropy difference), in terms of better perplexity and OOV rates on in-domain data. The cynical approach converges much quicker, covering nearly all of the in-domain vocabulary with 84% less data than the other methods. Furthermore, the new approaches can be used to select machine translation training data for training better systems. Our results confirm that class-based selection using Brown clusters is a viable alternative to POS-based class-based methods, and removes the reliance on a part-of-speech tagger. Additionally, we are able to validate the recently proposed cynical data selection method, showing that its performance in SMT models surpasses that of traditional cross-entropy difference methods and more closely matches the sentence length of the task corpus. |
| Tasks | Machine Translation |
| Published | 2019-04-09 |
| URL | http://arxiv.org/abs/1904.04900v1 |
| http://arxiv.org/pdf/1904.04900v1.pdf | |
| PWC | https://paperswithcode.com/paper/data-selection-with-cluster-based-language |
| Repo | |
| Framework | |
A Rademacher Complexity Based Method fo rControlling Power and Confidence Level in Adaptive Statistical Analysis
| Title | A Rademacher Complexity Based Method fo rControlling Power and Confidence Level in Adaptive Statistical Analysis |
| Authors | Lorenzo De Stefani, Eli Upfal |
| Abstract | While standard statistical inference techniques and machine learning generalization bounds assume that tests are run on data selected independently of the hypotheses, practical data analysis and machine learning are usually iterative and adaptive processes where the same holdout data is often used for testing a sequence of hypotheses (or models), which may each depend on the outcome of the previous tests on the same data. In this work, we present RadaBound a rigorous, efficient and practical procedure for controlling the generalization error when using a holdout sample for multiple adaptive testing. Our solution is based on a new application of the Rademacher Complexity generalization bounds, adapted to dependent tests. We demonstrate the statistical power and practicality of our method through extensive simulations and comparisons to alternative approaches. |
| Tasks | |
| Published | 2019-10-04 |
| URL | https://arxiv.org/abs/1910.03493v1 |
| https://arxiv.org/pdf/1910.03493v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-rademacher-complexity-based-method-fo |
| Repo | |
| Framework | |
Learning Improved Representations by Transferring Incomplete Evidence Across Heterogeneous Tasks
| Title | Learning Improved Representations by Transferring Incomplete Evidence Across Heterogeneous Tasks |
| Authors | Athanasios Davvetas, Iraklis A. Klampanos |
| Abstract | Acquiring ground truth labels for unlabelled data can be a costly procedure, since it often requires manual labour that is error-prone. Consequently, the available amount of labelled data is increasingly reduced due to the limitations of manual data labelling. It is possible to increase the amount of labelled data samples by performing automated labelling or crowd-sourcing the annotation procedure. However, they often introduce noise or uncertainty in the labelset, that leads to decreased performance of supervised deep learning methods. On the other hand, weak supervision methods remain robust during noisy labelsets or can be effective even with low amounts of labelled data. In this paper we evaluate the effectiveness of a representation learning method that uses external categorical evidence called “Evidence Transfer”, against low amount of corresponding evidence termed as incomplete evidence. Evidence transfer is a robust solution against external unknown categorical evidence that can introduce noise or uncertainty. In our experimental evaluation, evidence transfer proves to be effective and robust against different levels of incompleteness, for two types of incomplete evidence. |
| Tasks | Representation Learning |
| Published | 2019-12-22 |
| URL | https://arxiv.org/abs/1912.10490v1 |
| https://arxiv.org/pdf/1912.10490v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-improved-representations-by |
| Repo | |
| Framework | |
Deformable Filter Convolution for Point Cloud Reasoning
| Title | Deformable Filter Convolution for Point Cloud Reasoning |
| Authors | Yuwen Xiong, Mengye Ren, Renjie Liao, Kelvin Wong, Raquel Urtasun |
| Abstract | Point clouds are the native output of many real-world 3D sensors. To borrow the success of 2D convolutional network architectures, a majority of popular 3D perception models voxelize the points, which can result in a loss of local geometric details that cannot be recovered. In this paper, we propose a novel learnable convolution layer for processing 3D point cloud data directly. Instead of discretizing points into fixed voxels, we deform our learnable 3D filters to match with the point cloud shape. We propose to combine voxelized backbone networks with our deformable filter layer at 1) the network input stream and 2) the output prediction layers to enhance point level reasoning. We obtain state-of-the-art results on LiDAR semantic segmentation and producing a significant gain in performance on LiDAR object detection. |
| Tasks | Object Detection, Semantic Segmentation |
| Published | 2019-07-30 |
| URL | https://arxiv.org/abs/1907.13079v1 |
| https://arxiv.org/pdf/1907.13079v1.pdf | |
| PWC | https://paperswithcode.com/paper/deformable-filter-convolution-for-point-cloud |
| Repo | |
| Framework | |
JGAN: A Joint Formulation of GAN for Synthesizing Images and Labels
| Title | JGAN: A Joint Formulation of GAN for Synthesizing Images and Labels |
| Authors | Minje Park |
| Abstract | Image generation with explicit condition or label generally works better than unconditional image generation. In modern GAN frameworks, both generator and discriminator are formulated to model the conditional distribution of images given with labels. In this paper, we provide an alternative formulation of GAN which models joint distribution of images and labels. There are two advantages in this joint formulation over conditional approaches. The first advantage is that the joint formulation is more robust to label noises, and the second is we can use any kind of weak labels (or additional information which has dependence on the original image data) to enhance unconditional image generation. We will show the effectiveness of joint formulation in CIFAR-10, CIFAR-100, and STL dataset. |
| Tasks | Image Generation |
| Published | 2019-05-28 |
| URL | https://arxiv.org/abs/1905.11574v2 |
| https://arxiv.org/pdf/1905.11574v2.pdf | |
| PWC | https://paperswithcode.com/paper/jgan-a-joint-formulation-of-gan-for |
| Repo | |
| Framework | |
Maximum Weighted Loss Discrepancy
| Title | Maximum Weighted Loss Discrepancy |
| Authors | Fereshte Khani, Aditi Raghunathan, Percy Liang |
| Abstract | Though machine learning algorithms excel at minimizing the average loss over a population, this might lead to large discrepancies between the losses across groups within the population. To capture this inequality, we introduce and study a notion we call maximum weighted loss discrepancy (MWLD), the maximum (weighted) difference between the loss of a group and the loss of the population. We relate MWLD to group fairness notions and robustness to demographic shifts. We then show MWLD satisfies the following three properties: 1) It is statistically impossible to estimate MWLD when all groups have equal weights. 2) For a particular family of weighting functions, we can estimate MWLD efficiently. 3) MWLD is related to loss variance, a quantity that arises in generalization bounds. We estimate MWLD with different weighting functions on four common datasets from the fairness literature. We finally show that loss variance regularization can halve the loss variance of a classifier and hence reduce MWLD without suffering a significant drop in accuracy. |
| Tasks | |
| Published | 2019-06-08 |
| URL | https://arxiv.org/abs/1906.03518v1 |
| https://arxiv.org/pdf/1906.03518v1.pdf | |
| PWC | https://paperswithcode.com/paper/maximum-weighted-loss-discrepancy |
| Repo | |
| Framework | |
On the Adaptivity of Stochastic Gradient-Based Optimization
| Title | On the Adaptivity of Stochastic Gradient-Based Optimization |
| Authors | Lihua Lei, Michael I. Jordan |
| Abstract | Stochastic-gradient-based optimization has been a core enabling methodology in applications to large-scale problems in machine learning and related areas. Despite the progress, the gap between theory and practice remains significant, with theoreticians pursuing mathematical optimality at a cost of obtaining specialized procedures in different regimes (e.g., modulus of strong convexity, magnitude of target accuracy, signal-to-noise ratio), and with practitioners not readily able to know which regime is appropriate to their problem, and seeking broadly applicable algorithms that are reasonably close to optimality. To bridge these perspectives it is necessary to study algorithms that are adaptive to different regimes. We present the stochastically controlled stochastic gradient (SCSG) method for composite convex finite-sum optimization problems and show that SCSG is adaptive to both strong convexity and target accuracy. The adaptivity is achieved by batch variance reduction with adaptive batch sizes and a novel technique, which we referred to as \emph{geometrization}, which sets the length of each epoch as a geometric random variable. The algorithm achieves strictly better theoretical complexity than other existing adaptive algorithms, while the tuning parameters of the algorithm only depend on the smoothness parameter of the objective. |
| Tasks | |
| Published | 2019-04-09 |
| URL | https://arxiv.org/abs/1904.04480v2 |
| https://arxiv.org/pdf/1904.04480v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-adaptivity-of-stochastic-gradient |
| Repo | |
| Framework | |
SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning
| Title | SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning |
| Authors | Pramit Saha, Muhammad Abdul-Mageed, Sidney Fels |
| Abstract | Speech-related Brain Computer Interface (BCI) technologies provide effective vocal communication strategies for controlling devices through speech commands interpreted from brain signals. In order to infer imagined speech from active thoughts, we propose a novel hierarchical deep learning BCI system for subject-independent classification of 11 speech tokens including phonemes and words. Our novel approach exploits predicted articulatory information of six phonological categories (e.g., nasal, bilabial) as an intermediate step for classifying the phonemes and words, thereby finding discriminative signal responsible for natural speech synthesis. The proposed network is composed of hierarchical combination of spatial and temporal CNN cascaded with a deep autoencoder. Our best models on the KARA database achieve an average accuracy of 83.42% across the six different binary phonological classification tasks, and 53.36% for the individual token identification task, significantly outperforming our baselines. Ultimately, our work suggests the possible existence of a brain imagery footprint for the underlying articulatory movement related to different sounds that can be used to aid imagined speech decoding. |
| Tasks | Speech Recognition, Speech Synthesis |
| Published | 2019-04-08 |
| URL | http://arxiv.org/abs/1904.05746v1 |
| http://arxiv.org/pdf/1904.05746v1.pdf | |
| PWC | https://paperswithcode.com/paper/speak-your-mind-towards-imagined-speech |
| Repo | |
| Framework | |
