Paper Group ANR 235
A neuro-inspired architecture for unsupervised continual learning based on online clustering and hierarchical predictive coding. Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment. A multi-level convolutional LSTM model for the segmentation of left ventricle myocardium in infarcted porcine cine MR imag …
A neuro-inspired architecture for unsupervised continual learning based on online clustering and hierarchical predictive coding
| Title | A neuro-inspired architecture for unsupervised continual learning based on online clustering and hierarchical predictive coding |
| Authors | Constantine Dovrolis |
| Abstract | We propose that the Continual Learning desiderata can be achieved through a neuro-inspired architecture, grounded on Mountcastle’s cortical column hypothesis. The proposed architecture involves a single module, called Self-Taught Associative Memory (STAM), which models the function of a cortical column. STAMs are repeated in multi-level hierarchies involving feedforward, lateral and feedback connections. STAM networks learn in an unsupervised manner, based on a combination of online clustering and hierarchical predictive coding. This short paper only presents the architecture and its connections with neuroscience. A mathematical formulation and experimental results will be presented in an extended version of this paper. |
| Tasks | Continual Learning |
| Published | 2018-10-22 |
| URL | http://arxiv.org/abs/1810.09391v1 |
| http://arxiv.org/pdf/1810.09391v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-neuro-inspired-architecture-for |
| Repo | |
| Framework | |
Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment
| Title | Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment |
| Authors | Tu Vu, Vered Shwartz |
| Abstract | Supervised distributional methods are applied successfully in lexical entailment, but recent work questioned whether these methods actually learn a relation between two words. Specifically, Levy et al. (2015) claimed that linear classifiers learn only separate properties of each word. We suggest a cheap and easy way to boost the performance of these methods by integrating multiplicative features into commonly used representations. We provide an extensive evaluation with different classifiers and evaluation setups, and suggest a suitable evaluation setup for the task, eliminating biases existing in previous ones. |
| Tasks | |
| Published | 2018-04-24 |
| URL | http://arxiv.org/abs/1804.08845v1 |
| http://arxiv.org/pdf/1804.08845v1.pdf | |
| PWC | https://paperswithcode.com/paper/integrating-multiplicative-features-into |
| Repo | |
| Framework | |
A multi-level convolutional LSTM model for the segmentation of left ventricle myocardium in infarcted porcine cine MR images
| Title | A multi-level convolutional LSTM model for the segmentation of left ventricle myocardium in infarcted porcine cine MR images |
| Authors | Dongqing Zhang, Ilknur Icke, Belma Dogdas, Sarayu Parimal, Smita Sampath, Joseph Forbes, Ansuman Bagchi, Chih-Liang Chin, Antong Chen |
| Abstract | Automatic segmentation of left ventricle (LV) myocardium in cardiac short-axis cine MR images acquired on subjects with myocardial infarction is a challenging task, mainly because of the various types of image inhomogeneity caused by the infarctions. Among the approaches proposed to automate the LV myocardium segmentation task, methods based upon deep convolutional neural networks (CNN) have demonstrated their exceptional accuracy and robustness in recent years. However, most of the CNN-based approaches treat the frames in a cardiac cycle independently, which fails to capture the valuable dynamics of heart motion. Herein, an approach based on recurrent neural network (RNN), specifically a multi-level convolutional long short-term memory (ConvLSTM) model, is proposed to take the motion of the heart into consideration. Based on a ResNet-56 CNN, LV-related image features in consecutive frames of a cardiac cycle are extracted at both the low- and high-resolution levels, which are processed by the corresponding multi-level ConvLSTM models to generate the myocardium segmentations. A leave-one-out experiment was carried out on a set of 3,600 cardiac cine MR slices collected in-house for 8 porcine subjects with surgically induced myocardial infarction. Compared with a solely CNN-based approach, the proposed approach demonstrated its superior robustness against image inhomogeneity by incorporating information from adjacent frames. It also outperformed a one-level ConvLSTM approach thanks to its capabilities to take advantage of image features at multiple resolution levels. |
| Tasks | |
| Published | 2018-11-14 |
| URL | http://arxiv.org/abs/1811.06051v1 |
| http://arxiv.org/pdf/1811.06051v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-multi-level-convolutional-lstm-model-for |
| Repo | |
| Framework | |
Flexible Deep Neural Network Processing
| Title | Flexible Deep Neural Network Processing |
| Authors | Hokchhay Tann, Soheil Hashemi, Sherief Reda |
| Abstract | The recent success of Deep Neural Networks (DNNs) has drastically improved the state of the art for many application domains. While achieving high accuracy performance, deploying state-of-the-art DNNs is a challenge since they typically require billions of expensive arithmetic computations. In addition, DNNs are typically deployed in ensemble to boost accuracy performance, which further exacerbates the system requirements. This computational overhead is an issue for many platforms, e.g. data centers and embedded systems, with tight latency and energy budgets. In this article, we introduce flexible DNNs ensemble processing technique, which achieves large reduction in average inference latency while incurring small to negligible accuracy drop. Our technique is flexible in that it allows for dynamic adaptation between quality of results (QoR) and execution runtime. We demonstrate the effectiveness of the technique on AlexNet and ResNet-50 using the ImageNet dataset. This technique can also easily handle other types of networks. |
| Tasks | |
| Published | 2018-01-23 |
| URL | http://arxiv.org/abs/1801.07353v1 |
| http://arxiv.org/pdf/1801.07353v1.pdf | |
| PWC | https://paperswithcode.com/paper/flexible-deep-neural-network-processing |
| Repo | |
| Framework | |
Simplifying the minimax disparity model for determining OWA weights in large-scale problems
| Title | Simplifying the minimax disparity model for determining OWA weights in large-scale problems |
| Authors | Thuy Hong Nguyen |
| Abstract | In the context of multicriteria decision making, the ordered weighted averaging (OWA) functions play a crucial role in aggregating multiple criteria evaluations into an overall assessment supporting the decision makers’ choice. Determining OWA weights, therefore, is an essential part of this process. Available methods for determining OWA weights, however, often require heavy computational loads in real-life large-scale optimization problems. In this paper, we propose a new approach to simplify the well-known minimax disparity model for determining OWA weights. For this purpose, we use to the binomial decomposition framework in which natural constraints can be imposed on the level of complexity of the weight distribution. The original problem of determining OWA weights is thereby transformed into a smaller scale optimization problem, formulated in terms of the coefficients in the binomial decomposition. Our preliminary results show that a small set of these coefficients can encode for an appropriate full-dimensional set of OWA weights. |
| Tasks | Decision Making |
| Published | 2018-04-17 |
| URL | http://arxiv.org/abs/1804.06331v1 |
| http://arxiv.org/pdf/1804.06331v1.pdf | |
| PWC | https://paperswithcode.com/paper/simplifying-the-minimax-disparity-model-for |
| Repo | |
| Framework | |
A Flexible Convolutional Solver with Application to Photorealistic Style Transfer
| Title | A Flexible Convolutional Solver with Application to Photorealistic Style Transfer |
| Authors | Gilles Puy, Patrick Pérez |
| Abstract | We propose a new flexible deep convolutional neural network (convnet) to perform fast visual style transfer. In contrast to existing convnets that address the same task, our architecture derives directly from the structure of the gradient descent originally used to solve the style transfer problem [Gatys et al., 2016]. Like existing convnets, ours approximately solves the original problem much faster than the gradient descent. However, our network is uniquely flexible by design: it can be manipulated at runtime to enforce new constraints on the final solution. In particular, we show how to modify it to obtain a photorealistic result with no retraining. We study the modifications made by [Luan et al., 2017] to the original cost function of [Gatys et al., 2016] to achieve photorealistic style transfer. These modifications affect directly the gradient descent and can be reported on-the-fly in our network. These modifications are possible as the proposed architecture stems from unrolling the gradient descent. |
| Tasks | Style Transfer |
| Published | 2018-06-13 |
| URL | http://arxiv.org/abs/1806.05285v1 |
| http://arxiv.org/pdf/1806.05285v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-flexible-convolutional-solver-with |
| Repo | |
| Framework | |
Parallel Computation of PDFs on Big Spatial Data Using Spark
| Title | Parallel Computation of PDFs on Big Spatial Data Using Spark |
| Authors | Ji Liu, Noel Moreno Lemus, Esther Pacitti, Fabio Porto, Patrick Valduriez |
| Abstract | We consider big spatial data, which is typically produced in scientific areas such as geological or seismic interpretation. The spatial data can be produced by observation (e.g. using sensors or soil instrument) or numerical simulation programs and correspond to points that represent a 3D soil cube area. However, errors in signal processing and modeling create some uncertainty, and thus a lack of accuracy in identifying geological or seismic phenomenons. Such uncertainty must be carefully analyzed. To analyze uncertainty, the main solution is to compute a Probability Density Function (PDF) of each point in the spatial cube area. However, computing PDFs on big spatial data can be very time consuming (from several hours to even months on a parallel computer). In this paper, we propose a new solution to efficiently compute such PDFs in parallel using Spark, with three methods: data grouping, machine learning prediction and sampling. We evaluate our solution by extensive experiments on different computer clusters using big data ranging from hundreds of GB to several TB. The experimental results show that our solution scales up very well and can reduce the execution time by a factor of 33 (in the order of seconds or minutes) compared with a baseline method. |
| Tasks | Seismic Interpretation |
| Published | 2018-05-08 |
| URL | http://arxiv.org/abs/1805.03141v1 |
| http://arxiv.org/pdf/1805.03141v1.pdf | |
| PWC | https://paperswithcode.com/paper/parallel-computation-of-pdfs-on-big-spatial |
| Repo | |
| Framework | |
Demystifying Neural Network Filter Pruning
| Title | Demystifying Neural Network Filter Pruning |
| Authors | Zhuwei Qin, Fuxun Yu, ChenChen Liu, Xiang Chen |
| Abstract | Based on filter magnitude ranking (e.g. L1 norm), conventional filter pruning methods for Convolutional Neural Networks (CNNs) have been proved with great effectiveness in computation load reduction. Although effective, these methods are rarely analyzed in a perspective of filter functionality. In this work, we explore the filter pruning and the retraining through qualitative filter functionality interpretation. We find that the filter magnitude based method fails to eliminate the filters with repetitive functionality. And the retraining phase is actually used to reconstruct the remained filters for functionality compensation for the wrongly-pruned critical filters. With a proposed functionality-oriented pruning method, we further testify that, by precisely addressing the filter functionality redundancy, a CNN can be pruned without considerable accuracy drop, and the retraining phase is unnecessary. |
| Tasks | |
| Published | 2018-10-29 |
| URL | https://arxiv.org/abs/1811.02639v1 |
| https://arxiv.org/pdf/1811.02639v1.pdf | |
| PWC | https://paperswithcode.com/paper/demystifying-neural-network-filter-pruning |
| Repo | |
| Framework | |
A Method For Dynamic Ensemble Selection Based on a Filter and an Adaptive Distance to Improve the Quality of the Regions of Competence
| Title | A Method For Dynamic Ensemble Selection Based on a Filter and an Adaptive Distance to Improve the Quality of the Regions of Competence |
| Authors | Rafael M. O. Cruz, George D. C. Cavalcanti, Tsang Ing Ren |
| Abstract | Dynamic classifier selection systems aim to select a group of classifiers that is most adequate for a specific query pattern. This is done by defining a region around the query pattern and analyzing the competence of the classifiers in this region. However, the regions are often surrounded by noise which can difficult the classifier selection. This fact makes the performance of most dynamic selection systems no better than static selections. In this paper, we demonstrate that the performance dynamic selection systems end up limited by the quality of the regions extracted. Thereafter, we propose a new dynamic classifier selection that improves the regions of competence in order to achieve higher recognition rates. obtained from several classification databases show the proposed method not only increase the recognition performance but also decreases the computational cost. |
| Tasks | |
| Published | 2018-11-01 |
| URL | http://arxiv.org/abs/1811.00669v1 |
| http://arxiv.org/pdf/1811.00669v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-method-for-dynamic-ensemble-selection-based |
| Repo | |
| Framework | |
A Semi-Supervised and Inductive Embedding Model for Churn Prediction of Large-Scale Mobile Games
| Title | A Semi-Supervised and Inductive Embedding Model for Churn Prediction of Large-Scale Mobile Games |
| Authors | Xi Liu, Muhe Xie, Xidao Wen, Rui Chen, Yong Ge, Nick Duffield, Na Wang |
| Abstract | Mobile gaming has emerged as a promising market with billion-dollar revenues. A variety of mobile game platforms and services have been developed around the world. One critical challenge for these platforms and services is to understand user churn behavior in mobile games. Accurate churn prediction will benefit many stakeholders such as game developers, advertisers, and platform operators. In this paper, we present the first large-scale churn prediction solution for mobile games. In view of the common limitations of the state-of-the-art methods built upon traditional machine learning models, we devise a novel semi-supervised and inductive embedding model that jointly learns the prediction function and the embedding function for user-app relationships. We model these two functions by deep neural networks with a unique edge embedding technique that is able to capture both contextual information and relationship dynamics. We also design a novel attributed random walk technique that takes into consideration both topological adjacency and attribute similarities. To evaluate the performance of our solution, we collect real-world data from the Samsung Game Launcher platform that includes tens of thousands of games and hundreds of millions of user-app interactions. The experimental results with this data demonstrate the superiority of our proposed model against existing state-of-the-art methods. |
| Tasks | |
| Published | 2018-08-20 |
| URL | http://arxiv.org/abs/1808.06573v3 |
| http://arxiv.org/pdf/1808.06573v3.pdf | |
| PWC | https://paperswithcode.com/paper/a-semi-supervised-and-inductive-embedding |
| Repo | |
| Framework | |
Gnowee: A Hybrid Metaheuristic Optimization Algorithm for Constrained, Black Box, Combinatorial Mixed-Integer Design
| Title | Gnowee: A Hybrid Metaheuristic Optimization Algorithm for Constrained, Black Box, Combinatorial Mixed-Integer Design |
| Authors | James Bevins, Rachel Slaybaugh |
| Abstract | This paper introduces Gnowee, a modular, Python-based, open-source hybrid metaheuristic optimization algorithm (Available from https://github.com/SlaybaughLab/Gnowee). Gnowee is designed for rapid convergence to nearly globally optimum solutions for complex, constrained nuclear engineering problems with mixed-integer and combinatorial design vectors and high-cost, noisy, discontinuous, black box objective function evaluations. Gnowee’s hybrid metaheuristic framework is a new combination of a set of diverse, robust heuristics that appropriately balance diversification and intensification strategies across a wide range of optimization problems. This novel algorithm was specifically developed to optimize complex nuclear design problems; the motivating research problem was the design of material stack-ups to modify neutron energy spectra to specific targeted spectra for applications in nuclear medicine, technical nuclear forensics, nuclear physics, etc. However, there are a wider range of potential applications for this algorithm both within the nuclear community and beyond. To demonstrate Gnowee’s behavior for a variety of problem types, comparisons between Gnowee and several well-established metaheuristic algorithms are made for a set of eighteen continuous, mixed-integer, and combinatorial benchmarks. These results demonstrate Gnoweee to have superior flexibility and convergence characteristics over a wide range of design spaces. We anticipate this wide range of applicability will make this algorithm desirable for many complex engineering applications. |
| Tasks | |
| Published | 2018-04-15 |
| URL | http://arxiv.org/abs/1804.05429v1 |
| http://arxiv.org/pdf/1804.05429v1.pdf | |
| PWC | https://paperswithcode.com/paper/gnowee-a-hybrid-metaheuristic-optimization |
| Repo | |
| Framework | |
Deploy Large-Scale Deep Neural Networks in Resource Constrained IoT Devices with Local Quantization Region
| Title | Deploy Large-Scale Deep Neural Networks in Resource Constrained IoT Devices with Local Quantization Region |
| Authors | Yi Yang, Andy Chen, Xiaoming Chen, Jiang Ji, Zhenyang Chen, Yan Dai |
| Abstract | Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networks), incompatible with IoT world. We present a low-bit (range from 8-bit to 1-bit) scheme with our local quantization region algorithm. We use models in Caffe model zoo as our example tasks to evaluate the effect of our low precision data representation scheme. With the available of local quantization region, we find implementations on top of those schemes could greatly retain the model accuracy, besides the reduction of computational complexity. For example, our 8-bit scheme has no drops on top-1 and top-5 accuracy with 2x speedup on Intel Edison IoT platform. Implementations based on our 4-bit, 2-bit or 1-bit scheme are also applicable to IoT devices with advances of low computational complexity. For example, the drop on our task is only 0.7% when using 2-bit scheme, a scheme which could largely save transistors. Making low-bit scheme usable here opens a new door for further optimization on commodity IoT controller, i.e. extra speed-up could be achieved by replacing multiply-accumulate operations with the proposed table look-up operations. The whole study offers a new approach to relief the challenge of bring advanced deep learning algorithm to resource constrained low-cost IoT device. |
| Tasks | Quantization |
| Published | 2018-05-24 |
| URL | http://arxiv.org/abs/1805.09473v1 |
| http://arxiv.org/pdf/1805.09473v1.pdf | |
| PWC | https://paperswithcode.com/paper/deploy-large-scale-deep-neural-networks-in |
| Repo | |
| Framework | |
A Machine Learning Driven IoT Solution for Noise Classification in Smart Cities
| Title | A Machine Learning Driven IoT Solution for Noise Classification in Smart Cities |
| Authors | Yasser Alsouda, Sabri Pllana, Arianit Kurti |
| Abstract | We present a machine learning based method for noise classification using a low-power and inexpensive IoT unit. We use Mel-frequency cepstral coefficients for audio feature extraction and supervised classification algorithms (that is, support vector machine and k-nearest neighbors) for noise classification. We evaluate our approach experimentally with a dataset of about 3000 sound samples grouped in eight sound classes (such as, car horn, jackhammer, or street music). We explore the parameter space of support vector machine and k-nearest neighbors algorithms to estimate the optimal parameter values for classification of sound samples in the dataset under study. We achieve a noise classification accuracy in the range 85% – 100%. Training and testing of our k-nearest neighbors (k = 1) implementation on Raspberry Pi Zero W is less than a second for a dataset with features of more than 3000 sound samples. |
| Tasks | |
| Published | 2018-09-01 |
| URL | http://arxiv.org/abs/1809.00238v1 |
| http://arxiv.org/pdf/1809.00238v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-machine-learning-driven-iot-solution-for |
| Repo | |
| Framework | |
Regularized Loss Minimizers with Local Data Perturbation: Consistency and Data Irrecoverability
| Title | Regularized Loss Minimizers with Local Data Perturbation: Consistency and Data Irrecoverability |
| Authors | Zitao Li, Jean Honorio |
| Abstract | We introduce a new concept, data irrecoverability, and show that the well-studied concept of data privacy implies data irrecoverability. We show that there are several regularized loss minimization problems that can use locally perturbed data with theoretical guarantees of generalization, i.e., loss consistency. Our results quantitatively connect the convergence rates of the learning problems to the impossibility for any adversary for recovering the original data from perturbed observations. In addition, we show several examples where the convergence rates with perturbed data only increase the convergence rates with original data within a constant factor related to the amount of perturbation, i.e., noise. |
| Tasks | |
| Published | 2018-05-19 |
| URL | http://arxiv.org/abs/1805.07645v5 |
| http://arxiv.org/pdf/1805.07645v5.pdf | |
| PWC | https://paperswithcode.com/paper/regularized-loss-minimizers-with-local-data |
| Repo | |
| Framework | |
Semi-Supervised Clustering with Neural Networks
| Title | Semi-Supervised Clustering with Neural Networks |
| Authors | Ankita Shukla, Gullal Singh Cheema, Saket Anand |
| Abstract | Clustering using neural networks has recently demonstrated promising performance in machine learning and computer vision applications. However, the performance of current approaches is limited either by unsupervised learning or their dependence on large set of labeled data samples. In this paper, we propose ClusterNet that uses pairwise semantic constraints from very few labeled data samples (<5% of total data) and exploits the abundant unlabeled data to drive the clustering approach. We define a new loss function that uses pairwise semantic similarity between objects combined with constrained k-means clustering to efficiently utilize both labeled and unlabeled data in the same framework. The proposed network uses convolution autoencoder to learn a latent representation that groups data into k specified clusters, while also learning the cluster centers simultaneously. We evaluate and compare the performance of ClusterNet on several datasets and state of the art deep clustering approaches. |
| Tasks | Semantic Similarity, Semantic Textual Similarity |
| Published | 2018-06-05 |
| URL | http://arxiv.org/abs/1806.01547v2 |
| http://arxiv.org/pdf/1806.01547v2.pdf | |
| PWC | https://paperswithcode.com/paper/semi-supervised-clustering-with-neural |
| Repo | |
| Framework | |
