Paper Group ANR 44
Deep Neural Networks for Rotation-Invariance Approximation and Learning. Convolutional Mixture Density Recurrent Neural Network for Predicting User Location with WiFi Fingerprints. Improving Distant Supervised Relation Extraction by Dynamic Neural Network. Weakly-paired Cross-Modal Hashing. MeshGAN: Non-linear 3D Morphable Models of Faces. Guardian …
Deep Neural Networks for Rotation-Invariance Approximation and Learning
| Title | Deep Neural Networks for Rotation-Invariance Approximation and Learning |
| Authors | Charles K. Chui, Shao-Bo Lin, Ding-Xuan Zhou |
| Abstract | Based on the tree architecture, the objective of this paper is to design deep neural networks with two or more hidden layers (called deep nets) for realization of radial functions so as to enable rotational invariance for near-optimal function approximation in an arbitrarily high dimensional Euclidian space. It is shown that deep nets have much better performance than shallow nets (with only one hidden layer) in terms of approximation accuracy and learning capabilities. In particular, for learning radial functions, it is shown that near-optimal rate can be achieved by deep nets but not by shallow nets. Our results illustrate the necessity of depth in neural network design for realization of rotation-invariance target functions. |
| Tasks | |
| Published | 2019-04-03 |
| URL | http://arxiv.org/abs/1904.01814v1 |
| http://arxiv.org/pdf/1904.01814v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-neural-networks-for-rotation-invariance |
| Repo | |
| Framework | |
Convolutional Mixture Density Recurrent Neural Network for Predicting User Location with WiFi Fingerprints
| Title | Convolutional Mixture Density Recurrent Neural Network for Predicting User Location with WiFi Fingerprints |
| Authors | Weizhu Qian, Fabrice Lauri, Franck Gechter |
| Abstract | Predicting smartphone users activity using WiFi fingerprints has been a popular approach for indoor positioning in recent years. However, such a high dimensional time-series prediction problem can be very tricky to solve. To address this issue, we propose a novel deep learning model, the convolutional mixture density recurrent neural network (CMDRNN), which combines the strengths of convolutional neural networks, recurrent neural networks and mixture density networks. In our model, the CNN sub-model is employed to detect the feature of the high dimensional input, the RNN sub-model is utilized to capture the time dependency and the MDN sub-model is for predicting the final output. For validation, we conduct the experiments on the real-world dataset and the obtained results illustrate the effectiveness of our method. |
| Tasks | Time Series, Time Series Prediction |
| Published | 2019-11-21 |
| URL | https://arxiv.org/abs/1911.09344v1 |
| https://arxiv.org/pdf/1911.09344v1.pdf | |
| PWC | https://paperswithcode.com/paper/convolutional-mixture-density-recurrent |
| Repo | |
| Framework | |
Improving Distant Supervised Relation Extraction by Dynamic Neural Network
| Title | Improving Distant Supervised Relation Extraction by Dynamic Neural Network |
| Authors | Yanjie Gou, Yinjie Lei, Lingqiao Liu, Pingping Zhang, Xi Peng |
| Abstract | Distant Supervised Relation Extraction (DSRE) is usually formulated as a problem of classifying a bag of sentences that contain two query entities, into the predefined relation classes. Most existing methods consider those relation classes as distinct semantic categories while ignoring their potential connection to query entities. In this paper, we propose to leverage this connection to improve the relation extraction accuracy. Our key ideas are twofold: (1) For sentences belonging to the same relation class, the expression style, i.e. words choice, can vary according to the query entities. To account for this style shift, the model should adjust its parameters in accordance with entity types. (2) Some relation classes are semantically similar, and the entity types appear in one relation may also appear in others. Therefore, it can be trained cross different relation classes and further enhance those classes with few samples, i.e., long-tail classes. To unify these two arguments, we developed a novel Dynamic Neural Network for Relation Extraction (DNNRE). The network adopts a novel dynamic parameter generator that dynamically generates the network parameters according to the query entity types and relation classes. By using this mechanism, the network can simultaneously handle the style shift problem and enhance the prediction accuracy for long-tail classes. Through our experimental study, we demonstrate the effectiveness of the proposed method and show that it can achieve superior performance over the state-of-the-art methods. |
| Tasks | Relation Extraction |
| Published | 2019-11-15 |
| URL | https://arxiv.org/abs/1911.06489v2 |
| https://arxiv.org/pdf/1911.06489v2.pdf | |
| PWC | https://paperswithcode.com/paper/dnnre-a-dynamic-neural-network-for-distant |
| Repo | |
| Framework | |
Weakly-paired Cross-Modal Hashing
| Title | Weakly-paired Cross-Modal Hashing |
| Authors | Xuanwu Liu, Jun Wang, Guoxian Yu, Carlotta Domeniconi, Xiangliang Zhang |
| Abstract | Hashing has been widely adopted for large-scale data retrieval in many domains, due to its low storage cost and high retrieval speed. Existing cross-modal hashing methods optimistically assume that the correspondence between training samples across modalities are readily available. This assumption is unrealistic in practical applications. In addition, these methods generally require the same number of samples across different modalities, which restricts their flexibility. We propose a flexible cross-modal hashing approach (Flex-CMH) to learn effective hashing codes from weakly-paired data, whose correspondence across modalities are partially (or even totally) unknown. FlexCMH first introduces a clustering-based matching strategy to explore the local structure of each cluster, and thus to find the potential correspondence between clusters (and samples therein) across modalities. To reduce the impact of an incomplete correspondence, it jointly optimizes in a unified objective function the potential correspondence, the cross-modal hashing functions derived from the correspondence, and a hashing quantitative loss. An alternative optimization technique is also proposed to coordinate the correspondence and hash functions, and to reinforce the reciprocal effects of the two objectives. Experiments on publicly multi-modal datasets show that FlexCMH achieves significantly better results than state-of-the-art methods, and it indeed offers a high degree of flexibility for practical cross-modal hashing tasks. |
| Tasks | |
| Published | 2019-05-29 |
| URL | https://arxiv.org/abs/1905.12203v1 |
| https://arxiv.org/pdf/1905.12203v1.pdf | |
| PWC | https://paperswithcode.com/paper/weakly-paired-cross-modal-hashing |
| Repo | |
| Framework | |
MeshGAN: Non-linear 3D Morphable Models of Faces
| Title | MeshGAN: Non-linear 3D Morphable Models of Faces |
| Authors | Shiyang Cheng, Michael Bronstein, Yuxiang Zhou, Irene Kotsia, Maja Pantic, Stefanos Zafeiriou |
| Abstract | Generative Adversarial Networks (GANs) are currently the method of choice for generating visual data. Certain GAN architectures and training methods have demonstrated exceptional performance in generating realistic synthetic images (in particular, of human faces). However, for 3D object, GANs still fall short of the success they have had with images. One of the reasons is due to the fact that so far GANs have been applied as 3D convolutional architectures to discrete volumetric representations of 3D objects. In this paper, we propose the first intrinsic GANs architecture operating directly on 3D meshes (named as MeshGAN). Both quantitative and qualitative results are provided to show that MeshGAN can be used to generate high-fidelity 3D face with rich identities and expressions. |
| Tasks | |
| Published | 2019-03-25 |
| URL | http://arxiv.org/abs/1903.10384v1 |
| http://arxiv.org/pdf/1903.10384v1.pdf | |
| PWC | https://paperswithcode.com/paper/meshgan-non-linear-3d-morphable-models-of |
| Repo | |
| Framework | |
Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud
| Title | Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud |
| Authors | Ashkan Yousefpour, Siddartha Devic, Brian Q. Nguyen, Aboudy Kreidieh, Alan Liao, Alexandre M. Bayen, Jason P. Jue |
| Abstract | Partitioning and distributing deep neural networks (DNNs) over physical nodes such as edge, fog, or cloud nodes, could enhance sensor fusion, and reduce bandwidth and inference latency. However, when a DNN is distributed over physical nodes, failure of the physical nodes causes the failure of the DNN units that are placed on these nodes. The performance of the inference task will be unpredictable, and most likely, poor, if the distributed DNN is not specifically designed and properly trained for failures. Motivated by this, we introduce deepFogGuard, a DNN architecture augmentation scheme for making the distributed DNN inference task failure-resilient. To articulate deepFogGuard, we introduce the elements and a model for the resiliency of distributed DNN inference. Inspired by the concept of residual connections in DNNs, we introduce skip hyperconnections in distributed DNNs, which are the basis of deepFogGuard’s design to provide resiliency. Next, our extensive experiments using two existing datasets for the sensing and vision applications confirm the ability of deepFogGuard to provide resiliency for distributed DNNs in edge-cloud networks. |
| Tasks | Sensor Fusion |
| Published | 2019-09-03 |
| URL | https://arxiv.org/abs/1909.00995v2 |
| https://arxiv.org/pdf/1909.00995v2.pdf | |
| PWC | https://paperswithcode.com/paper/guardians-of-the-deep-fog-failure-resilient |
| Repo | |
| Framework | |
Training Deep Learning models with small datasets
| Title | Training Deep Learning models with small datasets |
| Authors | Miguel Romero, Yannet Interian, Timothy Solberg, Gilmer Valdes |
| Abstract | The growing use of Machine Learning has produced significant advances in many fields. For image-based tasks, however, the use of deep learning remains challenging in small datasets. In this article, we review, evaluate and compare the current state of the art techniques in training neural networks to elucidate which techniques work best for small datasets. We further propose a path forward for the improvement of model accuracy in medical imaging applications. We observed best results from one cycle training, discriminative learning rates with gradual freezing and parameter modification after transfer learning. We also established that when datasets are small, transfer learning plays an important role beyond parameter initialization by reusing previously learned features. Surprisingly we observed that there is little advantage in using pre-trained networks in images from another part of the body compared to Imagenet. On the contrary, if images from the same part of the body are available then transfer learning can produce a significant improvement in performance with as little as 50 images in the training data. |
| Tasks | Transfer Learning |
| Published | 2019-12-14 |
| URL | https://arxiv.org/abs/1912.06761v1 |
| https://arxiv.org/pdf/1912.06761v1.pdf | |
| PWC | https://paperswithcode.com/paper/training-deep-learning-models-with-small |
| Repo | |
| Framework | |
A Pattern-Hierarchy Classifier for Reduced Teaching
| Title | A Pattern-Hierarchy Classifier for Reduced Teaching |
| Authors | Kieran Greer |
| Abstract | This paper uses a branching classifier mechanism in an unsupervised scenario, to enable it to self-organise data into unknown categories. A teaching phase is then able to help the classifier to learn the true category for each input row, using a reduced number of training steps. The pattern ensembles are learned in an unsupervsised manner that use a closest-distance clustering. This is done without knowing what the actual output category is and leads to each actual category having several clusters associated with it. One measure of success is then that each of these sub-clusters is coherent, which means that every data row in the cluster belongs to the same category. The total number of clusters is also important and a teaching phase can then teach the classifier what the correct actual category is. During this phase, any classifier can also learn or infer correct classifications from some other classifier’s knowledge, thereby reducing the required number of presentations. As the information is added, cross-referencing between the two structures allows it to be used more widely. With this process, a unique structure can build up that would not be possible by either method separately. The lower level is a nested ensemble of patterns created by self-organisation. The upper level is a hierarchical tree, where each end node represents a single category only, so there is a transition from mixed ensemble masses to specific categories. The structure also has relations to brain-like modelling. |
| Tasks | |
| Published | 2019-04-16 |
| URL | http://arxiv.org/abs/1904.07786v1 |
| http://arxiv.org/pdf/1904.07786v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-pattern-hierarchy-classifier-for-reduced |
| Repo | |
| Framework | |
A Comprehensive Study on Deep Learning Bug Characteristics
| Title | A Comprehensive Study on Deep Learning Bug Characteristics |
| Authors | Md Johirul Islam, Giang Nguyen, Rangeet Pan, Hridesh Rajan |
| Abstract | Deep learning has gained substantial popularity in recent years. Developers mainly rely on libraries and tools to add deep learning capabilities to their software. What kinds of bugs are frequently found in such software? What are the root causes of such bugs? What impacts do such bugs have? Which stages of deep learning pipeline are more bug prone? Are there any antipatterns? Understanding such characteristics of bugs in deep learning software has the potential to foster the development of better deep learning platforms, debugging mechanisms, development practices, and encourage the development of analysis and verification frameworks. Therefore, we study 2716 high-quality posts from Stack Overflow and 500 bug fix commits from Github about five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand the types of bugs, root causes of bugs, impacts of bugs, bug-prone stage of deep learning pipeline as well as whether there are some common antipatterns found in this buggy software. The key findings of our study include: data bug and logic bug are the most severe bug types in deep learning software appearing more than 48% of the times, major root causes of these bugs are Incorrect Model Parameter (IPS) and Structural Inefficiency (SI) showing up more than 43% of the times. We have also found that the bugs in the usage of deep learning libraries have some common antipatterns that lead to a strong correlation of bug types among the libraries. |
| Tasks | |
| Published | 2019-06-03 |
| URL | https://arxiv.org/abs/1906.01388v1 |
| https://arxiv.org/pdf/1906.01388v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-comprehensive-study-on-deep-learning-bug |
| Repo | |
| Framework | |
Forecasting Intra-Hour Imbalances in Electric Power Systems
| Title | Forecasting Intra-Hour Imbalances in Electric Power Systems |
| Authors | Tárik S. Salem, Karan Kathuria, Heri Ramampiaro, Helge Langseth |
| Abstract | Keeping the electricity production in balance with the actual demand is becoming a difficult and expensive task in spite of an involvement of experienced human operators. This is due to the increasing complexity of the electric power grid system with the intermittent renewable production as one of the contributors. A beforehand information about an occurring imbalance can help the transmission system operator to adjust the production plans, and thus ensure a high security of supply by reducing the use of costly balancing reserves, and consequently reduce undesirable fluctuations of the 50 Hz power system frequency. In this paper, we introduce the relatively new problem of an intra-hour imbalance forecasting for the transmission system operator (TSO). We focus on the use case of the Norwegian TSO, Statnett. We present a complementary imbalance forecasting tool that is able to support the TSO in determining the trend of future imbalances, and show the potential to proactively alleviate imbalances with a higher accuracy compared to the contemporary solution. |
| Tasks | |
| Published | 2019-02-01 |
| URL | http://arxiv.org/abs/1902.00563v1 |
| http://arxiv.org/pdf/1902.00563v1.pdf | |
| PWC | https://paperswithcode.com/paper/forecasting-intra-hour-imbalances-in-electric |
| Repo | |
| Framework | |
Parallel optimization of fiber bundle segmentation for massive tractography datasets
| Title | Parallel optimization of fiber bundle segmentation for massive tractography datasets |
| Authors | Andrea Vázquez, Narciso López-López, Nicole Labra, Miguel Figueroa, Cyril Poupon, Jean-François Mangin, Cecilia Hernández, Pamela Guevara |
| Abstract | We present an optimized algorithm that performs automatic classification of white matter fibers based on a multi-subject bundle atlas. We implemented a parallel algorithm that improves upon its previous version in both execution time and memory usage. Our new version uses the local memory of each processor, which leads to a reduction in execution time. Hence, it allows the analysis of bigger subject and/or atlas datasets. As a result, the segmentation of a subject of 4,145,000 fibers is reduced from about 14 minutes in the previous version to about 6 minutes, yielding an acceleration of 2.34. In addition, the new algorithm reduces the memory consumption of the previous version by a factor of 0.79. |
| Tasks | |
| Published | 2019-12-24 |
| URL | https://arxiv.org/abs/1912.11494v1 |
| https://arxiv.org/pdf/1912.11494v1.pdf | |
| PWC | https://paperswithcode.com/paper/parallel-optimization-of-fiber-bundle |
| Repo | |
| Framework | |
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor
| Title | UR-FUNNY: A Multimodal Language Dataset for Understanding Humor |
| Authors | Md Kamrul Hasan, Wasifur Rahman, Amir Zadeh, Jianyuan Zhong, Md Iftekhar Tanveer, Louis-Philippe Morency, Mohammed, Hoque |
| Abstract | Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (vision) and prosodic cues (acoustic). Understanding humor from these three modalities falls within boundaries of multimodal language; a recent research trend in natural language processing that models natural language as it happens in face-to-face communication. Although humor detection is an established research area in NLP, in a multimodal context it is an understudied area. This paper presents a diverse multimodal dataset, called UR-FUNNY, to open the door to understanding multimodal language used in expressing humor. The dataset and accompanying studies, present a framework in multimodal humor detection for the natural language processing community. UR-FUNNY is publicly available for research. |
| Tasks | Humor Detection |
| Published | 2019-04-14 |
| URL | http://arxiv.org/abs/1904.06618v1 |
| http://arxiv.org/pdf/1904.06618v1.pdf | |
| PWC | https://paperswithcode.com/paper/ur-funny-a-multimodal-language-dataset-for |
| Repo | |
| Framework | |
Comparisonal study of Deep Learning approaches on Retinal OCT Image
| Title | Comparisonal study of Deep Learning approaches on Retinal OCT Image |
| Authors | Nowshin Tasnim, Mahmudul Hasan, Ishrak Islam |
| Abstract | In medical science, the use of computer science in disease detection and diagnosis is gaining popularity. Previously, the detection of disease used to take a significant amount of time and was less reliable. Machine learning (ML) techniques employed in recent biomedical researches are making revolutionary changes by gaining higher accuracy with more concise timing. At present, it is even possible to automatically detect diseases from the scanned images with the help of ML. In this research, we have taken such an attempt to detect retinal diseases from optical coherence tomography (OCT) X-ray images. Here, we propose a deep learning (DL) based approach in detecting retinal diseases from OCT images which can identify three conditions of the retina. Four different models used in this approach are compared with each other. On the test set, the detection accuracy is 98.00% for a vanilla convolutional neural network (CNN) model, 99.07% for Xception model, 97.00% for ResNet50 model, and 99.17% for MobileNetV2 model. The MobileNetV2 model acquires the highest accuracy, and the closest to the highest is the Xception model. The proposed approach has a potential impact on creating a tool for automatically detecting retinal diseases. |
| Tasks | |
| Published | 2019-12-16 |
| URL | https://arxiv.org/abs/1912.07783v1 |
| https://arxiv.org/pdf/1912.07783v1.pdf | |
| PWC | https://paperswithcode.com/paper/comparisonal-study-of-deep-learning |
| Repo | |
| Framework | |
Modified Actor-Critics
| Title | Modified Actor-Critics |
| Authors | Erinc Merdivan, Sten Hanke, Matthieu Geist |
| Abstract | Recent successful deep reinforcement learning algorithms, such as Trust Region Policy Optimization (TRPO) or Proximal Policy Optimization (PPO), are fundamentally variations of conservative policy iteration (CPI). These algorithms iterate policy evaluation followed by a softened policy improvement step. As so, they are naturally on-policy. In this paper, we propose to combine (any kind of) soft greediness with Modified Policy Iteration (MPI). The proposed abstract framework applies repeatedly: (i) a partial policy evaluation step that allows off-policy learning and (ii) any softened greedy step. Our contribution can be seen as a new generic tool for the deep reinforcement learning toolbox. As a proof of concept, we instantiate this framework with the PPO greediness. Comparison to the original PPO shows that our algorithm is much more sample efficient. We also show that it is competitive with the state-of-art off-policy algorithm Soft Actor Critic (SAC). |
| Tasks | |
| Published | 2019-07-02 |
| URL | https://arxiv.org/abs/1907.01298v2 |
| https://arxiv.org/pdf/1907.01298v2.pdf | |
| PWC | https://paperswithcode.com/paper/modified-actor-critics |
| Repo | |
| Framework | |
Binarized Neural Architecture Search
| Title | Binarized Neural Architecture Search |
| Authors | Hanlin Chen, Li’an Zhuo, Baochang Zhang, Xiawu Zheng, Jianzhuang Liu, David Doermann, Rongrong Ji |
| Abstract | Neural architecture search (NAS) can have a significant impact in computer vision by automatically designing optimal neural network architectures for various tasks. A variant, binarized neural architecture search (BNAS), with a search space of binarized convolutions, can produce extremely compressed models. Unfortunately, this area remains largely unexplored. BNAS is more challenging than NAS due to the learning inefficiency caused by optimization requirements and the huge architecture space. To address these issues, we introduce channel sampling and operation space reduction into a differentiable NAS to significantly reduce the cost of searching. This is accomplished through a performance-based strategy used to abandon less potential operations. Two optimization methods for binarized neural networks are used to validate the effectiveness of our BNAS. Extensive experiments demonstrate that the proposed BNAS achieves a performance comparable to NAS on both CIFAR and ImageNet databases. An accuracy of $96.53%$ vs. $97.22%$ is achieved on the CIFAR-10 dataset, but with a significantly compressed model, and a $40%$ faster search than the state-of-the-art PC-DARTS. |
| Tasks | Neural Architecture Search |
| Published | 2019-11-25 |
| URL | https://arxiv.org/abs/1911.10862v2 |
| https://arxiv.org/pdf/1911.10862v2.pdf | |
| PWC | https://paperswithcode.com/paper/binarized-neural-architecture-search |
| Repo | |
| Framework | |
