Paper Group ANR 366
Understanding Global Loss Landscape of One-hidden-layer ReLU Neural Networks. CNN-based fast source device identification. Combining PRNU and noiseprint for robust and efficient device source identification. Complexity of Shapes Embedded in ${\mathbb Z^n}$ with a Bias Towards Squares. Simulation Pipeline for Traffic Evacuation in Urban Areas and Em …
Understanding Global Loss Landscape of One-hidden-layer ReLU Neural Networks
| Title | Understanding Global Loss Landscape of One-hidden-layer ReLU Neural Networks |
| Authors | Bo Liu |
| Abstract | For one-hidden-layer ReLU networks, we show that all local minima are global in each differentiable region, and these local minima can be unique or continuous, depending on data, activation pattern of hidden neurons and network size. We give criteria to identify whether local minima lie inside their defining regions, and if so (we call them genuine differentiable local minima), their locations and loss values. Furthermore, we give necessary and sufficient conditions for the existence of saddle points as well as non-differentiable local minima. Finally, we compute the probability of getting stuck in genuine local minima for Gaussian input data and parallel weight vectors, and show that it is exponentially vanishing when the weights are located in regions where data are not too scarce. This may give a hint to the question why gradient-based local search methods usually do not get trapped in local minima when training deep ReLU neural networks. |
| Tasks | |
| Published | 2020-02-12 |
| URL | https://arxiv.org/abs/2002.04763v1 |
| https://arxiv.org/pdf/2002.04763v1.pdf | |
| PWC | https://paperswithcode.com/paper/understanding-global-loss-landscape-of-one |
| Repo | |
| Framework | |
CNN-based fast source device identification
| Title | CNN-based fast source device identification |
| Authors | Sara Mandelli, Davide Cozzolino, Paolo Bestagini, Luisa Verdoliva, Stefano Tubaro |
| Abstract | Source identification is an important topic in image forensics, since it allows to trace back the origin of an image. This represents a precious information to claim intellectual property but also to reveal the authors of illicit materials. In this paper we address the problem of device identification based on sensor noise and propose a fast and accurate solution using convolutional neural networks (CNNs). Specifically, we propose a 2-channel-based CNN that learns a way of comparing camera fingerprint and image noise at patch level. The proposed solution turns out to be much faster than the conventional approach and to ensure an increased accuracy. This makes the approach particularly suitable in scenarios where large databases of images are analyzed, like over social networks. In this vein, since images uploaded on social media usually undergo at least two compression stages, we include investigations on double JPEG compressed images, always reporting higher accuracy than standard approaches. |
| Tasks | |
| Published | 2020-01-31 |
| URL | https://arxiv.org/abs/2001.11847v1 |
| https://arxiv.org/pdf/2001.11847v1.pdf | |
| PWC | https://paperswithcode.com/paper/cnn-based-fast-source-device-identification |
| Repo | |
| Framework | |
Combining PRNU and noiseprint for robust and efficient device source identification
| Title | Combining PRNU and noiseprint for robust and efficient device source identification |
| Authors | Davide Cozzolino, Francesco Marra, Diego Gragnaniello, Giovanni Poggi, Luisa Verdoliva |
| Abstract | PRNU-based image processing is a key asset in digital multimedia forensics. It allows for reliable device identification and effective detection and localization of image forgeries, in very general conditions. However, performance impairs significantly in challenging conditions involving low quality and quantity of data. These include working on compressed and cropped images, or estimating the camera PRNU pattern based on only a few images. To boost the performance of PRNU-based analyses in such conditions we propose to leverage the image noiseprint, a recently proposed camera-model fingerprint that has proved effective for several forensic tasks. Numerical experiments on datasets widely used for source identification prove that the proposed method ensures a significant performance improvement in a wide range of challenging situations. |
| Tasks | |
| Published | 2020-01-17 |
| URL | https://arxiv.org/abs/2001.06440v1 |
| https://arxiv.org/pdf/2001.06440v1.pdf | |
| PWC | https://paperswithcode.com/paper/combining-prnu-and-noiseprint-for-robust-and |
| Repo | |
| Framework | |
Complexity of Shapes Embedded in ${\mathbb Z^n}$ with a Bias Towards Squares
| Title | Complexity of Shapes Embedded in ${\mathbb Z^n}$ with a Bias Towards Squares |
| Authors | M. Ferhat Arslan, Sibel Tari |
| Abstract | Shape complexity is a hard-to-quantify quality, mainly due to its relative nature. Biased by Euclidean thinking, circles are commonly considered as the simplest. However, their constructions as digital images are only approximations to the ideal form. Consequently, complexity orders computed in reference to circle are unstable. Unlike circles which lose their circleness in digital images, squares retain their qualities. Hence, we consider squares (hypercubes in $\mathbb Z^n$) to be the simplest shapes relative to which complexity orders are constructed. Using the connection between $L^\infty$ norm and squares we effectively encode squareness-adapted simplification through which we obtain multi-scale complexity measure, where scale determines the level of interest to the boundary. The emergent scale above which the effect of a boundary feature (appendage) disappears is related to the ratio of the contacting width of the appendage to that of the main body. We discuss what zero complexity implies in terms of information repetition and constructibility and what kind of shapes in addition to squares have zero complexity. |
| Tasks | |
| Published | 2020-03-16 |
| URL | https://arxiv.org/abs/2003.07341v1 |
| https://arxiv.org/pdf/2003.07341v1.pdf | |
| PWC | https://paperswithcode.com/paper/complexity-of-shapes-embedded-in-mathbb-zn |
| Repo | |
| Framework | |
Simulation Pipeline for Traffic Evacuation in Urban Areas and Emergency Traffic Management Policy Improvements
| Title | Simulation Pipeline for Traffic Evacuation in Urban Areas and Emergency Traffic Management Policy Improvements |
| Authors | Yu Chen, S. Yusef Shafi, Yi-fan Chen |
| Abstract | Traffic evacuation plays a critical role in saving lives in devastating disasters such as hurricanes, wildfires, floods, earthquakes, etc. An ability to evaluate evacuation plans in advance for these rare events, including identifying traffic flow bottlenecks, improving traffic management policies, and understanding the robustness of the traffic management policy are critical for emergency management. Given the rareness of such events and the corresponding lack of real data, traffic simulation provides a flexible and versatile approach for such scenarios, and furthermore allows dynamic interaction with the simulated evacuation. In this paper, we build a traffic simulation pipeline to explore the above problems, covering many aspects of evacuation, including map creation, demand generation, vehicle behavior, bottleneck identification, traffic management policy improvement, and results analysis. We apply the pipeline to two case studies in California. The first is Paradise, which was destroyed by a large wildfire in 2018 and experienced catastrophic traffic jams during the evacuation. The second is Mill Valley, which has high risk of wildfire and potential traffic issues since the city is situated in a narrow valley. |
| Tasks | |
| Published | 2020-02-14 |
| URL | https://arxiv.org/abs/2002.06198v1 |
| https://arxiv.org/pdf/2002.06198v1.pdf | |
| PWC | https://paperswithcode.com/paper/simulation-pipeline-for-traffic-evacuation-in |
| Repo | |
| Framework | |
Adversarial Multi-Binary Neural Network for Multi-class Classification
| Title | Adversarial Multi-Binary Neural Network for Multi-class Classification |
| Authors | Haiyang Xu, Junwen Chen, Kun Han, Xiangang Li |
| Abstract | Multi-class text classification is one of the key problems in machine learning and natural language processing. Emerging neural networks deal with the problem using a multi-output softmax layer and achieve substantial progress, but they do not explicitly learn the correlation among classes. In this paper, we use a multi-task framework to address multi-class classification, where a multi-class classifier and multiple binary classifiers are trained together. Moreover, we employ adversarial training to distinguish the class-specific features and the class-agnostic features. The model benefits from better feature representation. We conduct experiments on two large-scale multi-class text classification tasks and demonstrate that the proposed architecture outperforms baseline approaches. |
| Tasks | Text Classification |
| Published | 2020-03-25 |
| URL | https://arxiv.org/abs/2003.11184v1 |
| https://arxiv.org/pdf/2003.11184v1.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-multi-binary-neural-network-for |
| Repo | |
| Framework | |
NAS-Count: Counting-by-Density with Neural Architecture Search
| Title | NAS-Count: Counting-by-Density with Neural Architecture Search |
| Authors | Yutao Hu, Xiaolong Jiang, Xuhui Liu, Baochang Zhang, Jungong Han, Xianbin Cao, David Doermann |
| Abstract | Most of the recent advances in crowd counting have evolved from hand-designed density estimation networks, where multi-scale features are leveraged to address scale variation, but at the expense of demanding design efforts. In this work, we automate the design of counting models with Neural Architecture Search (NAS) and introduce an end-to-end searched encoder-decoder architecture, Automatic Multi-Scale Network (AMSNet). The encoder and decoder in AMSNet are composed of different cells discovered from counting-specific search spaces, each dedicated to extracting and aggregating multi-scale features adaptively. To resolve the pixel-level isolation issue in training density estimation models, AMSNet is optimized with a novel Scale Pyramid Pooling Loss (SPPLoss), which exploits a pyramidal architecture to achieve structural supervision at multiple scales. During training time, AMSNet and SPPLoss are searched end-to-end efficiently with differentiable NAS techniques. When testing, AMSNet produces state-of-the-art results that are considerably better than hand-designed models on four challenging datasets, fully demonstrating the efficacy of NAS-Count. |
| Tasks | Crowd Counting, Density Estimation, Neural Architecture Search |
| Published | 2020-02-29 |
| URL | https://arxiv.org/abs/2003.00217v1 |
| https://arxiv.org/pdf/2003.00217v1.pdf | |
| PWC | https://paperswithcode.com/paper/nas-count-counting-by-density-with-neural |
| Repo | |
| Framework | |
Convolutional Neural Networks and a Transfer Learning Strategy to Classify Parkinson’s Disease from Speech in Three Different Languages
| Title | Convolutional Neural Networks and a Transfer Learning Strategy to Classify Parkinson’s Disease from Speech in Three Different Languages |
| Authors | J. C. Vásquez-Correa, T. Arias-Vergara, C. D. Rios-Urrego, M. Schuster, J. Rusz, J. R. Orozco-Arroyave, E. Nöth |
| Abstract | Parkinson’s disease patients develop different speech impairments that affect their communication capabilities. The automatic assessment of the speech of the patients allows the development of computer aided tools to support the diagnosis and the evaluation of the disease severity. This paper introduces a methodology to classify Parkinson’s disease from speech in three different languages: Spanish, German, and Czech. The proposed approach considers convolutional neural networks trained with time frequency representations and a transfer learning strategy among the three languages. The transfer learning scheme aims to improve the accuracy of the models when the weights of the neural network are initialized with utterances from a different language than the used for the test set. The results suggest that the proposed strategy improves the accuracy of the models in up to 8% when the base model used to initialize the weights of the classifier is robust enough. In addition, the results obtained after the transfer learning are in most cases more balanced in terms of specificity-sensitivity than those trained without the transfer learning strategy. |
| Tasks | Transfer Learning |
| Published | 2020-02-11 |
| URL | https://arxiv.org/abs/2002.04374v1 |
| https://arxiv.org/pdf/2002.04374v1.pdf | |
| PWC | https://paperswithcode.com/paper/convolutional-neural-networks-and-a-transfer |
| Repo | |
| Framework | |
A “Network Pruning Network” Approach to Deep Model Compression
| Title | A “Network Pruning Network” Approach to Deep Model Compression |
| Authors | Vinay Kumar Verma, Pravendra Singh, Vinay P. Namboodiri, Piyush Rai |
| Abstract | We present a filter pruning approach for deep model compression, using a multitask network. Our approach is based on learning a a pruner network to prune a pre-trained target network. The pruner is essentially a multitask deep neural network with binary outputs that help identify the filters from each layer of the original network that do not have any significant contribution to the model and can therefore be pruned. The pruner network has the same architecture as the original network except that it has a multitask/multi-output last layer containing binary-valued outputs (one per filter), which indicate which filters have to be pruned. The pruner’s goal is to minimize the number of filters from the original network by assigning zero weights to the corresponding output feature-maps. In contrast to most of the existing methods, instead of relying on iterative pruning, our approach can prune the network (original network) in one go and, moreover, does not require specifying the degree of pruning for each layer (and can learn it instead). The compressed model produced by our approach is generic and does not need any special hardware/software support. Moreover, augmenting with other methods such as knowledge distillation, quantization, and connection pruning can increase the degree of compression for the proposed approach. We show the efficacy of our proposed approach for classification and object detection tasks. |
| Tasks | Model Compression, Network Pruning, Object Detection, Quantization |
| Published | 2020-01-15 |
| URL | https://arxiv.org/abs/2001.05545v1 |
| https://arxiv.org/pdf/2001.05545v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-network-pruning-network-approach-to-deep |
| Repo | |
| Framework | |
NASS: Optimizing Secure Inference via Neural Architecture Search
| Title | NASS: Optimizing Secure Inference via Neural Architecture Search |
| Authors | Song Bian, Weiwen Jiang, Qing Lu, Yiyu Shi, Takashi Sato |
| Abstract | Due to increasing privacy concerns, neural network (NN) based secure inference (SI) schemes that simultaneously hide the client inputs and server models attract major research interests. While existing works focused on developing secure protocols for NN-based SI, in this work, we take a different approach. We propose NASS, an integrated framework to search for tailored NN architectures designed specifically for SI. In particular, we propose to model cryptographic protocols as design elements with associated reward functions. The characterized models are then adopted in a joint optimization with predicted hyperparameters in identifying the best NN architectures that balance prediction accuracy and execution efficiency. In the experiment, it is demonstrated that we can achieve the best of both worlds by using NASS, where the prediction accuracy can be improved from 81.6% to 84.6%, while the inference runtime is reduced by 2x and communication bandwidth by 1.9x on the CIFAR-10 dataset. |
| Tasks | Neural Architecture Search |
| Published | 2020-01-30 |
| URL | https://arxiv.org/abs/2001.11854v3 |
| https://arxiv.org/pdf/2001.11854v3.pdf | |
| PWC | https://paperswithcode.com/paper/nass-optimizing-secure-inference-via-neural |
| Repo | |
| Framework | |
Quantifying the Value of Lateral Views in Deep Learning for Chest X-rays
| Title | Quantifying the Value of Lateral Views in Deep Learning for Chest X-rays |
| Authors | Mohammad Hashir, Hadrien Bertrand, Joseph Paul Cohen |
| Abstract | Most deep learning models in chest X-ray prediction utilize the posteroanterior (PA) view due to the lack of other views available. PadChest is a large-scale chest X-ray dataset that has almost 200 labels and multiple views available. In this work, we use PadChest to explore multiple approaches to merging the PA and lateral views for predicting the radiological labels associated with the X-ray image. We find that different methods of merging the model utilize the lateral view differently. We also find that including the lateral view increases performance for 32 labels in the dataset, while being neutral for the others. The increase in overall performance is comparable to the one obtained by using only the PA view with twice the amount of patients in the training set. |
| Tasks | |
| Published | 2020-02-07 |
| URL | https://arxiv.org/abs/2002.02582v1 |
| https://arxiv.org/pdf/2002.02582v1.pdf | |
| PWC | https://paperswithcode.com/paper/quantifying-the-value-of-lateral-views-in |
| Repo | |
| Framework | |
On the Approximability of Weighted Model Integration on DNF Structures
| Title | On the Approximability of Weighted Model Integration on DNF Structures |
| Authors | Ralph Abboud, Ismail Ilkan Ceylan, Radoslav Dimitrov |
| Abstract | Weighted model counting (WMC) consists of computing the weighted sum of all satisfying assignments of a propositional formula. WMC is well-known to be #P-hard for exact solving, but admits a fully polynomial randomized approximation scheme (FPRAS) when restricted to DNF structures. In this work, we study weighted model integration, a generalization of weighted model counting which involves real variables in addition to propositional variables, and pose the following question: Does weighted model integration on DNF structures admit an FPRAS? Building on classical results from approximate volume computation and approximate weighted model counting, we show that weighted model integration on DNF structures can indeed be approximated for a class of weight functions. Our approximation algorithm is based on three subroutines, each of which can be a weak (i.e., approximate), or a strong (i.e., exact) oracle, and in all cases, comes along with accuracy guarantees. We experimentally verify our approach over randomly generated DNF instances of varying sizes, and show that our algorithm scales to large problem instances, involving up to 1K variables, which are currently out of reach for existing, general-purpose weighted model integration solvers. |
| Tasks | |
| Published | 2020-02-17 |
| URL | https://arxiv.org/abs/2002.06726v2 |
| https://arxiv.org/pdf/2002.06726v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-approximability-of-weighted-model |
| Repo | |
| Framework | |
Multicategory Angle-based Learning for Estimating Optimal Dynamic Treatment Regimes with Censored Data
| Title | Multicategory Angle-based Learning for Estimating Optimal Dynamic Treatment Regimes with Censored Data |
| Authors | Fei Xue, Yanqing Zhang, Wenzhuo Zhou, Haoda Fu, Annie Qu |
| Abstract | An optimal dynamic treatment regime (DTR) consists of a sequence of decision rules in maximizing long-term benefits, which is applicable for chronic diseases such as HIV infection or cancer. In this paper, we develop a novel angle-based approach to search the optimal DTR under a multicategory treatment framework for survival data. The proposed method targets maximization the conditional survival function of patients following a DTR. In contrast to most existing approaches which are designed to maximize the expected survival time under a binary treatment framework, the proposed method solves the multicategory treatment problem given multiple stages for censored data. Specifically, the proposed method obtains the optimal DTR via integrating estimations of decision rules at multiple stages into a single multicategory classification algorithm without imposing additional constraints, which is also more computationally efficient and robust. In theory, we establish Fisher consistency of the proposed method under regularity conditions. Our numerical studies show that the proposed method outperforms competing methods in terms of maximizing the conditional survival function. We apply the proposed method to two real datasets: Framingham heart study data and acquired immunodeficiency syndrome (AIDS) clinical data. |
| Tasks | |
| Published | 2020-01-14 |
| URL | https://arxiv.org/abs/2001.04629v1 |
| https://arxiv.org/pdf/2001.04629v1.pdf | |
| PWC | https://paperswithcode.com/paper/multicategory-angle-based-learning-for |
| Repo | |
| Framework | |
Rethinking Class Relations: Absolute-relative Few-shot Learning
| Title | Rethinking Class Relations: Absolute-relative Few-shot Learning |
| Authors | Hongguang Zhang, Philip H. S. Torr, Hongdong Li, Songlei Jian, Piotr Koniusz |
| Abstract | The majority of existing few-shot learning describe image relations with {0,1} binary labels. However, such binary relations are insufficient to teach the network complicated real-world relations, due to the lack of decision smoothness. Furthermore, current few-shot learning models capture only the similarity via relation labels, but they are not exposed to class concepts associated with objects, which is likely detrimental to the classification performance due to underutilization of the available class labels. To paraphrase, while children learn the concept of tiger from a few of examples with ease, and while they learn from comparisons of tiger to other animals, they are also taught the actual concept names. Thus, we hypothesize that in fact both similarity and class concept learning must be occurring simultaneously. With these observations at hand, we study the fundamental problem of simplistic class modeling in current few-shot learning, we rethink the relations between class concepts, and propose a novel absolute-relative learning paradigm to fully take advantage of label information to refine the image representations and correct the relation understanding. Our proposed absolute-relative learning paradigm improves the performance of several the state-of-the-art models on publicly available datasets. |
| Tasks | Few-Shot Learning |
| Published | 2020-01-12 |
| URL | https://arxiv.org/abs/2001.03919v1 |
| https://arxiv.org/pdf/2001.03919v1.pdf | |
| PWC | https://paperswithcode.com/paper/rethinking-class-relations-absolute-relative |
| Repo | |
| Framework | |
Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot
| Title | Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot |
| Authors | Xuan Liu, Renato Gasoto, Cagdas Onal, Jie Fu |
| Abstract | In this paper, we present a new locomotion control method for soft robot snakes. Inspired by biological snakes, our control architecture is composed of two key modules: A deep reinforcement learning (RL) module for achieving adaptive goal-tracking behaviors with changing goals, and a central pattern generator (CPG) system with Matsuoka oscillators for generating stable and diverse locomotion patterns. The two modules are interconnected into a closed-loop system: The RL module, analogizing the locomotion region located in the midbrain of vertebrate animals, regulates the input to the CPG system given state feedback from the robot. The output of the CPG system is then translated into pressure inputs to pneumatic actuators of the soft snake robot. Based on the fact that the oscillation frequency and wave amplitude of the Matsuoka oscillator can be independently controlled under different time scales, we further adapt the option-critic framework to improve the learning performance measured by optimality and data efficiency. The performance of the proposed controller is experimentally validated with both simulated and real soft snake robots. |
| Tasks | |
| Published | 2020-01-13 |
| URL | https://arxiv.org/abs/2001.04059v2 |
| https://arxiv.org/pdf/2001.04059v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-locomote-with-deep-neural-network |
| Repo | |
| Framework | |
