Paper Group ANR 1537
Motion Corrected Multishot MRI Reconstruction Using Generative Networks with Sensitivity Encoding. Optimization of Inf-Convolution Regularized Nonconvex Composite Problems. Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures. Landscape Complexity for the Empirical Risk of Generalized Linear Models. Self-supervis …
Motion Corrected Multishot MRI Reconstruction Using Generative Networks with Sensitivity Encoding
| Title | Motion Corrected Multishot MRI Reconstruction Using Generative Networks with Sensitivity Encoding |
| Authors | Muhammad Usman, Muhammad Umar Farooq, Siddique Latif, Muhammad Asim, Junaid Qadir |
| Abstract | Multishot Magnetic Resonance Imaging (MRI) is a promising imaging modality that can produce a high-resolution image with relatively less data acquisition time. The downside of multishot MRI is that it is very sensitive to subject motion and even small amounts of motion during the scan can produce artifacts in the final MR image that may cause misdiagnosis. Numerous efforts have been made to address this issue; however, all of these proposals are limited in terms of how much motion they can correct and the required computational time. In this paper, we propose a novel generative networks based conjugate gradient SENSE (CG-SENSE) reconstruction framework for motion correction in multishot MRI. The proposed framework first employs CG-SENSE reconstruction to produce the motion-corrupted image and then a generative adversarial network (GAN) is used to correct the motion artifacts. The proposed method has been rigorously evaluated on synthetically corrupted data on varying degrees of motion, numbers of shots, and encoding trajectories. Our analyses (both quantitative as well as qualitative/visual analysis) establishes that the proposed method significantly robust and outperforms state-of-the-art motion correction techniques and also reduces severalfold of computational times. |
| Tasks | Motion Correction In Multishot Mri |
| Published | 2019-02-20 |
| URL | https://arxiv.org/abs/1902.07430v6 |
| https://arxiv.org/pdf/1902.07430v6.pdf | |
| PWC | https://paperswithcode.com/paper/motion-corrected-multishot-mri-reconstruction |
| Repo | |
| Framework | |
Optimization of Inf-Convolution Regularized Nonconvex Composite Problems
| Title | Optimization of Inf-Convolution Regularized Nonconvex Composite Problems |
| Authors | Emanuel Laude, Tao Wu, Daniel Cremers |
| Abstract | In this work, we consider nonconvex composite problems that involve inf-convolution with a Legendre function, which gives rise to an anisotropic generalization of the proximal mapping and Moreau-envelope. In a convex setting such problems can be solved via alternating minimization of a splitting formulation, where the consensus constraint is penalized with a Legendre function. In contrast, for nonconvex models it is in general unclear that this approach yields stationary points to the infimal convolution problem. To this end we analytically investigate local regularity properties of the Moreau-envelope function under prox-regularity, which allows us to establish the equivalence between stationary points of the splitting model and the original inf-convolution model. We apply our theory to characterize stationary points of the penalty objective, which is minimized by the elastic averaging SGD (EASGD) method for distributed training. Numerically, we demonstrate the practical relevance of the proposed approach on the important task of distributed training of deep neural networks. |
| Tasks | |
| Published | 2019-03-27 |
| URL | http://arxiv.org/abs/1903.11690v1 |
| http://arxiv.org/pdf/1903.11690v1.pdf | |
| PWC | https://paperswithcode.com/paper/optimization-of-inf-convolution-regularized |
| Repo | |
| Framework | |
Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures
| Title | Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures |
| Authors | Xiaoting Shao, Alejandro Molina, Antonio Vergari, Karl Stelzner, Robert Peharz, Thomas Liebig, Kristian Kersting |
| Abstract | Probabilistic graphical models are a central tool in AI; however, they are generally not as expressive as deep neural models, and inference is notoriously hard and slow. In contrast, deep probabilistic models such as sum-product networks (SPNs) capture joint distributions in a tractable fashion, but still lack the expressive power of intractable models based on deep neural networks. Therefore, we introduce conditional SPNs (CSPNs), conditional density estimators for multivariate and potentially hybrid domains which allow harnessing the expressive power of neural networks while still maintaining tractability guarantees. One way to implement CSPNs is to use an existing SPN structure and condition its parameters on the input, e.g., via a deep neural network. This approach, however, might misrepresent the conditional independence structure present in data. Consequently, we also develop a structure-learning approach that derives both the structure and parameters of CSPNs from data. Our experimental evidence demonstrates that CSPNs are competitive with other probabilistic models and yield superior performance on multilabel image classification compared to mean field and mixture density networks. Furthermore, they can successfully be employed as building blocks for structured probabilistic models, such as autoregressive image models. |
| Tasks | Image Classification |
| Published | 2019-05-21 |
| URL | https://arxiv.org/abs/1905.08550v2 |
| https://arxiv.org/pdf/1905.08550v2.pdf | |
| PWC | https://paperswithcode.com/paper/conditional-sum-product-networks-imposing |
| Repo | |
| Framework | |
Landscape Complexity for the Empirical Risk of Generalized Linear Models
| Title | Landscape Complexity for the Empirical Risk of Generalized Linear Models |
| Authors | Antoine Maillard, Gérard Ben Arous, Giulio Biroli |
| Abstract | We present a method to obtain the average and the typical value of the number of critical points of the empirical risk landscape for generalized linear estimation problems and variants. This represents a substantial extension of previous applications of the Kac-Rice method since it allows to analyze the critical points of high dimensional non-Gaussian random functions. We obtain a rigorous explicit variational formula for the annealed complexity, which is the logarithm of the average number of critical points at fixed value of the empirical risk. This result is simplified, and extended, using the non-rigorous Kac-Rice replicated method from theoretical physics. In this way we find an explicit variational formula for the quenched complexity, which is generally different from its annealed counterpart, and allows to obtain the number of critical points for typical instances up to exponential accuracy. |
| Tasks | |
| Published | 2019-12-04 |
| URL | https://arxiv.org/abs/1912.02143v1 |
| https://arxiv.org/pdf/1912.02143v1.pdf | |
| PWC | https://paperswithcode.com/paper/landscape-complexity-for-the-empirical-risk |
| Repo | |
| Framework | |
Self-supervised Hyperspectral Image Restoration using Separable Image Prior
| Title | Self-supervised Hyperspectral Image Restoration using Separable Image Prior |
| Authors | Ryuji Imamura, Tatsuki Itasaka, Masahiro Okuda |
| Abstract | Supervised learning with a convolutional neural network is recognized as a powerful means of image restoration. However, most such methods have been designed for application to grayscale and/or color images; therefore, they have limited success when applied to hyperspectral image restoration. This is partially owing to large datasets being difficult to collect, and also the heavy computational load associated with the restoration of an image with many spectral bands. To address this difficulty, we propose a novel self-supervised learning strategy for application to hyperspectral image restoration. Our method automatically creates a training dataset from a single degraded image and trains a denoising network without any clear images. Another notable feature of our method is the use of a separable convolutional layer. We undertake experiments to prove that the use of a separable network allows us to acquire the prior of a hyperspectral image and to realize efficient restoration. We demonstrate the validity of our method through extensive experiments and show that our method has better characteristics than those that are currently regarded as state-of-the-art. |
| Tasks | Denoising, Image Restoration |
| Published | 2019-07-01 |
| URL | https://arxiv.org/abs/1907.00651v1 |
| https://arxiv.org/pdf/1907.00651v1.pdf | |
| PWC | https://paperswithcode.com/paper/self-supervised-hyperspectral-image |
| Repo | |
| Framework | |
Algorithmic Design and Implementation of Unobtrusive Multistatic Serial LiDAR Image
| Title | Algorithmic Design and Implementation of Unobtrusive Multistatic Serial LiDAR Image |
| Authors | Chi Ding, Zheng Cao, Matthew S. Emigh, Jose C. Principe, Bing Ouyang, Anni Vuorenkoski, Fraser Dalgleish, Brian Ramos, Yanjun Li |
| Abstract | To fully understand interactions between marine hydrokinetic (MHK) equipment and marine animals, a fast and effective monitoring system is required to capture relevant information whenever underwater animals appear. A new automated underwater imaging system composed of LiDAR (Light Detection and Ranging) imaging hardware and a scene understanding software module named Unobtrusive Multistatic Serial LiDAR Imager (UMSLI) to supervise the presence of animals near turbines. UMSLI integrates the front end LiDAR hardware and a series of software modules to achieve image preprocessing, detection, tracking, segmentation and classification in a hierarchical manner. |
| Tasks | Scene Understanding |
| Published | 2019-11-08 |
| URL | https://arxiv.org/abs/1911.03267v1 |
| https://arxiv.org/pdf/1911.03267v1.pdf | |
| PWC | https://paperswithcode.com/paper/algorithmic-design-and-implementation-of |
| Repo | |
| Framework | |
Hierarchical Photo-Scene Encoder for Album Storytelling
| Title | Hierarchical Photo-Scene Encoder for Album Storytelling |
| Authors | Bairui Wang, Lin Ma, Wei Zhang, Wenhao Jiang, Feng Zhang |
| Abstract | In this paper, we propose a novel model with a hierarchical photo-scene encoder and a reconstructor for the task of album storytelling. The photo-scene encoder contains two sub-encoders, namely the photo and scene encoders, which are stacked together and behave hierarchically to fully exploit the structure information of the photos within an album. Specifically, the photo encoder generates semantic representation for each photo while exploiting temporal relationships among them. The scene encoder, relying on the obtained photo representations, is responsible for detecting the scene changes and generating scene representations. Subsequently, the decoder dynamically and attentively summarizes the encoded photo and scene representations to generate a sequence of album representations, based on which a story consisting of multiple coherent sentences is generated. In order to fully extract the useful semantic information from an album, a reconstructor is employed to reproduce the summarized album representations based on the hidden states of the decoder. The proposed model can be trained in an end-to-end manner, which results in an improved performance over the state-of-the-arts on the public visual storytelling (VIST) dataset. Ablation studies further demonstrate the effectiveness of the proposed hierarchical photo-scene encoder and reconstructor. |
| Tasks | Visual Storytelling |
| Published | 2019-02-02 |
| URL | http://arxiv.org/abs/1902.00669v1 |
| http://arxiv.org/pdf/1902.00669v1.pdf | |
| PWC | https://paperswithcode.com/paper/hierarchical-photo-scene-encoder-for-album |
| Repo | |
| Framework | |
Higher-order algorithms and implicit regularization for nonlinearly parameterized adaptive control
| Title | Higher-order algorithms and implicit regularization for nonlinearly parameterized adaptive control |
| Authors | Nicholas M. Boffi, Jean-Jacques E. Slotine |
| Abstract | Stable concurrent learning and control of dynamical systems is the subject of adaptive control. Adaptive control is a field with many practical applications and a rich theory, but much of the development for nonlinear systems revolves around a few key algorithms. By exploiting strong connections between nonlinear adaptive control techniques and recent progress in optimization and machine learning, we show that there exists considerable untapped potential in algorithm development for nonlinear adaptive control. We present a large set of new globally convergent adaptive control algorithms that are applicable both to linearly parameterized systems and to nonlinearly parameterized systems satisfying certain monotonicity or convexity requirements. We adopt a variational formalism based on the Bregman Lagrangian to define a general framework that systematically generates higher-order in-time velocity gradient algorithms. We generalize our algorithms to the non-Euclidean setting and show that the Euler Lagrange equations for the Bregman Lagrangian lead to natural gradient and mirror descent-like adaptation laws with momentum that incorporate local geometry through a Hessian metric specified by a convex function. We prove that these non-Euclidean adaptation laws implicitly regularize the system model by minimizing the convex function that specifies the metric throughout adaptation. Local geometry imposed during adaptation thus may be used to select parameter vectors - out of the many that will lead to perfect tracking - for desired properties such as sparsity. We illustrate our analysis with simulations using a higher-order algorithm for nonlinearly parameterized systems to learn regularized hidden layer weights in a three-layer feedforward neural network. |
| Tasks | |
| Published | 2019-12-31 |
| URL | https://arxiv.org/abs/1912.13154v3 |
| https://arxiv.org/pdf/1912.13154v3.pdf | |
| PWC | https://paperswithcode.com/paper/higher-order-algorithms-for-nonlinearly |
| Repo | |
| Framework | |
Semi-flat minima and saddle points by embedding neural networks to overparameterization
| Title | Semi-flat minima and saddle points by embedding neural networks to overparameterization |
| Authors | Kenji Fukumizu, Shoichiro Yamaguchi, Yoh-ichi Mototake, Mirai Tanaka |
| Abstract | We theoretically study the landscape of the training error for neural networks in overparameterized cases. We consider three basic methods for embedding a network into a wider one with more hidden units, and discuss whether a minimum point of the narrower network gives a minimum or saddle point of the wider one. Our results show that the networks with smooth and ReLU activation have different partially flat landscapes around the embedded point. We also relate these results to a difference of their generalization abilities in overparameterized realization. |
| Tasks | |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.04868v2 |
| https://arxiv.org/pdf/1906.04868v2.pdf | |
| PWC | https://paperswithcode.com/paper/semi-flat-minima-and-saddle-points-by |
| Repo | |
| Framework | |
Extrinsic Kernel Ridge Regression Classifier for Planar Kendall Shape Space
| Title | Extrinsic Kernel Ridge Regression Classifier for Planar Kendall Shape Space |
| Authors | Hwiyoung Lee, Vic Patrangenaru |
| Abstract | Kernel methods have had great success in the statistics and machine learning community. Despite their growing popularity, however, less effort has been drawn towards developing kernel based classification methods on manifold due to the non-Euclidean geometry. In this paper, motivated by the extrinsic framework of manifold-valued data analysis, we propose two types of new kernels on planar Kendall shape space $\Sigma_2^k$, called extrinsic Veronese Whitney Gaussian kernel and extrinsic complex Gaussian kernel. We show that our approach can be extended to develop Gaussian like kernels on any embedded manifold. Furthermore, kernel ridge regression classifier (KRRC) is implemented to address the shape classification problem on $\Sigma_2^k$, and their promising performances are illustrated through the real dataset. |
| Tasks | |
| Published | 2019-12-17 |
| URL | https://arxiv.org/abs/1912.08202v1 |
| https://arxiv.org/pdf/1912.08202v1.pdf | |
| PWC | https://paperswithcode.com/paper/extrinsic-kernel-ridge-regression-classifier |
| Repo | |
| Framework | |
Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help?
| Title | Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help? |
| Authors | Yunyang Xiong, Ronak Mehta, Vikas Singh |
| Abstract | The design of neural network architectures is frequently either based on human expertise using trial/error and empirical feedback or tackled via large scale reinforcement learning strategies performed over distinct discrete architecture choices. In the latter case, the optimization is often non-differentiable and also not very amenable to derivative-free optimization methods. Most methods in use today require sizable computational resources. And if we want networks that additionally satisfy resource constraints, the above challenges are exacerbated because the search must now balance accuracy with certain budget constraints on resources. We formulate this problem as the optimization of a set function – we find that the empirical behavior of this set function often (but not always) satisfies marginal gain and monotonicity principles – properties central to the idea of submodularity. Based on this observation, we adapt algorithms within discrete optimization to obtain heuristic schemes for neural network architecture search, where we have resource constraints on the architecture. This simple scheme when applied on CIFAR-100 and ImageNet, identifies resource-constrained architectures with quantifiably better performance than current state-of-the-art models designed for mobile devices. Specifically, we find high-performing architectures with fewer parameters and computations by a search method that is much faster. |
| Tasks | Neural Architecture Search |
| Published | 2019-04-08 |
| URL | https://arxiv.org/abs/1904.03786v2 |
| https://arxiv.org/pdf/1904.03786v2.pdf | |
| PWC | https://paperswithcode.com/paper/resource-constrained-neural-network |
| Repo | |
| Framework | |
Source Dependency-Aware Transformer with Supervised Self-Attention
| Title | Source Dependency-Aware Transformer with Supervised Self-Attention |
| Authors | Chengyi Wang, Shuangzhi Wu, Shujie Liu |
| Abstract | Recently, Transformer has achieved the state-of-the-art performance on many machine translation tasks. However, without syntax knowledge explicitly considered in the encoder, incorrect context information that violates the syntax structure may be integrated into source hidden states, leading to erroneous translations. In this paper, we propose a novel method to incorporate source dependencies into the Transformer. Specifically, we adopt the source dependency tree and define two matrices to represent the dependency relations. Based on the matrices, two heads in the multi-head self-attention module are trained in a supervised manner and two extra cross entropy losses are introduced into the training objective function. Under this training objective, the model is trained to learn the source dependency relations directly. Without requiring pre-parsed input during inference, our model can generate better translations with the dependency-aware context information. Experiments on bi-directional Chinese-to-English, English-to-Japanese and English-to-German translation tasks show that our proposed method can significantly improve the Transformer baseline. |
| Tasks | Machine Translation |
| Published | 2019-09-05 |
| URL | https://arxiv.org/abs/1909.02273v1 |
| https://arxiv.org/pdf/1909.02273v1.pdf | |
| PWC | https://paperswithcode.com/paper/source-dependency-aware-transformer-with |
| Repo | |
| Framework | |
Accelerating CNN Training by Pruning Activation Gradients
| Title | Accelerating CNN Training by Pruning Activation Gradients |
| Authors | Xucheng Ye, Pengcheng Dai, Junyu Luo, Xin Guo, Yingjie Qi, Jianlei Yang, Yiran Chen, Weisheng Zhao |
| Abstract | Sparsification is an efficient approach to accelerate CNN inference, but it is challenging to take advantage of sparsity in training procedures because the involved gradients are usually dynamically changed. Actually, an important observation shows that most of the activation gradients in back-propagation are very close to zero and only have a tiny impact on weight-updating. Hence, we consider pruning these very small gradients randomly to accelerate CNN training according to the statistical distribution of activation gradients. Meanwhile, we theoretically analyze the impact of pruning algorithm on the convergence. The proposed approach is evaluated on AlexNet and ResNet-{18, 34, 50, 101, 152} with CIFAR-{10, 100} and ImageNet datasets. Experimental results show that our training approach could substantially achieve up to 5.92 $\times$ speedups at back-propagation stage with negligible accuracy loss. |
| Tasks | |
| Published | 2019-08-01 |
| URL | https://arxiv.org/abs/1908.00173v2 |
| https://arxiv.org/pdf/1908.00173v2.pdf | |
| PWC | https://paperswithcode.com/paper/accelerating-cnn-training-by-sparsifying |
| Repo | |
| Framework | |
Optimal Delivery with Budget Constraint in E-Commerce Advertising
| Title | Optimal Delivery with Budget Constraint in E-Commerce Advertising |
| Authors | Chao Wei, Weiru Zhang, Shengjie Sun, Fei Li, Xiaonan Meng, Yi Hu, Hao Wang |
| Abstract | Online advertising in E-commerce platforms provides sellers an opportunity to achieve potential audiences with different target goals. Ad serving systems (like display and search advertising systems) that assign ads to pages should satisfy objectives such as plenty of audience for branding advertisers, clicks or conversions for performance-based advertisers, at the same time try to maximize overall revenue of the platform. In this paper, we propose an approach based on linear programming subjects to constraints in order to optimize the revenue and improve different performance goals simultaneously. We have validated our algorithm by implementing an offline simulation system in Alibaba E-commerce platform and running the auctions from online requests which takes system performance, ranking and pricing schemas into account. We have also compared our algorithm with related work, and the results show that our algorithm can effectively improve campaign performance and revenue of the platform. |
| Tasks | |
| Published | 2019-09-29 |
| URL | https://arxiv.org/abs/1909.13221v2 |
| https://arxiv.org/pdf/1909.13221v2.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-delivery-with-budget-constraint-in-e |
| Repo | |
| Framework | |
Detecting Glaucoma Using 3D Convolutional Neural Network of Raw SD-OCT Optic Nerve Scans
| Title | Detecting Glaucoma Using 3D Convolutional Neural Network of Raw SD-OCT Optic Nerve Scans |
| Authors | Erfan Noury, Suria S. Mannil, Robert T. Chang, An Ran Ran, Carol Y. Cheung, Suman S. Thapa, Harsha L. Rao, Srilakshmi Dasari, Mohammed Riyazuddin, Sriharsha Nagaraj, Reza Zadeh |
| Abstract | We propose developing and validating a three-dimensional (3D) deep learning system using the entire unprocessed OCT optic nerve volumes to distinguish true glaucoma from normals in order to discover any additional imaging biomarkers within the cube through saliency mapping. The algorithm has been validated against 4 additional distinct datasets from different countries using multimodal test results to define glaucoma rather than just the OCT alone. 2076 OCT (Cirrus SD-OCT, Carl Zeiss Meditec, Dublin, CA) cube scans centered over the optic nerve, of 879 eyes (390 healthy and 489 glaucoma) from 487 patients, age 18-84 years, were exported from the Glaucoma Clinic Imaging Database at the Byers Eye Institute, Stanford University, from March 2010 to December 2017. A 3D deep neural network was trained and tested on this unique OCT optic nerve head dataset from Stanford. A total of 3620 scans (all obtained using the Cirrus SD-OCT device) from 1458 eyes obtained from 4 different institutions, from United States (943 scans), Hong Kong (1625 scans), India (672 scans), and Nepal (380 scans) were used for external evaluation. The 3D deep learning system achieved an area under the receiver operation characteristics curve (AUROC) of 0.8883 in the primary Stanford test set identifying true normal from true glaucoma. The system obtained AUROCs of 0.8571, 0.7695, 0.8706, and 0.7965 on OCT cubes from United States, Hong Kong, India, and Nepal, respectively. We also analyzed the performance of the model separately for each myopia severity level as defined by spherical equivalent and the model was able to achieve F1 scores of 0.9673, 0.9491, and 0.8528 on severe, moderate, and mild myopia cases, respectively. Saliency map visualizations highlighted a significant association between the optic nerve lamina cribrosa region in the glaucoma group. |
| Tasks | |
| Published | 2019-10-14 |
| URL | https://arxiv.org/abs/1910.06302v1 |
| https://arxiv.org/pdf/1910.06302v1.pdf | |
| PWC | https://paperswithcode.com/paper/detecting-glaucoma-using-3d-convolutional |
| Repo | |
| Framework | |
