| 155 | 1 | 72 |
| 下载次数 | 被引频次 | 阅读次数 |
在图像超分辨率任务中通过加大网络参数量和计算复杂度可以提升性能，但该方法不适用于很多计算能力受限的应用场景。因此当超分图像较大时，轻量化超分网络的设计是个至关重要的方向。针对该问题的一个经典加速策略是分治法(将大问题拆解成小问题逐一击破),现有方法通常将大图像的超分问题分解成不同子图像块的超分问题，并根据每个子图像块的超分难易程度，使用不同计算量规模的网络分别进行超分处理，从而减少了冗余的计算。然而，该分治策略仅将子问题分解到了子图像块级别，并未达到满意的加速效果。据此，将分治策略中的子问题分解进一步深入到了像素级，根据不同像素的超分难易程度采用不同计算量的网络来分而治之。具体来说，引入一个不确定度估计的思想在训练中自适应预测每个像素所在位置的超分难易程度；提出了一个自适应像素特征精炼模块，根据不同像素的超分难易程度，对超分困难的像素点进行采样和特征修复，从而为困难点的超分问题分配了更多的计算量进行处理。大量实验表明，相比现有的子图像块级的超分网络加速方法，提出的像素级超分网络加速方法更为高效，在相同精度的情况下，有效减少了计算复杂度。
Abstract：In image super-resolution tasks, performance can be improved by increasing the amount of network parameters and computational complexity, but this approach is not applicable in many applications where computational power is limited.Therefore, when the super-resolution image is large, the design of lightweight super-resolution network is a crucial direction.A classic acceleration strategy for this problem is the divide-and-conquer approach(breaking down big problems into small ones one by one).The existing methods usually decompose the super-resolution problem of large images into the super-resolution problem of different patches, and process each patches separately using networks of different scales according to the difficulty of super-resolution of each patches, thus reducing redundant calculations.However, this divide-and-conquer strategy only decomposes the sub-problem to the patch level, and does not achieve satisfied acceleration effect.Therefore, the sub-problem decomposition in the divide-and-conquer strategy is further deepened to the pixel level, and different computational networks are used to perform divide-and-conquer strategy according to the super-resolution difficulty of different pixels.To be specific, firstly, an uncertainty estimation method is introduced to adaptively predict the super-resolution difficulty of each pixel in the training.Secondly, an adaptive pixel feature refining module is proposed to perform sampling and feature restoration to the pixels with difficulty in super-resolution according to the super-resolution difficulty of different pixels, therefore the super-resolution of the difficult pixels is allocated more computation to process.A large number of experiments show that, compared with the existing patch-level super-resolution network acceleration methods, the proposed pixel-level super-resolution network acceleration method is more efficient and reduces the computational complexity with the same accuracy.
[1] ZOU Z X,CHEN K Y,SHI Z W,et al.Object Detection in 20 Years:A Survey[J/OL].(2019-05-13)[2022-06-23].https://arxiv.org/abs/1905.05055.
[2] YU S Y,XIAO J M,ZHANG B F,et al.Democracy Does Matter:Comprehensive Feature Mining for Co-salient Object Detection[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).New Orleans:IEEE,2022:969-978.
[3] XU X,WANG Y,ZHENG Y,et al.Back to Reality:Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement[J/OL].(2022-05-10)[2022-06-23].https://arxiv.org/abs/2203.05238.
[4] ANDRILUKA M,IQBAL U,INSAFUTDINOV E,et al.PoseTrack:A Benchmark for Human Pose Estimation and Tracking[J/OL].(2017-10-27)[2022-06-23].https://arxiv.org/abs/1710.10000.
[5] CHO N G,YUILLE A L,LEE S W.Adaptive Occlusion State Estimation for Human Pose Tracking Under Self-occlusions[J].Pattern Recognition,2013,46(3):649-661.
[6] HE Y,YANG D,ROTH H,et al.DiNTS:Differentiable Neural Network Topology Search for 3D Medical Image Segmentation[J/OL].(2021-05-29)[2022-06-23].https://arxiv.org/abs/2103.15954.
[7] LUZ E,SILVA P,SILVA R,et al.Towards an Effective and Efficient Deep Learning Model for COVID-19 Patterns Detection in X-ray Images[J].Research on Biomedical Engineering,2022,38:149-162.
[8] HE D,ZHENG Y,SUN B,et al.Checkerboard Context Model for Efficient Learned Image Compression[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Nashville:IEEE,2021:14766-14775.
[9] ZHANG X,WU X L.Attention-guided Image Compression by Deep Reconstruction of Compressive Sensed Saliency Skeleton[J/OL].(2021-03-29)[2022-06-23].https://arxiv.org/abs/2103.15368.
[10] DONG C,LOY C C,HE K,et al.Image Super-resolution Using Deep Convolutional Networks[J/OL].(2015-07-31)[2022-06-23].https://arxiv.org/abs/1501.00092.
[11] LEDIG C,THEIS L,HUSZáR F,et al.Photo-realistic Single Image Super-resolution Using a Generative Adversarial Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu:IEEE,2016:105-114.
[12] HE X,MO Z,WANG P,et al.ODE-inspired Network Design for Single Image Super-resolution[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach:IEEE,2019:1732-1741.
[13] HUI Z,GAO X,YANG Y,et al.Lightweight Image Super-resolution with Information Multi-distillation Network[J/OL].(2019-09-26)[2022-06-23].https://arxiv.org/abs/1909.11856.
[14] CIPOLLA R,GAL Y,KENDALL A.Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Salt Lake City:IEEE,2018:7482-7491.
[15] CAI J,ZENG H,YONG H,et al.Toward Real-world Single Image Super-resolution:A New Benchmark and a New Model[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).Seoul:IEEE,2020:3086-3095.
[16] POGGI M,ALEOTTI F,TOSI F,et al.On the Uncertainty of Self-supervised Monocular Depth Estimation[J/OL].(2020-05-13)[2022-06-23].https://arxiv.org/abs/2005.06209v1.
[17] 周登文，李文斌，李金新，等.一种轻量级的多尺度通道注意图像超分辨率重建网络[J].电子学报，2022,50(10):2336-2346.
[18] HARDIANSYAH B,LU Y.Single Image Super-resolution via Multiple Linear Mapping Anchored Neighborhood Regression[J].Multimedia Tools and Applications,2021,80:28713-28730.
[19] CHOI J S,KIM M.Single Image Super-resolution Using Global Regression Based on Multiple Local Linear Mappings[J].IEEE Transactions on Image Process,2017,26(3):1300-1314.
[20] 刘磊，张燕，张佳芬.基于邻域嵌入的X光图像超分辨率区域融合[J].激光杂志，2021,42(8):52-56.
[21] LIU W,LI S.Multi-morphology Image Super-resolution via Sparse Representation[J].Neurocomputing,2013,120:645-654.
[22] 葛鹏，游耀堂.基于稀疏表示的光场图像超分辨率重建[J].激光与光电子学进展，2022,59(2):86-92.
[23] 杨雪，李峰，鹿明，等.混合稀疏表示模型的超分辨率重建[J].遥感学报，2022,26(8):1685-1697.
[24] LIU N,XU X,LI Y,et al.Sparse Representation Based Image Super-resolution on the KNN Based Dictionaries[J].Optics & Laser Technology,2019,110:135-144.
[25] POLAT K,?AHAN S,GüNE? S.Automatic Detection of Heart Disease Using an Artificial Immune Recognition System (AIRS) with Fuzzy Resource Allocation Mechanism and k-nn (Nearest Neighbour) Based Weighting Preprocessing[J].Expert Systems with Applications,2007,32(2):625-631.
[26] 刘娜，李翠华.基于多层卷积神经网络学习的单帧图像超分辨率重建方法[J].中国科技论文，2015,10(2):201-206.
[27] LAI W S,HUANG J B,AHUJA N,et al.Deep Laplacian Pyramid Networks for Fast and Accurate Super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu:IEEE,2017:5835-5843.
[28] AHN N,KANG B,SOHN K A.Fast,Accurate,and Lightweight Super-resolution with Cascading Residual Network[J/OL].(2018-10-04)[2022-06-23].https://arxiv.org/abs/1803.08664v2.
[29] ZHENG H,WANG X,GAO X.Fast and Accurate Single Image Super-resolution via Information Distillation Network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:723-731.
[30] KONG X,ZHAO H,QIAO Y,et al.ClassSR:A General Framework to Accelerate Super-resolution Networks by Data Characteristic[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Nashville:IEEE,2021:12011-12020.
[31] KIRILLOV A,WU Y,HE K,et al.PointRend:Image Segmentation as Rendering[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle:IEEE,2020:9796-9805.
[32] TIMOFTE R,AGUSTSSON E,VAN COOL L,et al.NTIRE 2017 Challenge on Single Image Super-resolution:Methods and Results[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Honolulu:IEEE,2017:114-125.
[33] GU S,LUGMAYR A,DANELLJAN M,et al.DIV8K:DIVerse 8K Resolution Image Dataset[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).Seoul:IEEE,2019:3512-3516.
[34] ZHANG K,GU S,TIMOFTE R,et al.AIM 2019 Challenge on Constrained Super-resolution:Methods and Results[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).Seoul:IEEE,2019:3565-3574.
[35] DONG C,LOY C C,TANG X.Accelerating the Super-resolution Convolutional Neural Network[J/OL].(2016-08-01)[2022-06-23].https://arxiv.org/abs/1608.00367.
基本信息:
DOI：
中图分类号:TP391.41
引用信息:
[1]刘智轩,陆善贵,蓝如师.基于像素级分治策略的超分网络加速方法[J].无线电工程,2023,53(03):508-518.
基金信息:
广西科技计划项目(2019GXNSFFA245014,ZY20198016); 国家自然科学基金(62172120,61936002); 广西图像图形与智能处理重点实验室开发课题(GIIP2001)~~