Fast-SRGAN:一种快速深度学习模型,可将低分辨率视频以30fps的速率上采样到高分辨率

  • A4_362645
    了解作者
  • 620.2KB
    文件大小
  • zip
    文件格式
  • 0
    收藏次数
  • VIP专享
    资源类型
  • 0
    下载次数
  • 2022-04-20 06:38
    上传日期
快速SRGAN 该存储库的目标是实现实时超分辨率,以对低分辨率视频进行升采样。 目前,该设计遵循架构。 但是代替残差块,采用反向残差块以提高参数效率和快速操作。 这种想法在某种程度上受到。 培训设置如下图所示: 速度基准 通过平均800帧以上的运行时间获得以下运行时间/ fps。 在GTX 1080上测得。 输入图像尺寸 输出尺寸 时间(秒) 第一人称射击 128x128 512x512 0.019 52 256x256 1024x1024 0.034 30 384x384 1536x1536 0.068 15 我们看到有可能以30fps的速度将其上采样到720
Fast-SRGAN-master.zip
  • Fast-SRGAN-master
  • models
  • generator.h5
    850.9KB
  • dataloader.py
    4.4KB
  • main.py
    6.3KB
  • model.py
    9.9KB
  • infer.py
    1.4KB
  • requirements.txt
    55B
  • LICENSE
    1KB
  • README.md
    4.1KB
内容介绍
# Fast-SRGAN The goal of this repository is to enable real time super resolution for upsampling low resolution videos. Currently, the design follows the [SR-GAN](https://arxiv.org/pdf/1609.04802.pdf) architecture. But instead of residual blocks, inverted residual blocks are employed for parameter efficiency and fast operation. This idea is somewhat inspired by [Real time image enhancement GANs](http://www.micc.unifi.it/seidenari/wp-content/papercite-data/pdf/caip_2019.pdf). The training setup looks like the following diagram: <p align="center"> <img src="https://user-images.githubusercontent.com/4294680/67164120-22157480-f377-11e9-87c1-5b6acace0e47.png"> </p> # Speed Benchmarks The following runtimes/fps are obtained by averaging runtimes over 800 frames. Measured on a GTX 1080. | Input Image Size | Output Size | Time (s) | FPS | | ------------- |:-------------:|:---------:|:---:| | 128x128 | 512x512 | 0.019 | 52 | | 256x256 | 1024x1024 | 0.034 | 30 | | 384x384 | 1536x1536 | 0.068 | 15 | We see it's possible to upsample to 720p at around 30fps. # Requirements This was tested on Python 3.7. To install the required packages, use the provided requirements.txt file like so: ```bash pip install -r requirements.txt ``` # Pre-trained Model A pretrained generator model on the DIV2k dataset is provided in the 'models' directory. It uses 6 inverted residual blocks, with 32 filters in every layer of the generator. Upsampling is done via phase shifts in the low resolution space for speed. To try out the provided pretrained model on your own images, run the following: ```bash python infer.py --image_dir 'path/to/your/image/directory' --output_dir 'path/to/save/super/resolution/images' ``` # Training To train, simply execute the following command in your terminal: ```bash python main.py --image_dir 'path/to/image/directory' --hr_size 384 --lr 1e-4 --save_iter 200 --epochs 10 --batch_size 14 ``` Model checkpoints and training summaries are saved in tensorboard. To monitor training progress, open up tensorboard by pointing it to the 'logs' directory that will created when you start training. # Samples Following are some results from the provided trained model. Left shows the low res image, after 4x bicubic upsampling. Middle is the output of the model. Right is the actual high resolution image. <p align="center"> <b>384x384 to 1536x1536 Upsampling</b> <img src="https://user-images.githubusercontent.com/4294680/67642055-4f7a9900-f908-11e9-93d7-5efc902bd81c.png"> <b>256x256 to 1024x1024 Upsampling</b> <img src="https://user-images.githubusercontent.com/4294680/67642086-8fda1700-f908-11e9-8428-8a69ea86dedb.png"> <b>128x128 to 512x512 Upsampling</b> <img src="https://user-images.githubusercontent.com/4294680/67641979-5ead1700-f907-11e9-866c-b72d2e1dec8a.png"> </p> # Extreme Super Resolution Upsampling HQ images 4x as a check to see the image is not destroyed (since the network is trained on low quality, it should also upsample high quality images while preserving their quality). <p align="center"> <img src="https://user-images.githubusercontent.com/4294680/67641915-b434f400-f906-11e9-88d1-44a7f2a80923.png"> <img src="https://user-images.githubusercontent.com/4294680/67641917-b8611180-f906-11e9-8539-81f17d69653f.png"> <img src="https://user-images.githubusercontent.com/4294680/67641946-fbbb8000-f906-11e9-95af-2873eb01e2ec.png"> </p> # Changing Input Size The provided model was trained on 384x384 inputs, but to run it on inputs of arbitrary size, you'll have to change the input shape like so: ```python from tensorflow import keras # Load the model model = keras.models.load_model('models/generator.h5') # Define arbitrary spatial dims, and 3 channels. inputs = keras.Input((None, None, 3)) # Trace out the graph using the input: outputs = model(inputs) # Override the model: model = keras.models.Model(inputs, outputs) # Now you are free to predict on images of any size. ``` # Contributing If you have ideas on improving model performance, adding metrics, or any other changes, please make a pull request or open an issue. I'd be happy to accept any contributions.
评论
    相关推荐