Real ESRGAN

PyTorch implementation of a Real-ESRGAN model trained on custom dataset. This model shows better results on faces compared to the original version. It is also easier to integrate this model into your projects.

Real-ESRGAN is an upgraded ESRGAN trained with pure synthetic data is capable of enhancing details while removing annoying artifacts for common real-world images.

You can try it in Google Colab

Paper: Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

Official GitHub: https://github.com/xinntao/Real-ESRGAN

Installation

Clone repo

git clone https://github.com/sberbank-ai/Real-ESRGAN
cd Real-ESRGAN

Install requirements

pip install -r requirements.txt

Download pretrained weights and put them into weights/ folder

Usage

import torch
from PIL import Image
import numpy as np
from RealESRGAN import RealESRGAN

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model = RealESRGAN(device, scale=4)
model.load_weights('weights/RealESRGAN_x4.pth')

path_to_image = 'inputs/lr_image.png'
image = Image.open(path_to_image).convert('RGB')

sr_image = model.predict(image)

sr_image.save('results/sr_image.png')

Model Weights

This repository provides weights of the models from the benchmarking study conducted in "https://arxiv.org/abs/2006.13365">"Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework" which have been upgraded to compatible with PyKEEN 1.9.

The weights are organized as zipfiles, which are named by the dataset-interaction function configuration. For each of these combinations, we chose the best according to validation Hits@10 to include into this repository. For each model, we have three files:

1. configuration.json contains the (pipeline) configuration used to train the model. It can loaded as

import pathlib
import json
configuration = json.loads(pathlib.Path("configuration.json").read_text())

Since the configuration is intended for the pipeline, we need some custom code to re-create the model without re-training it.

from pykeen.datasets import get_dataset
from pykeen.models import ERModel, model_resolver

configuration = configuration["pipeline"]
# load the triples factory
dataset = get_dataset(
  dataset=configuration["dataset"], dataset_kwargs=configuration.get("dataset_kwargs", None)
)
model: ERModel = model_resolver.make(
  configuration["model"], configuration["model_kwargs"], triples_factory=dataset.training
)

Note, that this only creates the model instance, but does not load the weights, yet.

1. state_dict.pt contains the weights, stored via torch.save. They can be loaded via

import torch
state_dict = torch.load("state_dict.pt")

We can load these weights into the model by using Module.load_state_dict

model.load_state_dict(state_dict, strict=False)

Note that we set strict=False, since the exported weights do not contain regularizers' state, while the re-instantiated models may have regularizers.

1. results.json contains the results obtained by the original runs. It can be read by

import pathlib
import json
configuration = json.loads(pathlib.Path("results.json").read_text())

Note that some of the recently added metrics are not available in those results.

Datasets to NeurIPS 2021 accepted paper "Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction".

Datasets are pytorch files containing a dictionary with training, validation and test sets. Train, validation and test sets are custom dataset classes which inherit from the standard torch dataset class. Corresponding code an be found at https://github.com/HSG-AIML/NeurIPS_2021-Weight_Space_Learning.

Datasets 41, 42, 43 and 44 are our dataset format wrapped around the zoos from Unterthiner et al, 2020 (https://github.com/google-research/google-research/tree/master/dnn_predict_accuracy)

Abstract: Self-Supervised Learning (SSL) has been shown to learn useful and information-preserving representations. Neural Networks (NNs) are widely applied, yet their weight space is still not fully understood. Therefore, we propose to use SSL to learn neural representations of the weights of populations of NNs. To that end, we introduce domain specific data augmentations and an adapted attention architecture. Our empirical evaluation demonstrates that self-supervised representation learning in this domain is able to recover diverse NN model characteristics. Further, we show that the proposed learned representations outperform prior work for predicting hyper-parameters, test accuracy, and generalization gap as well as transfer to out-of-distribution settings.

This is a synthetic dataset built to benchmark the capabilities of explainable AI methods against difficult tasks. The dataset has two classes, which depend on whether there is an odd or an even number of triangles in the image. Also attached are the PyTorch model weights of a classifier trained on the task, both the raw weights and a JIT-compiled version. The raw weights can be loaded with the Funke Lab torch package.