From a85ab454efab2455b2f64098362ae2a5f26ccd56 Mon Sep 17 00:00:00 2001 From: DannHuang Date: Thu, 24 Oct 2024 11:05:52 +0800 Subject: [PATCH] project abstract --- docs/index.html | 94 +++++++++++++++++++++++++++++-------------------- 1 file changed, 56 insertions(+), 38 deletions(-) diff --git a/docs/index.html b/docs/index.html index f8c9042..ee909ad 100644 --- a/docs/index.html +++ b/docs/index.html @@ -181,7 +181,7 @@

InstantIR: Blind Image Restoration with
-
+

InstantIR recovers extrem quality visual details from your image.

@@ -197,13 +197,20 @@

Abstract

- Handling test-time unknown degradation is the major challenge in Blind Image Restoration (BIR), necessitating high model generalization. An effective strategy is to incorporate prior knowledge, either from human input or generative model. + Handling test-time unknown degradation is the major challenge in Blind Image Restoration (BIR), necessitating high + model generalization. An effective strategy is to incorporate prior knowledge, either from human input or generative model.

- We introduce Instant-reference Image Restoration (InstantIR), a novel diffusion-based BIR method which dynamically adjusts generation condition during inference. InstantIR first extracts a compact representation of the input via a pre-trained vision encoder. At each generation step, this representation is used to decode current diffusion latent and instantiate a generative prior. The degraded input is then encoded with this reference, providing robust generation condition. + We introduce Instant-reference Image Restoration (InstantIR), a novel diffusion-based BIR method which dynamically adjusts + generation condition during inference. InstantIR first extracts a compact representation of the input via a pre-trained + vision encoder. At each generation step, this representation is used to decode current diffusion latent and instantiate + a generative prior. The degraded input is then encoded with this reference, providing robust generation condition.

- We observe the variance of generative references fluctuate with degradation intensity, which we further leverage as an indicator for developing a sampling algorithm adaptive to input quality. Extensive experiments demonstrate InstantIR achieves state-of-the-art performance and offering outstanding visual quality. Through modulating generative references with textual description, InstantIR can restore extreme degradation and additionally feature creative restoration. + We observe the variance of generative references fluctuate with degradation intensity, which we further leverage as an indicator + for developing a sampling algorithm adaptive to input quality. Extensive experiments demonstrate InstantIR achieves + state-of-the-art performance and offering outstanding visual quality. Through modulating generative references with + textual description, InstantIR can restore extreme degradation and additionally feature creative restoration.

@@ -216,18 +223,56 @@

Abstract

+ +
+
+

Pipelines

+ + +

Qualitative comparison

+
+

+ Through leveraging the generative prior, InstantIR. +

+
+
+
+ Interpolate start reference image. +

Start Frame

+
+
+
+ Loading... +
+ +
+
+ Interpolation end reference image. +

End Frame

+
+
+
+ +
+
+ +
-

Visual Examples

+

Restoration Examples

-

Interpolating states

+

Qualitative comparison

- We can also animate the scene by interpolating the deformation latent codes of two input - frames. Use the slider here to linearly interpolate between the left frame and the right - frame. + Through leveraging the generative prior, InstantIR.

@@ -263,7 +308,7 @@

Interpolating states

-

Text-guided Details Enhancement

+

Text-guided Details Enhancement

Using nerfies you can create fun visual effects. This Dolly zoom effect would be impossible without nerfies since it would require going through a wall. @@ -278,7 +323,7 @@

Text-guided Details Enhancement

-

Text-guided Restoration-Editing

+

Text-guided Restoration-Editing

@@ -296,33 +341,6 @@

Text-guided Restoration-Editing

- - - -