東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

High-Fidelity Image and Video Editin...

Qi, Chenyang,

Linked to FindBook

Google Book

Amazon

博客來

High-Fidelity Image and Video Editing with Generative Models /

Record Type:	Electronic resources : Monograph/item
Title/Author:	High-Fidelity Image and Video Editing with Generative Models // Chenyang Qi.
Author:	Qi, Chenyang,
Description:	1 electronic resource (107 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-06, Section: A.
Contained By:	Dissertations Abstracts International86-06A.
Subject:	Language. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31810275
ISBN:	9798346822295

High-Fidelity Image and Video Editing with Generative Models /
Qi, Chenyang,

High-Fidelity Image and Video Editing with Generative Models /Chenyang Qi. - 1 electronic resource (107 pages)

Source: Dissertations Abstracts International, Volume: 86-06, Section: A.

Recent years have witnessed a growing demand for visual content, such as 2D images and multi-frame videos, in fields like computational photography, virtual reality, gaming, and the film industry. In response to those demands, various generative models, including VQVAE, GAN, and Diffusion models, have been proposed to facilitate visual content generation from noise or text. However, it remains an open challenge to adopt these models for more practical image-to-image generation, also known as image processing and editing. This thesis explores the paradigm of image editing with generative models, with a focus on the foundational models from large-scale pretraining.We begin this thesis by exploring real-time image rescaling. Images from modern cameras can reach 6K resolution, but they also take up too much storage space. Here, we propose a quantized auto-encoder to compress a large 6K image to a JPEG thumbnail and reduce the file size by optimizing entropy loss. Then, an efficient decoder can upscale the low-resolution thumbnail back to a high-resolution image in real-time.Then, we move on to text-driven image restoration. Camera motion, digital circuit noise, and bad weather (e.g., rain and fog) can degrade photographers' images. We propose to restore images using a diffusion model guided by semantic and restoration instructions. To enable such multi-modal application at a lower training cost, we fine-tune an adaptor for the pre-trained latent diffusion model using synthetic degradation images.Finally, we discuss the text-driven video editing method. In addition to image-level computational photography, creative special effects are also widely used in games, movies, and short video applications. These effects typically require temporal motion consistency and semantic-level editing, such as identity and style. Since there is no robust, open-sourced, video generative model available, we focus on exploiting the text-to-image latent diffusion model in a zero-shot way. Specifically, we transform the image generative model to a video model, and extract the spatial-temporal attention maps in diffusion Unet during DDIM inversion as motion and geometry representation. Then, we refuse these attention maps during DDIM denoising following the target prompt. Our succinct framework allows shape, attribute, and global style while maintaining impressive temporal consistency.

English

ISBN: 9798346822295Subjects--Topical Terms:

643551
Language.

High-Fidelity Image and Video Editing with Generative Models /
LDR:03579nmm a22003493i 4500 001 2432871
005 20250807123510.5
006 m o d
007 cr|nu||||||||
008 261011s2024 miu||||||m |||||||eng d
020 $a 9798346822295
035 $a (MiAaPQD)AAI31810275
035 $a AAI31810275
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Qi, Chenyang, $e author. $3 3822265
245 1 0 $a High-Fidelity Image and Video Editing with Generative Models / $c Chenyang Qi.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 1 electronic resource (107 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-06, Section: A.
500 $a Advisors: Chen, Qifeng.
502 $b Ph.D. $c Hong Kong University of Science and Technology (Hong Kong) $d 2024.
520 $a Recent years have witnessed a growing demand for visual content, such as 2D images and multi-frame videos, in fields like computational photography, virtual reality, gaming, and the film industry. In response to those demands, various generative models, including VQVAE, GAN, and Diffusion models, have been proposed to facilitate visual content generation from noise or text. However, it remains an open challenge to adopt these models for more practical image-to-image generation, also known as image processing and editing. This thesis explores the paradigm of image editing with generative models, with a focus on the foundational models from large-scale pretraining.We begin this thesis by exploring real-time image rescaling. Images from modern cameras can reach 6K resolution, but they also take up too much storage space. Here, we propose a quantized auto-encoder to compress a large 6K image to a JPEG thumbnail and reduce the file size by optimizing entropy loss. Then, an efficient decoder can upscale the low-resolution thumbnail back to a high-resolution image in real-time.Then, we move on to text-driven image restoration. Camera motion, digital circuit noise, and bad weather (e.g., rain and fog) can degrade photographers' images. We propose to restore images using a diffusion model guided by semantic and restoration instructions. To enable such multi-modal application at a lower training cost, we fine-tune an adaptor for the pre-trained latent diffusion model using synthetic degradation images.Finally, we discuss the text-driven video editing method. In addition to image-level computational photography, creative special effects are also widely used in games, movies, and short video applications. These effects typically require temporal motion consistency and semantic-level editing, such as identity and style. Since there is no robust, open-sourced, video generative model available, we focus on exploiting the text-to-image latent diffusion model in a zero-shot way. Specifically, we transform the image generative model to a video model, and extract the spatial-temporal attention maps in diffusion Unet during DDIM inversion as motion and geometry representation. Then, we refuse these attention maps during DDIM denoising following the target prompt. Our succinct framework allows shape, attribute, and global style while maintaining impressive temporal consistency.
546 $a English
590 $a School code: 1223
650 4 $a Language. $3 643551
650 4 $a Internet. $3 527226
650 4 $a User experience. $3 3543994
650 4 $a Video recordings. $3 575241
650 4 $a Photography. $3 518339
650 4 $a Real time. $3 3562675
650 4 $a Neural networks. $3 677449
650 4 $a Natural language processing. $3 1073412
650 4 $a Cloud computing. $3 1016782
650 4 $a Entropy. $3 546219
650 4 $a Semantics. $3 520060
690 $a 0679
710 2 $a Hong Kong University of Science and Technology (Hong Kong). $e degree granting institution. $3 3814653
720 1 $a Chen, Qifeng $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 86-06A.
790 $a 1223
791 $a Ph.D.
792 $a 2024
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31810275