This work focuses on the high frequency restoration of video sequences captured by a hybrid camera, using key-frames as high frequency samples. The proposed method outlines a hierarchy to the super-resolution process, and is aimed at maximizing both speed and performance. Additionally, an advanced image processing simulator (EngineX) was developed to fine tune the algorithm. / Super-resolution algorithms are designed to enhance the detail level of a
particular image or video sequence. However, it is very difficult to achieve in
practice due to the problem being ill-posed, and often requires regularization
based on assumptions about texture or edges. The process can be aided using
high-resolution key-frames such as those generated from a hybrid camera. A
hybrid camera is capable of capturing footage in multiple spatial and temporal
resolutions. The typical output consists of a high resolution stream captured at
low frame rate, and a low resolution stream captured at a high frame rate.
Key-frame based super-resolution algorithms exploit the spatial and temporal
correlation between the high resolution and low resolution streams to
reconstruct a high resolution and high frame rate output stream.
The proposed algorithm outlines a hierarchy to the super-resolution process,
combining several different classical and novel methods. A residue formulation
decides which pixels are required to be further reconstructed if a particular
hierarchy stage fails to provide the expected results when compared to the low
resolution prior. The hierarchy includes the optical flow based estimation which
warps high frequency information from adjacent key-frames to the current frame.
Specialized candidate pixel selection reduces the total number of pixels
considered in the NLM stage. Occlusion is handled by a final fallback stage in
the hierarchy. Additionally, the running time for a CIF sequence of 30 frames
has been significantly reduced to within 3 minutes by identifying which pixels
require reconstruction with a particular method.
A custom simulation environment implements the proposed method as well as many
common image processing algorithms. EngineX provides a graphical interface where
video sequences and image processing methods can be manipulated and combined.
The framework allows for advanced features such as multithreading, parameter
sweeping, and block level abstraction which aided the development of the
proposed super-resolution algorithm. Both speed and performance were fine tuned
using the simulator which is the key to its improved quality over other traditional
super-resolution schemes. / Thesis / Master of Applied Science (MASc)
| Identifer | oai:union.ndltd.org:mcmaster.ca/oai:macsphere.mcmaster.ca:11375/18248 |
| Date | 11 1900 |
| Creators | Lengyel, Robert |
| Contributors | Shirani, Shahram, Electrical and Computer Engineering |
| Source Sets | McMaster University |
| Language | English |
| Detected Language | English |
| Type | Thesis |
Page generated in 0.002 seconds