Live video communications over bandwidth constrained ad-hoc radio networks necessitates high
compression rates. To this end, a model based video communication system that incorporates
flexible and accurate 3D modelling and reconstruction is proposed in part. Model-based video coding
(MBVC) is known to provide the highest compression rates, but usually compromises photorealism
and object detail. High compression ratios are achieved at the encoder by extracting and transmit-
ting only the parameters which describe changes to object orientation and motion within the scene.
The decoder uses the received parameters to animate reconstructed objects within the synthesised
scene. This is scene understanding rather than video compression. 3D reconstruction of objects
and scenes present at the encoder is the focus of this research.
3D Reconstruction is accomplished by utilizing the Patch-based Multi-view Stereo (PMVS) frame-
work of Yasutaka Furukawa and Jean Ponce. Surface geometry is initially represented as a sparse
set of orientated rectangular patches obtained from matching feature correspondences in the input
images. To increase reconstruction density these patches are iteratively expanded, and filtered
using visibility constraints to remove outliers. Depending on the availability of segmentation in-
formation, there are two methods for initialising a mesh model from the reconstructed patches.
The first method initialises the mesh from the object's visual hull. The second technique initialises
the mesh directly from the reconstructed patches. The resulting mesh is then refined by enforcing
patch reconstruction consistency and regularization constraints for each vertex on the mesh.
To improve robustness to outliers, two enhancements to the above framework are proposed. The
first uses photometric consistency during feature matching to increase the probability of selecting
the correct matching point first. The second approach estimates the orientation of the patch
such that its photometric discrepancy score for each of its visible images is minimised prior to
optimisation. The overall reconstruction algorithm is shown to be flexible and robust in that it can reconstruct
3D models for objects and scenes. It is able to automatically detect and discard outliers and may be
initialised by simple visual hulls. The demonstrated ability to account for surface orientation of the
patches during photometric consistency computations is a key performance criterion. Final results
show that the algorithm is capable of accurately reconstructing objects containing fine surface
details, deep concavities and regions without salient textures. / Thesis (M.Sc.Eng.)-University of KwaZulu-Natal, Durban, 2012.
| Identifer | oai:union.ndltd.org:netd.ac.za/oai:union.ndltd.org:ukzn/oai:http://researchspace.ukzn.ac.za:10413/8496 |
| Date | January 2012 |
| Creators | Ramchunder, Naren. |
| Contributors | Naidoo, Bashan. |
| Source Sets | South African National ETD Portal |
| Language | en_ZA |
| Detected Language | English |
| Type | Thesis |
Page generated in 0.006 seconds