Real-time video communication over wireless networks is a challenging problem because wireless channels suffer from fading, additive noise and interference, which translate into packet loss and delay. Since modern video encoders deliver video packets with decoding dependencies, packet loss and delay can significantly degrade the video quality at the receiver. Many error resilience mechanisms have been proposed to combat packet loss in wireless networks, but only a few were specifically designed for packet switched video telephony over Third Generation (3G) networks. The first part of the thesis presents an error resilience technique for packet switched video telephony that combines application layer Forward Error Correction (FEC) with rateless codes, Reference Picture Selection (RPS) and cross layer optimization. Rateless codes have lower encoding and decoding computational complexity compared to traditional error correcting codes. One can use them on complexity constrained hand-held devices. Also, their redundancy does not need to be fixed in advance and any number of encoded symbols can be generated on the fly. Reference picture selection is used to limit the effect of spatio-temporal error propagation. Limiting the effect of spatio-temporal error propagation results in better video quality. Cross layer optimization is used to minimize the data loss at the application layer when data is lost at the data link layer. Experimental results on a High Speed Packet Access (HSPA) network simulator for H.264 compressed standard video sequences show that the proposed technique achieves significant Peak Signal to Noise Ratio (PSNR) and Percentage Degraded Video Duration (PDVD) improvements over a state of the art error resilience technique known as Interactive Error Control (IEC), which is a combination of Error Tracking and feedback based Reference Picture Selection. The improvement is obtained at a cost of higher end-to-end delay. The proposed technique is improved by making the FEC (Rateless code) redundancy channel adaptive. Automatic Repeat Request (ARQ) is used to adjust the redundancy of the Rateless codes according to the channel conditions. Experimental results show that the channel adaptive scheme achieves significant PSNR and PDVD improvements over the static scheme for a simulated Long Term Evolution (LTE) network. In the third part of the thesis, the performance of the previous two schemes is improved by making the transmitter predict when rateless decoding will fail. In this case, reference picture selection is invoked early and transmission of encoded symbols for that source block is aborted. Simulations for an LTE network show that this results in video quality improvement and bandwidth savings. In the last part of the thesis, the performance of the adaptive technique is improved by exploiting the history of the wireless channel. In a Rayleigh fading wireless channel, the RLC-PDU losses are correlated under certain conditions. This correlation is exploited to adjust the redundancy of the Rateless code and results in higher Rateless code decoding success rate and higher video quality. Simulations for an LTE network show that the improvement was significant when the packet loss rate in the two wireless links was 10%. To facilitate the implementation of the proposed error resilience techniques in practical scenarios, RTP/UDP/IP level packetization schemes are also proposed for each error resilience technique. Compared to existing work, the proposed error resilience techniques provide better video quality. Also, more emphasis is given to implementation issues in 3G networks.

The demand for applications like transmission and sharing of video is ever-increasing. Although network resources (bandwidth in particular) and coverage, networking technologies, compression ratio of state-of-the-art video coders have improved, unreliability of the transmission medium prevents us from gaining the most benefit from these applications. This thesis introduces a video coder that is resilient to network failures for transmission applications by using the framework of multiple description coding (MDC). Unlike traditional video coding which compresses the video into single bitstream, in MDC the video is compressed into more than one bitstream which can be independently decoded. It not only averages out the effect of network errors over the bitstreams but it also makes it possible to utilize the multipath nature of most network topologies. An end-to-end rate-distortion optimization is proposed for the codec to make sure that the codec exhibits improved compression performance and that the descriptions are equally efficient to improve the final video quality. An optimized strategy for packetizing the compressed bitstreams of the descriptions is also proposed which guarantees that each packet is self-contained and efficient. The evaluation of the developed MD codec over simulated unreliable packet networks shows that it is possible to achieve improved resilience with the proposed strategies and the end video quality is significantly improved as a result. This is further verified with subjective evaluation over a range of different types of video test sequences.

Distributed three-dimensional (3D) graphics applications exhibit both resemblance and uniqueness in comparison with conventional streaming media applications. The resemblance relates to the large data volume and the bandwidth-limited and error-prone transmission channel. The uniqueness is due to the polygon-based representation of 3D geometric meshes and their accompanying attributes such as textures. This specific data format introduces sophisticated rendering computation to display graphics models and therefore places an additional constraint on the streaming application. The objective of this research is to provide scalable, error-resilient, and time-efficient solutions for high-quality 3D graphics applications in distributed and resource-constrained environments. Resource constraints range from rate-limited and error-prone channels to insufficient data-reception, computing, and display capabilities of client devices. Optimal resource treatment with transmission and rendering scalability is important under such circumstances. The proposed research consists of three milestones. In the first milestone, we develop a joint mesh and texture optimization framework for scalable transmission and rendering of textured 3D models. Then, we address network behaviors and develop a hybrid retransmission and error protection mechanism for the on-demand delivery of 3D models. Next, we advance from individual 3D models to 3D scene databases, which contain numerous objects interacting in one geometric space, and study joint application and transport approaches. By properly addressing the properties of 3D scenes represented in multi-resolution hierarchies, we develop a joint source and channel coding method and a multi-streaming framework for streaming the content-rich 3D scene databases toward optimized transmission and rendering scalability under resource constraints.

In this thesis, a number of novel techniques for error resilient coding and streaming for multiview video are presented. First of all, a novel coding technique for stereoscopic video is proposed where additional coding gain is achieved by downsampling one of the views spatially or temporally based on the well-known theory that the human visual system can perceive high frequencies in 3D from the higher quality view. Stereoscopic videos can be coded at a rate upto 1.2 times that of monoscopic videos with little visual quality degradation with the proposed coding technique. Next, a systematic method for design and optimization of multi-threaded multi-view video encoding/decoding algorithms using multi-core processors is proposed. The proposed multi-core decoding architectures are compliant with the current international standards, and enable multi-threaded processing with negligible loss of encoding efficiency and minimum processing overhead. End-to-end 3D Streaming system over Internet using current standards is implemented. A heuristic methodology for modeling the end-toend rate-distortion characteristic of this system is suggested and the parameters of the system is optimally selected using this model. End-to-end 3D Broadcasting system over DVB-H using current standards is also implemented. Extensive testing is employed to show the importance and characteristics of several error resilient tools. Finally we modeled end-to-end RD characteristics to optimize the encoding and protection parameters.

To support video communication over mobile environments has been one of the objectives of many engineers of telecommunication networks and it has become a basic requirement of a third generation of mobile communication systems. This dissertation explores the possibility of optimizing the utilization of shared scarce radio channels for live video transmission over a GSM (Global System for Mobile telecommunications) network and realizing error resilient video communication in unfavorable channel conditions, especially in mobile radio channels. The main contribution describes the adoption of a SEC (Second Error Correction) approach using ECC (Error Correction Coding) based on a Punctured Convolutional Coding scheme, to cope with residual errors at the application layer and enhance the error resilience of a compressed video bitstream. The approach is developed further for improved performance in different circumstances, with some additional enhancements involving Intra Frame Relay and Interleaving, and the combination of the approach with Packetization. Simulation results of applying the various techniques to test video sequences Akiyo and Salesman are presented and analyzed for performance comparisons with conventional video coding standard. The proposed approach shows consistent improvements under these conditions. For instance, to cope with random residual errors, the simulation results show that when the residual BER (Bit Error Rate) reaches 10-4, the video output reconstructed from a video bitstream protected using the standard resynchronization approach is of unacceptable quality, while the proposed scheme can deliver a video output which is absolutely error free in a more efficient way. When the residual BER reaches 10-3, the standard approach fails to deliver a recognizable video output, while the SEC scheme can still correct all the residual errors with modest bit rate increase. In bursty residual error conditions, the proposed scheme also outperforms the resynchronization approach. Future works to extend the scope and applicability of the research are suggested in the last chapter of the thesis.