High quality videos would not be possible on contemporary devices if not for codecs. High quality videos on the Internet are possible due to streaming architectures. We take a closer look at both these technologies.
In the previous article I gave a broad overview of video streaming and described the growth forecasts for the coming years. The importance of video on the internet is widely accepted and, hence, the need for a reliable video streaming architecture is undisputed. The two questions I was asked by readers of that article were:
- what is a codec and why does one need it?
- Why does one need streaming? Couldn’t one just ‘send’ it?
This article is going to deal with both those questions. To start, however, I will take a step back and look at the source of video. Most often it comes from a camera of some sort. The sensor consists of a rectangular grid of pixels that produces huge quantities of data. Let’s consider a sensor size 1920×1080 pixels, 1 byte per color and 60 frames per second for HD (note: for simplicity I am ignoring the reduced size of chrominance as compared to luminance data). This will produce 1920x1080x3x60 megabytes of data in one second. This comes to 355 megabytes per second. A CPU and the bus on the motherboard of a modern-day desktop will be kept busy with this constant stream. The obvious way to handle it is compression. That’s where a codec comes in.
A codec is a device or software for coding and decoding such streams [Ref]. A stream from a camera has to be coded or compressed; in order to display this compressed data the opposite, ie decoding, has to be performed. Over the years there have been many codecs but the current workhorse is H.264 (AVC – Advanced Video Codec). Believe it or not, this codec compresses streams like the one described above to about 0.01% of the original size without any appreciable loss in quality! If you are interested what they do to squeeze it down by that amount read H.264 is Magic.
Work on codecs is ongoing. The next one – H.265 (HEVC – High Efficiency Video Codec) – is being used but there are some uncertainities about IP and patents that is holding it back. In the meantime, Google, the open source community and others are working on codecs such as the VP9, Daala, etc. This new generation can compress video about twice as much as the H.264 and will bring enormous savings once widespread.
Though a codec can drastically reduce the amount of data needed to transmit video, it can overwhelm networks when thousands of users view them. The solution here is to send/receive over the internet just as much data as is needed for a small period of time and to repeat that over and over again. In the early days, servers sent (ie pushed streams) data to those viewing it. It was realized that the clients can judge the momentary network characteristics better than servers and thus started the ‘client pull’ systems. On the web everything eventually gets reduced to HTTP and so it was with video. From ‘chunked transfers’ over HTTP it has evolved to HLS and DASH. These have the advantage that the client can request a lower or higher resolution streams dynamically, depending on the perceived network load. This gives the viewer a better QoE (Quality of Experience).
Though what I have described so far is for video, those basic principles of that can be applied to audio as well. Audio, however, requires a lot less data than video. This makes it possible to have compression that is lossless in contrast to the lossy compression that all video codecs provide. Standard WAV format is a lossless format that has been around for decades; standard CD’s use them. In the past few years lossless formats such as FLAC (Free Lossless Audio Codec) and others have make it possible to recreate the raw digital stream without any compromises for the benefit of audiophiles who have high demands. The rest of us are happy with MP3, a lossy format that compresses well enough.
Though video produces a lot of digital data, compression and streaming make it possible to disseminate it over current network architectures and display it on standard desktops, tablets and mobile phones at reasonable quality.
This is the second part of a series of three articles about video hosting in the cloud.
About the author:
Raju Varghese is a consultant at a large Swiss financial firm. Though he started with computer hardware he quickly moved to software when it was still done on punched cards. Before the web he worked on systems and network software; now anything related to the web and multimedia grabs his interest. Raju is a senior member of the IEEE.