I-Frames, P-Frames and GOP Length (IPStreamMedia #2)

A little bit of theory

In the last devlog I spoke about how the IPStreamMedia idea came to me. In this one I’m going to talk about a few concepts I learned to understand how RTSP streams and video compression works. But this rabbit hole turned out to be a bit deeper than I expected. I will just discuss a few things that I learned so that the jargons I might use later don’t confuse y’all.

Codec, Container, Protocol

Instead of me verbally explaining the difference between the three with long explanations, I think this table just works better.

Frames

Before we start livestreaming video (or even saving it on disk), compression is an important step that enables us to transmit/store high-quality video data in a much smaller memory footprint. Compression techniques used by codecs like H.265, H.264, etc. can compress a raw video bitstream upto 150x to 600x !

This compression happens by converting the raw video frames into a special set of frames as discussed below,

• I-frames: Single frame of video which is only compressed on it’s own, kind of like standard JPEG compression for an image. It can be read on its own if needed.

• P-frames: Single frame of video which is compressed temporally, these frames carry motion vector data which only defines what changed between the current frame and the previous ones, instead of the whole picture. Hence these frames cannot be decoded or read on their own. These are much more efficient compared to I-frames, but they need an I-frame as an anchor/reference point to be decoded correctly.

• B-frames: Similar to P-frames, these are also compressed temporally. But they are created using frames from the past as well as the future, which makes them extremely efficient memory-wise, but their encoding and decoding takes longer as the codec needs to always process the next frame before fully encoding/decoding the present frame. Hence these frames are not very popular for low-latency compression.

GOP

GOP or Group of Pictures is a block which consists of an I-frame followed by a bunch of P-frames. The length of a GOP is either defined in frames or seconds and it basically defines the number of consecutive P-frames between two I-frames.

I P P P P P P P P I P P P P P P P P I ...
└──────GOP───────┘ └──────GOP───────┘

Now why is GOP length important ? As I said before, a P-frame cannot be decoded by itself, it always needs it’s previous P-frames as well as an anchor/reference I-frame to be decoded properly. At the same time a P-frame is much cheaper than an I-frame memory-wise. So engineers have to decide the GOP length of a video stream as per their use-case.

For livestreaming, GOP length can define the maximum latency experienced by a user trying to connect to the livestream url.

In case the user joined in the middle of a GOP, they are going to start receiving P-frames without a reference I-frame as it has already passed before the user joined. So now the user will not be able to see any video until the next I-frame arrives which will mark the start of a new GOP. You may think that why not just keep the GOP length as small as possible ? This is because then the number of I-frames in the stream will increase, which in turn will increase the memory footprint of the video. Hence it’s a balancing act between latency and bandwidth.

For livestreams, I have personally seen GOPs ranging from 4s to 1s and can be even lower in low-latency scenarios where high bandwidth is available. In case of video recordings the GOPs can be longer, which results in much better compression.

What’s Next ?

The idea behind explaining this theory was to help y’all make sense of the experiments that I will be documenting in the next devlog, where I used FFmpeg to understand the contents of a live RTSP stream coming from one of my CCTV cameras.

Initially I had planned on documenting that in this devlog — but this one’s already gotten a bit too long. Cheers! 🍻