01

What Protocol Delivers This Video?

The network protocol used to deliver this video depends on the device and browser.

  • Mobile app on a phone → uses native app protocol
  • Web browser → depends on browser version

All three protocols—UDP, TCP, and HTTP—are theoretically possible.

02

Why All Three Protocols Are Viable

  • UDP is ideal for live streaming, teleconferences, and video calls
  • UDP avoids retransmitting lost packets, which is critical for real-time data
  • TCP and HCCP are better for recorded videos, where image quality matters more than speed
03

Browser-Based Streaming Options

  • Raw TCP and UDP sockets cannot be created in browsers
  • Modern browsers support technologies that use TCP or UDP

WebRTC

  • A suite of protocols and interfaces for peer-to-peer communication
  • Supports high-quality video and audio
  • Uses UDP by default

WebSocket

  • Enables bidirectional communication between client and server
  • Uses TCP
04

Streaming Video Over HTTP

We will discuss adaptive streaming shortly.

05

The Core Problem in Media Streaming

At first glance, streaming seems simple.

The Simple Design Approach

  • Video owner uploads a video to a website
  • Stored in object storage (e.g., S3)
  • Metadata stored in a database
  • When a user opens a video, the server returns a link
  • Browser downloads and plays the video

This approach was common in 2005.

06

Limitations of the Simple Design

  • Cannot adapt to changing network conditions
  • No dynamic switching between video qualities
  • No support for on-the-fly language changes

Example: YouTube and Netflix

  • YouTube allows switching video quality during playback
  • Netflix lets users change language mid-stream

Key Limitations

  • The design downloads the entire video file at once

  • No adaptation to bandwidth

  • Requires separate video files for each resolution and language

  • Separating video and audio tracks helps, but is not always feasible

07

A Better Solution

Instead of a single video file, we should design a system that supports multiple video qualities and languages

This enables dynamic adaptation to network conditions and user preferences

The system must allow seamless switching without restarting the download

This approach supports real-time user experience improvements

The foundation of modern video streaming lies in adaptive delivery

The shift from static files to dynamic, segmented content is essential

This design enables responsive, user-friendly video experiences

08

Video Segmentation: Breaking Content into Time-Based Pieces

Each segment represents a piece of video content, typically 2 to 10 seconds long.

09

Media Player Retrieves Segments via HTTP

A media player—implemented as a JavaScript application in the browser—retrieves individual segments from a web server or directly from object storage using HTTP.

10

Playback Begins After Initial Buffering

Playback starts as soon as the player has buffered the first several segments.

11

Adaptive Streaming Through Variable Bit Rates

Segmentation enables adaptive streaming when content is available in multiple bit rates.

12

Video Transcoding Creates Quality-Specific Segments

When a video is uploaded, segments are generated for different resolutions.

13

Client Selects Bit Rate Dynamically

A browser-based media player uses a bit rate adaptation algorithm to select the highest possible bit rate segment.

14

Network-Driven Quality Switching

If network conditions degrade, the player switches to lower-quality segments.

When network performance improves, it resumes downloading high-quality segments.

15

Audio Streams Also Support Dynamic Switching

The same principle applies to audio: when language changes, only the selected language’s audio segments begin downloading.

16

Manifest File Stores Metadata

A metadata file called the manifest index contains information about available qualities, supported languages, and segment locations.

17

Client Downloads and Parses the Manifest

The client first downloads the manifest file, parses it, and then begins downloading segments sequentially.

18

Full Workflow: Upload to Segmentation

A video owner uploads a video, which is stored in object storage.

This triggers a job to generate segments for various qualities and a manifest file.

19

Multiple Jobs for Scalable Processing

The process may involve multiple jobs, each responsible for generating segments at a specific quality.

20

Alignment with Compute Design Pattern

This scenario exemplifies the big compute design pattern—thousands of videos requiring multiple transcode jobs.

21

Jobs Upload Segments to Object Storage

Each job produces segment files and uploads them back to object storage.

22

Manifest Storage Options

The manifest file can be stored in either object storage or a database.

23

Viewer Interaction Flow

When a viewer opens a video, a media player is downloaded by the browser.

On pressing play, the browser sends an HTTP request to a database service to retrieve the manifest file.

24

Media Player Parses and Uses Manifest

The media player parses the manifest to obtain segment locations for the video.

25

Browser Downloads and Plays Segments

The browser then retrieves each segment via HTTP and plays them in sequence.

26

Live Streaming Applies the Same Model

This segmentation model can be used for live streaming, such as on Twitch or YouTube.

27

Streamer Sends Content to Server

The streamer sends live content to a server, which produces segments of several seconds.

28

Segments Stored in Object Storage

These segments are stored in object storage.

29

Metadata Persisted in Database

Metadata about each new segment is stored in a database.

30

Client Polls for New Segments

The client continuously checks for new segment data.

31

Browser Retrieves New Segments Immediately

As soon as a new segment becomes available, the browser retrieves its content.

Questions

  • What network protocol is used to deliver a video when watching it on a mobile native app versus a web browser?
  • Why might UDP be preferred over TCP or HCCP for live video streaming?
  • Can a browser use raw TCP or UDP sockets to stream video content, and if so, under what conditions?
  • What is the role of the manifest file in adaptive streaming, and how is it accessed by a client?
  • How does a media player adapt to changing network conditions when playing a video?
  • What happens when a viewer changes the video language or resolution during playback?
  • How is the video segmentation process triggered after a video is uploaded?
  • Can the same segmentation model be applied to live streaming, and if so, how?

Key takeaways

  • The protocol used to deliver video depends on the device and browser, and all three—UDP, TCP, and HCCP—are theoretically possible.
  • UDP is suitable for live video due to its low latency, while TCP and HCCP are better for recorded videos where image quality matters more than speed.
  • Modern browsers support WebRTC (using UDP) and WebSockets (using TCP) for video streaming, even though raw sockets aren't directly available.
  • A simple video file approach fails to adapt to network conditions or support multiple qualities and languages without significant overhead.
  • Video content is split into segments (2–10 seconds) to enable adaptive streaming, where the player selects quality based on network bandwidth.
  • A manifest file provides metadata about available video qualities, languages, and segment locations and is downloaded first by the client.
  • After upload, a job generates segments for different qualities, which may be processed in multiple parallel jobs for large-scale video uploads.
  • The same segmentation model can be applied to live streaming, where segments are produced in real time and delivered as they become available.
  • The media player retrieves segments via HTTP from object storage after parsing the manifest file, enabling on-the-fly quality adaptation.