Encoding Ffmpeg to Mpeg-Dash - or Webm with Keyframe Clusters - for Mediasource API

Encoding FFMPEG to MPEG-DASH – or WebM with Keyframe Clusters – for MediaSource API

To ensure every cluster in your WebM starts with a keyframe, try something like this:

ffmpeg \
  [...inputs] \
  -vcodec libvpx \
  -keyint_min 60 \
  -g 60 \
  -vb 4000k \
  -f webm \
  -cluster_size_limit 10M \
  -cluster_time_limit 2100 \
  [...output]

Basically, as implemented, every keyframe has to be at the beginning of a cluster but the inverse is not true. That is, on key frame there will be a new cluster, but on new cluster there won't necessarily be a keyframe. To get around this issue, we simply set the cluster size to something large that we'll never hit.

In this example, we'll have a keyframe every 2 seconds, and the cluster time limit is 2.1 seconds, so we'll never hit it. The bitrate is 4Mbit, and the cluster size limit is 10M-something. Not sure if it's bit or byte there but it doesn't matter as we'll never hit it as I've set it much greater than it needs to be.

Media Source Api not working for a custom webm file (Chrome Version 23.0.1271.97 m)

The most likely problem is your WebM file has Clusters that don't start with a keyframe.

In Chrome dev-channel builds (ie Chrome 25 or later), you can verify this with the following steps.

1. Open chrome:media-internals in another tab.
2. Return to the tab with your test page and reload it.
3. When the error occurs again, switch back to chrome:media-internals tab and look for the bottom entry under the "Active media players:" header. It should have the same blob: URL that you passed to the video element src attribute.
4. Click on the blob: URL to expand the player data.
5. Click on the "Log:" entry to expose the player logging data.
6. Search for entries that have "MEDIA_SOURCE_ERROR" in the "Event:" column. These entries should provide information about what is wrong with the content passed to the browser.

If you see a message that says something like "Media segment did not begin with keyframe." then it means your file has Clusters that don't start with a keyframe. This is common with content that is generated by FFmpeg. You can fix your file in one of the following ways:

1. Run the sample_muxer program mentioned in section 2.2.5 of the WebM adaptive streaming guide
2. Run the mse_webm_remuxer program I wrote as part of my mse-tools project.

Segment FFmpeg output at its keyframe

The segment muxer cuts only at keyframes (normally) but not automatically at each keyframe. You haven't set, or more precisely, overridden the segment duration option.

So,

ffmpeg -f x11grab -framerate 30 -i :0.0 \
       -vcodec libx264 -x264-params keyint=1 -t 5 -pix_fmt yuv420p \
       -f ssegment -segment_time 0.001 -segment_format mp4 \
       -segment_format_options movflags=+frag_keyframe+empty_moov+default_base_moof \
  out%03d.mp4

With keyint=1 (which is equivalent to max KF interval), you don't need min-keyint or scenecut or force_key_frames.

Live streaming dash content using mp4box

You don't need MP4Box to generate the required output, but you'll need to chunk the content yourself looking for boxes in the generated file.

Basically you'll generate an fMP4 with H264, and send to the browser the moov box for initialization and the moof+mdat boxes for each fragment of MP4 that you generate. You'll have to code the player in JavaScript, you probably won't be able to use a standard DASH player.

To generate the correct fragmented MP4, you need to pass this to ffmpeg: -movflags empty_moov+omit_tfhd_offset+frag_keyframe+default_base_moof.

Be sure to use the latest version available.

How should frames be packed in a live WebM stream?

So I finally managed to mux ar working live stream.

It seems that the initial approach I described (having a single cluster with a single SimpleBlock) actually works as such, but it has several downsides:

• It is kind of violating the recommendations on the official WebM page

Key frames SHOULD be placed at the beginning of clusters

• It breaks up possible seeking if the live stream is stored in a local file with curl or other means. From my understanding a Cluster should consist of a full GOP.

One of my initial assumptions is that a Cluster cannot have an "unknown" size, but in practice it seemed out that Chrome, VLC and ffplay were happy with that and so there is no need to buffer a full GOP to determine the size and the Cluster can be emitted on the fly.

Another important aspect is that the timestamps in the SimpleBlock elements are signed 16bit integers so you basically can encode an offset from the cluster timecode up to 32767 in that. So if you are using the default timescale where 1 tick is 1ms, this means a Cluster cannot be longer than 32 seconds. In case the GOP size is huge this criteria must also be taken into account when deciding whether to emit a new cluster.

Finally, here is a link to a live stream (The "Big Buck Bunny" trailer, but in a live format) that seems to work with all the players and is generated as per the description above.

Hope this information helps anyone.

---

Related Topics