title: How to use "segments" mode at SourceBuffer of MediaSource to render same result at Chomium, Chorme and Firefox?
body: <p>Reference to my original question: <a href="https://stackoverflow.com/questions/45217962/how-to-use-blob-url-mediasource-or-other-methods-to-play-concatenated-blobs-of">How to use Blob URL, MediaSource or other methods to play concatenated Blobs of media fragments?</a></p>
  <p>In lieu of the potential for deprecation of the <code>&quot;sequence&quot;</code> mode for multiple tracks, which the current code is using for both Chromium and Firefox browsers my additional questions are:</p>
  <ol>
  <li><p>Which adjustments need to be made in my <code>MediaSource</code> code to render the same result using both Chromium which Firefox browsers - currently renders as expected using <code>&quot;segments&quot;</code> <code>.mode</code>?</p>
  </li>
  <li><p>Or, is there a bug in the implementation of multitrack support using Chromium browsers when <code>SourceBuffer</code> <code>.mode</code> is set to <code>&quot;segments&quot;</code>?</p>
  </li>
  </ol>
  <hr />
  <h2>Background information</h2>
  <p>I have been able to record discrete media fragments using <code>MediaRecorder</code>, adding cues to the resulting <code>webm</code> file using <a href="https://github.com/legokichi/ts-ebml" rel="nofollow noreferrer"><code>ts-ebml</code></a> and recording the discrete media fragments as a single media file using <code>MediaSource</code> with <code>.mode</code> of <code>SourceBuffer</code> set to <code>&quot;sequence&quot;</code> using both Chromium and Firefox browsers.</p>
  <p>The Chromium issue at <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=737757" rel="nofollow noreferrer">Monitor and potentially deprecate support for multitrack SourceBuffer support of 'sequence' AppendMode</a> discusses <code>&quot;sequence&quot;</code> mode is being considered for deprecation for multitrack <code>SourceBuffer</code> objects. When asked in the original references question regarding how to implement the code using <code>&quot;segments&quot;</code> <code>.mode</code> (default <code>AppendMode</code> of <code>SourceBuffer</code>) the response was essentially that <code>&quot;segments&quot;</code> mode also supports multitrack input at <code>SourceBuffer</code>.</p>
  <p>However, when trying code with <code>.mode</code> of <code>SourceBuffer</code> set to <code>&quot;segments&quot;</code> Chromium 60 only plays approximately one second, the first buffer of multiple appended buffers, of an expected ten second playback of recorded media fragments having cues set at <code>webm</code> file which is converted to <code>ArrayBuffer</code> and passed to <code>.appendBuffer()</code>, while Firefox renders same result when <code>.mode</code> is set to either <code>&quot;sequence&quot;</code> and <code>&quot;segments&quot;</code>.</p>
  <p><a href="https://github.com/guest271314/recordMediaFragments" rel="nofollow noreferrer">Code</a> which renders expected result at both Chromium and Firefox. Note, Firefox does not play <code>.mp4</code> at <code>&lt;video&gt;</code> element if <code>multipleUrls</code> is tried, though Firefox does support playing <code>.mp4</code> at <code>MediaSource</code> when proper media codec is set.</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="false" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>&lt;!DOCTYPE html&gt;
  &lt;html&gt;
  &lt;!-- recordMediaFragments.js demo https://github.com/guest271314/recordMediaFragments/tree/master/demos 2017 guest271314 --&gt;
  
  &lt;head&gt;
    &lt;!-- https://github.com/guest271314/recordMediaFragments/ts-ebml --&gt;
  &lt;/head&gt;
  
  &lt;body&gt;
    &lt;video width="320" height="280" controls="true"&gt;&lt;/video&gt;
    &lt;script&gt;
      (async() =&gt; {
        let request = await fetch("https://raw.githubusercontent.com/guest271314/recordMediaFragments/master/ts-ebml/ts-ebml-min.js");
  
        let blob = await request.blob();
  
        const script = document.createElement("script");
  
        document.head.appendChild(script);
  
        script.src = URL.createObjectURL(blob);
  
        script.onload = () =&gt; {
  
          const tsebml = require("ts-ebml");
  
          const video = document.querySelector("video");
  
          const videoStream = document.createElement("video");
  
          // `MediaSource`
          const mediaSource = new MediaSource();
          // for firefox 
          // see https://bugzilla.mozilla.org/show_bug.cgi?id=1259788
          const hasCaptureStream = HTMLMediaElement.prototype.hasOwnProperty("captureStream");
  
          // handle firefox and chromium
          const captureStream = mediaElement =&gt;
            !!mediaElement.mozCaptureStream 
            ? mediaElement.mozCaptureStream() 
            : mediaElement.captureStream();
  
          let currentFragmentURL, currentBlobURL, fragments;
  
          videoStream.width = video.width;
  
          videoStream.height = video.height;
  
          const mimeCodec = "video/webm;codecs=vp8,opus";
          // set to `.currentTime` of `videoStream` at `pause`
          // to set next media fragment starting `.currentTime`
          // if URL to be set at `.src` has same origin and pathname
          let cursor = 0;
  
          // https://gist.github.com/jsturgis/3b19447b304616f18657
          // https://www.w3.org/2010/05/video/mediaevents.html
          const multipleUrls = [
            "https://media.w3.org/2010/05/sintel/trailer.mp4#t=0,5",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=55,60",
            "https://raw.githubusercontent.com/w3c/web-platform-tests/master/media-source/mp4/test.mp4#t=0,5",
            "https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4#t=0,5",
            "https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerJoyrides.mp4#t=0,5",
            "https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerMeltdowns.mp4#t=0,6",
            "https://media.w3.org/2010/05/video/movie_300.mp4#t=30,36"
          ];
  
          const singleUrl = [
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=0,1",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=1,2",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=2,3",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=3,4",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=4,5",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=5,6",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=6,7",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=7,8",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=8,9",
            "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=9,10"
          ];
  
          const geckoUrl = [
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=10,11",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=11,12",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=12,13",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=13,14",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=14,15",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=15,16",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=16,17",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=17,18",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=18,19",
            "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=19,20"
  
          ];
  
          const mediaFragmentRecorder = async(urls) =&gt; {
            // `ts-ebml`
            const tsebmlTools = async() =&gt; ({
              decoder: new tsebml.Decoder(),
              encoder: new tsebml.Encoder(),
              reader: new tsebml.Reader(),
              tools: tsebml.tools
            });
            // create `ArrayBuffer` from `Blob`
            const readAsArrayBuffer = (blob) =&gt; {
              return new Promise((resolve, reject) =&gt; {
                const fr = new FileReader();
                fr.readAsArrayBuffer(blob);
                fr.onloadend = () =&gt; {
                  resolve(fr.result);
                };
                fr.onerror = (ev) =&gt; {
                  reject(ev.error);
                };
              });
            }
            // `urls`: string or array of URLs
            // record each media fragment
            const recordMediaFragments = async(video, mimeCodec, decoder, encoder, reader, tools, ...urls) =&gt; {
              urls = [].concat(...urls);
              const media = [];
              for (let url of urls) {
                await new Promise(async(resolve) =&gt; {
  
                  let mediaStream, recorder;
  
                  videoStream.onprogress = e =&gt; {
                    videoStream.onprogress = null;
                    console.log("loading " + url)
                  }
  
                  videoStream.oncanplay = async(e) =&gt; {
  
                    videoStream.oncanplay = null;
                    videoStream.play();
  
                    mediaStream = captureStream(videoStream);
                    console.log(mediaStream);
  
                    recorder = new MediaRecorder(mediaStream, {
                      mimeType: mimeCodec
                    });
  
                    recorder.ondataavailable = async(e) =&gt; {
                      // set metadata of recorded media fragment `Blob`
                      const mediaBlob = await setMediaMetadata(e.data);
                      // create `ArrayBuffer` of `Blob` of recorded media fragment
                      const mediaBuffer = await readAsArrayBuffer(mediaBlob);
                      const mediaDuration = videoStream.played.end(0) - videoStream.played.start(0);
                      const mediaFragmentId = currentFragmentURL || new URL(url);
                      const mediaFileName = mediaFragmentId.pathname.split("/").pop() + mediaFragmentId.hash;
                      const mediaFragmentType = "singleMediaFragment";
                      if (currentBlobURL) {
                        URL.revokeObjectURL(currentBlobURL);
                      }
                      media.push({
                        mediaBlob,
                        mediaBuffer,
                        mediaDuration,
                        mediaFragmentType,
                        mediaFileName
                      });
                      resolve();
  
                    }
                    recorder.start();
                  }
                  videoStream.onpause = e =&gt; {
                    videoStream.onpause = null;
                    cursor = videoStream.currentTime;
                    recorder.stop();
                    // stop `MediaStreamTrack`s
                    for (let track of mediaStream.getTracks()) {
                      track.stop();
                    }
                  }
                  currentFragmentURL = new URL(url);
                  // for firefox to load cross origin media without silence 
                  if (!hasCaptureStream) {
                    console.log(currentFragmentURL);
                    request = new Request(currentFragmentURL.href);
                    blob = await fetch(request).then(response =&gt; response.blob());
                    console.log(blob);
                    currentBlobURL = URL.createObjectURL(blob);
                    // set next media fragment URL to `.currentTime` at `pause` event
                    // of previous media fragment if `url` has same `origin` and `pathname`
                    if (urls.indexOf(currentFragmentURL.href) &gt; 0 
                    &amp;&amp; new URL(urls[urls.indexOf(currentFragmentURL.href) - 1]).origin === currentFragmentURL.origin 
                    &amp;&amp; new URL(urls[urls.indexOf(currentFragmentURL.href) - 1]).pathname === currentFragmentURL.pathname) {
                      if (cursor &gt; 0) {
                        url = url = currentBlobURL + currentFragmentURL.hash.replace(/=\d+/, "=" + cursor);
                        console.log(url)
                      }
                    } else {
                      url = currentBlobURL + currentFragmentURL.hash;
                    }
                  } else {
                    if (cursor &gt; 0 
                    &amp;&amp; new URL(urls[urls.indexOf(url) - 1]).origin === currentFragmentURL.origin 
                    &amp;&amp; new URL(urls[urls.indexOf(currentFragmentURL.href) - 1]).pathname === currentFragmentURL.pathname) {
                      url = url.replace(/=\d+/, "=" + cursor);
                      console.log(url)
                    }
                  }
  
  
                  videoStream.src = url;
                }).catch(err =&gt; err)
              }
              return media
            }
            // set metadata of media `Blob`
            // see https://github.com/legokichi/ts-ebml/issues/14#issuecomment-325200151
            const setMediaMetadata = async(blob) =&gt;
              tsebmlTools()
              .then(async({
                decoder,
                encoder,
                tools,
                reader
              }) =&gt; {
  
                let webM = new Blob([], {
                  type: "video/webm"
                });
  
                webM = new Blob([webM, blob], {
                  type: blob.type
                });
  
                const buf = await readAsArrayBuffer(blob);
                const elms = decoder.decode(buf);
                elms.forEach((elm) =&gt; {
                  reader.read(elm);
                });
  
                reader.stop();
  
                const refinedMetadataBuf = tools.makeMetadataSeekable(reader.metadatas, reader.duration, reader.cues);
  
                const webMBuf = await readAsArrayBuffer(webM);
  
                const body = webMBuf.slice(reader.metadataSize);
                const refinedWebM = new Blob([refinedMetadataBuf, body], {
                  type: webM.type
                });
                // close Blobs
                if (webM.close &amp;&amp; blob.close) {
                  webM.close();
                  blob.close();
                }
  
                return refinedWebM;
              })
              .catch(err =&gt; console.error(err));
  
  
            let mediaTools = await tsebmlTools();
  
            const {
              decoder,
              encoder,
              reader,
              tools
            } = mediaTools;
  
            const mediaFragments = await recordMediaFragments(video, mimeCodec, decoder, encoder, reader, tools, urls);
  
            const recordedMedia = await new Promise((resolveAllMedia, rejectAllMedia) =&gt; {
              console.log(decoder, encoder, tools, reader, mediaFragments);
  
              let mediaStream, recorder;
  
              mediaSource.onsourceended = e =&gt; {
                console.log(video.buffered.start(0), video.buffered.end(0));
                video.currentTime = video.buffered.start(0);
  
                console.log(video.paused, video.readyState);
  
                video.ontimeupdate = e =&gt; {
  
                  console.log(video.currentTime, mediaSource.duration);
                  if (video.currentTime &gt;= mediaSource.duration) {
                    video.ontimeupdate = null;
                    video.oncanplay = null;
                    video.onwaiting = null;
                    if (recorder.state === "recording") {
                      recorder.stop();
                    }
                    console.log(e, recorder);
  
                  }
                }
              }
              video.onended = (e) =&gt; {
                video.onended = null;
                console.log(e, video.currentTime,
                  mediaSource.duration);
              }
              video.oncanplay = e =&gt; {
                console.log(e, video.duration, video.buffered.end(0));
                video.play()
              }
              video.onwaiting = e =&gt; {
                console.log(e, video.currentTime);
              }
              // record `MediaSource` playback of recorded media fragments
              video.onplaying = async(e) =&gt; {
                console.log(e);
                video.onplaying = null;
  
                mediaStream = captureStream(video);
                if (!hasCaptureStream) {
                  videoStream.srcObject = mediaStream;
                  videoStream.play();
                }
                recorder = new MediaRecorder(mediaStream, {
                  mimeType: mimeCodec
                });
                console.log(recorder);
  
                recorder.ondataavailable = async(e) =&gt; {
                  console.log(e);
  
                  const mediaFragmentsRecording = {};
  
                  mediaFragmentsRecording.mediaBlob = await setMediaMetadata(e.data);
                  mediaFragmentsRecording.mediaBuffer = await readAsArrayBuffer(mediaFragmentsRecording.mediaBlob);
                  mediaFragmentsRecording.mediaFileName = urls.map(url =&gt; {
                    const id = new URL(url);
                    return id.pathname.split("/").pop() + id.hash
                  }).join("-");
                  mediaFragmentsRecording.mediaFragmentType = "multipleMediaFragments";
                  // `&lt;video&gt;` to play concatened media fragments
                  // recorded from playback of `MediaSource`
                  fragments = document.createElement("video");
                  fragments.id = "fragments";
                  fragments.width = video.width;
                  fragments.height = video.height;
                  fragments.controls = true;
                  fragments.onloadedmetadata = () =&gt; {
                    fragments.onloadedmetadata = null;
                    mediaFragmentsRecording.mediaDuration = fragments.duration;
                    URL.revokeObjectURL(currentBlobURL);
                    // stop `MediaStreamTrack`s
                    for (let track of mediaStream.getTracks()) {
                      track.stop();
                    }
                    resolveAllMedia([
                      ...mediaFragments, mediaFragmentsRecording
                    ]);
  
                  }
                  currentBlobURL = URL.createObjectURL(mediaFragmentsRecording.mediaBlob);
                  fragments.src = currentBlobURL;
                  document.body.appendChild(fragments);
  
                }
  
                recorder.start();
              }
  
              video.src = URL.createObjectURL(mediaSource);
  
              mediaSource.addEventListener("sourceopen", sourceOpen);
  
              async function sourceOpen(e) {
  
                if (MediaSource.isTypeSupported(mimeCodec)) {
                  const sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
                  sourceBuffer.mode = "segments";
                  for (let {
                      mediaBuffer,
                      mediaDuration
                    } of mediaFragments) {
  
                    await new Promise((resolveUpdatedMediaSource) =&gt; {
  
                      sourceBuffer.onupdateend = async(e) =&gt; {
                        sourceBuffer.onupdateend = null;
                        console.log(e, mediaDuration, mediaSource.duration
                        , video.paused, video.ended, video.currentTime
                        , "media source playing", video.readyState);
  
                        // https://bugzilla.mozilla.org/show_bug.cgi?id=1400587
                        // https://bugs.chromium.org/p/chromium/issues/detail?id=766002&amp;q=label%3AMSEptsdtsCleanup
                        try {
                          sourceBuffer.timestampOffset += mediaDuration;
                          resolveUpdatedMediaSource();
                        } catch (err) {
                          console.error(err);
                          resolveUpdatedMediaSource();
                        }
                      }
                      sourceBuffer.appendBuffer(mediaBuffer);
                    })
                  }
  
                  mediaSource.endOfStream()
  
                } else {
                  console.warn(mimeCodec + " not supported");
                }
              };
  
            })
  
            return recordedMedia
          };
  
          mediaFragmentRecorder(geckoUrl)
            .then(recordedMediaFragments =&gt; {
              // do stuff with recorded media fragments
              console.log(recordedMediaFragments);
              const select = document.createElement("select");
              for (let {
                  mediaFileName,
                  mediaBlob,
                  mediaFragmentType
                } of Object.values(recordedMediaFragments)) {
                const option = new Option(mediaFileName, URL.createObjectURL(mediaBlob));
                select.appendChild(option);
              }
              select.onchange = () =&gt; {
                document.getElementById("fragments").src = select.value;
              }
              video.parentNode.insertBefore(select, video);
              video.controls = true;
              video.currentTime = video.buffered.start(0);
            })
            .catch(err =&gt; console.error(err));
        }
      })()
    &lt;/script&gt;
  &lt;/body&gt;
  
  &lt;/html&gt;</code></pre>
  </div>
  </div>
  </p>