title: Streaming RTP/RTSP: sync/timestamp problems
body: <p>I'm having some trouble streaming H.264 video over RTSP. The goal is to live-stream a camera image to an RTSP client (ideally a browser plugin in the end). This has been working pretty well so far, except for one problem: the video will lag on startup, stutter every few seconds, and has a ~4-second delay. This is bad.</p>
  
  <p>Our setup is to encode with x264 (w/ zerolatency &amp; ultrafast) and packed into RTSP/RTP with libavformat from ffmpeg 0.6.5. For testing, I'm receiving the stream with a GStreamer pipeline with gst-launch when connecting to an RTSP server. <strong>However</strong>, I've been able to reproduce the same issue when streaming straight from another GStreamer instance with just RTP.</p>
  
  <p>Sending machine:</p>
  
  <pre><code>gst-launch videotestsrc ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=10.89.6.3
  </code></pre>
  
  <p>Receiving machine:</p>
  
  <pre><code>gst-launch udpsrc ! application/x-rtp,payload=96 ! rtph264depay ! decodebin ! xvimagesink
  </code></pre>
  
  <p>You can also run these both on the same machine, just change the host to 127.0.0.1 on the sender. On the receiving end, you should notice stuttering and generally poor-performing video, along with repeated warnings on the console:</p>
  
  <pre><code>WARNING: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: A lot of buffers are being dropped.
  Additional debug info:
  gstbasesink.c(2875): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
  There may be a timestamping problem, or this computer is too slow.
  </code></pre>
  
  <p>One commonly-suggested "fix" that I've seen all over the Internet is to use <code>sync=false</code> with xvimagesink:</p>
  
  <pre><code>gst-launch udpsrc ! application/x-rtp,payload=96 ! rtph264depay ! decodebin ! xvimagesink sync=false
  </code></pre>
  
  <p>The video will then play back with near-zero latency, even when tested with our camera software. This is useful for testing, but is not very useful for deployment, as it won't work with Totem, VLC, or their browser plugin embeds.</p>
  
  <p>I'd like to try to solve the issue at the source; I'm suspicious that there's some sort of timestamp info missing on the H.264 stream by x264 or perhaps on the RTP payloads. Is there any way to modify the <em>source</em> gst pipeline so that I do <strong>not</strong> need to use <code>sync=false</code> on the receiver?</p>
  
  <p>If that's not possible, how can I tell clients (via SDP or otherwise) that the stream should not be synchronized? Ultimately, we'd embed this in the browser using a VLC plugin of sorts, so a solution that would work there would be even better.</p>