title: How to convert H.264 UDP packets to playable media stream or file (defragmentation)
body: <p>I am missing some fundamental thing in translating an UDP stream of a SDP session into a decodable H.264 stream. I am testing with a H.264 capable camera and can play the stream with a player directly. When I try to play the translated stream it will not be recognized by the player (missing header error). However I have to decode the UDP stream to be able to integrate this in a Java application for which there are some decoders around.</p>
  
  <p>I have seen very good answers to following questions already:</p>
  
  <ol>
  <li><a href="https://stackoverflow.com/questions/7665217/how-to-process-raw-udp-packets-so-that-they-can-be-decoded-by-a-decoder-filter-i">How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter</a></li>
  <li><a href="https://stackoverflow.com/questions/3493742/problem-to-decode-h264-video-over-rtp-with-ffmpeg-libavcodec">Problem to Decode H264 video over RTP with ffmpeg (libavcodec)</a></li>
  </ol>
  
  <p>Both have some small differences which are confusing (see below). </p>
  
  <p>But first let us look at the easy part. As I see the from the camera there are SPS and PPS packets sent. All the remaining packets are fragmented frames indexed or not.</p>
  
  <p>For all the packets without frames (only NALUnitType 7 and 8 in my case) I strip of the RTP Header (12 Bytes) and add starting bytes 3 x 0 bytes and 1 x 1 in front (00 00 00 01).</p>
  
  <p>For all fragmented frame packets I reconstruct them according the description of the answer 1. So in detail this means:
  Strip of the RTP header (just use this for data verification).
  Then decode from the payload the fragment information:</p>
  
  <p>First byte:  [ 3 NAL UNIT BITS | 5 FRAGMENT TYPE BITS]<br>Second byte: [ START BIT | END BIT | RESERVED BIT | 5 NAL UNIT BITS]</p>
  
  <p>If start bit is set there is a new payload header constructed as this: [3 NAL UNIT BITS (from first byte)| 5 NAL UNIT BITS (from second byte)]<br>
  This gives us a NALUnitType 1 for an non idr slice or a 5 for an idr slice. Which is according to the specification.</p>
  
  <p>I take this new payload header (1 byte) and attach the payload without the 2 bytes  header into a new package. All consecutive fragments are added the same way (strip of 12 bytes RTP header, strip of 2 bytes of unit type information) until there is an end bit information seen. When the end is seen I put start bytes (00 00 00 01) in front of this packet and write it out to the stream.</p>
  
  <p>The problem is it can not be decoded for unknown reason. The difference in answer 2 of the answers I have read is that the second byte of the payload header might be put into the translated packet as well. But I tried both and still no luck.</p>
  
  <p>Probably there is something other missing in the newly constructed stream ? Or do I make a mistake in the defragmentation?</p>