title: Implement sub millisecond processing in python without busywait
body: <p>How would i implement processing of an array with millisecond precision using python under linux (running on a single core Raspberry Pi).</p>
  
  <p>I am trying to parse information from a MIDI file, which has been preprocessed to an array where each millisecond i check if the array has entries at the current timestamp and trigger some functions if it does.</p>
  
  <p>Currently i am using time.time() and employ busy waiting (as concluded <a href="https://raspberrypi.stackexchange.com/questions/3898/nanosleep-wont-sleep-short-time">here</a>). This eats up all the CPU, therefor i opt for a better solution.</p>
  
  <pre><code># iterate through all milliseconds
  for current_ms in xrange(0, last+1):
    start = time()
  
    # check if events are to be processed
    try:
      events = allEvents[current_ms]
      # iterate over all events for this millisecond
      for event in events:
        # check if event contains note information
        if 'note' in event:
          # check if mapping to pin exists
          if event['note'] in mapping:
            pin = mapping[event['note']]
            # check if event contains on/off information
            if 'mode' in event:
              if event['mode'] == 0:
                pin_off(pin)
              elif event['mode'] == 1:
                pin_on(pin)
              else:
                debug("unknown mode in event:"+event)
            else:
              debug("no mapping for note:" + event['note'])
    except:
      pass
  
    end = time()
  
    # fill the rest of the millisecond
    while (end-start) &lt; (1.0/(1000.0)):
      end = time()
  </code></pre>
  
  <p>where <code>last</code> is the millisecond of the last event (known from preprocessing)</p>
  
  <p>This is not a question about <a href="https://stackoverflow.com/q/85451/875020">time() vs clock()</a> more about <a href="https://stackoverflow.com/q/1107593/875020">sleep vs busy wait</a>.</p>
  
  <p>I cant really sleep in the "fill rest of millisecond" loop, because of the <a href="https://stackoverflow.com/questions/17499837/python-time-sleep-vs-busy-wait-accuracy">too low</a> <a href="https://stackoverflow.com/a/1133888/875020">accuracy of sleep()</a>. If i were to use <a href="https://stackoverflow.com/a/1212070/875020">ctypes</a>, how would i go about it correctly?</p>
  
  <p>Is there some Timer library which would call a callback each millisecond reliably?</p>
  
  <p>My current implementation is on <a href="https://github.com/x29a/ceremony/blob/15444d38f99732b7d3cb9d2b41a78b66718b9916/player/player.py#L114" rel="nofollow noreferrer">GitHub</a>. With this approach i get a skew of around 4 or 5ms on the drum_sample, which is 3.7s total (with mock, so no real hardware attached). On a 30.7s sample, the skew is around 32ms (so its at least not linear!).</p>
  
  <p>I have tried using <a href="http://benjiyork.com/blog/2007/04/sleep-considered-harmful.html" rel="nofollow noreferrer">time.sleep()</a> and <a href="https://github.com/graycatlabs/PyBBIO/blob/master/tests/sleep_test.py" rel="nofollow noreferrer">nanosleep() via ctypes</a> with the following code</p>
  
  <pre><code>import time
  import timeit
  import ctypes
  libc = ctypes.CDLL('libc.so.6')
  
  class Timespec(ctypes.Structure):
    """ timespec struct for nanosleep, see:
        http://linux.die.net/man/2/nanosleep """
    _fields_ = [('tv_sec', ctypes.c_long),
                ('tv_nsec', ctypes.c_long)]
  
  libc.nanosleep.argtypes = [ctypes.POINTER(Timespec),
                             ctypes.POINTER(Timespec)]
  nanosleep_req = Timespec()
  nanosleep_rem = Timespec()
  
  def nsleep(us):
    #print('nsleep: {0:.9f}'.format(us)) 
    """ Delay microseconds with libc nanosleep() using ctypes. """
    if (us &gt;= 1000000):
      sec = us/1000000
      us %= 1000000
    else: sec = 0
    nanosleep_req.tv_sec = sec
    nanosleep_req.tv_nsec = int(us * 1000)
  
    libc.nanosleep(nanosleep_req, nanosleep_rem)
  
  LOOPS = 10000
  
  def do_sleep(min_sleep):
    #print('try: {0:.9f}'.format(min_sleep))
    total = 0.0
    for i in xrange(0, LOOPS):
      start = timeit.default_timer()
      nsleep(min_sleep*1000*1000)
      #time.sleep(min_sleep)
      end = timeit.default_timer()
      total += end - start
    return (total / LOOPS)
  
  iterations = 5
  iteration = 1
  min_sleep = 0.001
  result = None
  while True:
      result = do_sleep(min_sleep)
      #print('res: {0:.9f}'.format(result))
      if result &gt; 1.5 * min_sleep:
        if iteration &gt; iterations:
          break
        else:
          min_sleep = result
          iteration += 1
      else:
        min_sleep /= 2.0
  
  print('FIN: {0:.9f}'.format(result))
  </code></pre>
  
  <p>The result on my i5 is </p>
  
  <blockquote>
    <p>FIN: 0.000165443</p>
  </blockquote>
  
  <p>while on the RPi it is</p>
  
  <blockquote>
    <p>FIN: 0.000578617</p>
  </blockquote>
  
  <p>which suggest a sleep period of about 0.1 or 0.5 milliseconds, with the given jitter (tend to sleep longer) that at most helps me reduce the load a little bit.</p>