title: Using pandas to fill gaps only, and not NaNs on the ends
body: <p>I have some housing price data that spans about 8 months, and tracks the price as houses come onto the market up until they are sold. There are a couple gaps in the data in the middle that I'd like to fill in, but I'd like to leave the NaNs on the end of each untouched. </p>
  
  <p>To use a simple example, let's say we have house1 that comes on the market for 200000 on 'Day 4', and sells for 190000 on 'Day 9'. And we have house2 that stays at 180000 for Days 1 - 12 and doesn't sell in that time window. But, something went wrong on days 6 and 7 and I lost the data:</p>
  
  <pre><code>house1 = [NaN, NaN, NaN, 200000, 200000, NaN, NaN, 200000, 190000, NaN, NaN, NaN]
  house2 = [180000, 180000, 180000, 180000, 180000, NaN, NaN, 180000, 180000, 180000, 180000, 180000]
  </code></pre>
  
  <p>Now imagine instead of regular arrays these were columns in Pandas Dataframes indexed by date.</p>
  
  <p>The trouble is, the function I would normally use to fill the gaps here would be <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html" rel="nofollow">DataFrame.fillna()</a> using either the backfill or ffill methods. If I use ffill, house1 returns this:</p>
  
  <pre><code>house1 = [NaN, NaN, NaN, 200000, 200000, 200000, 200000, 200000, 190000, 190000, 190000, 190000]
  </code></pre>
  
  <p>Which fills the gap, but also incorrectly fills the data past the day of sale. If I use backfill instead, I get this:</p>
  
  <pre><code>house1 = [200000, 200000, 200000, 200000, 200000, 200000, 200000, 200000, 190000, NaN, NaN, NaN]
  </code></pre>
  
  <p>Again, it fills the gap, but this time it also fills the front end of the data. If I use 'limit=2' with ffill, then what I get is:</p>
  
  <pre><code>house1 = [NaN, NaN, NaN, 200000, 200000, 200000, 200000, 200000, 190000, 190000, 190000, NaN]
  </code></pre>
  
  <p>Once again, it fills the gap, but then it also begins to fill the data beyond the end of where the 'real' data ends.</p>
  
  <p>My solution so far was to write the following function:
  </p>
  
  <pre><code>def fillGaps(houseDF):
      """Fills up holes in the housing data"""
  
      def fillColumns(column):
          filled_col = column
          lastValue = None
          # Keeps track of if we are dealing with a gap in numbers
          gap = False
          i = 0
          for currentValue in filled_col:
              # Loops over all the nans before the numbers begin
              if not isANumber(currentValue) and lastValue is None:
                  pass
              # Keeps track of the last number we encountered before a gap
              elif isANumber(currentValue) and (gap is False):
                  lastIndex = i
                  lastValue = currentValue
              # Notes when we encounter a gap in numbers
              elif not isANumber(currentValue):
                  gap = True
              # Fills in the gap
              elif isANumber(currentValue):
                  gapIndicies = range(lastIndex + 1, i)
                  for j in gapIndicies:
                      filled_col[j] = lastValue
                  gap = False
              i += 1
          return filled_col
  
      filled_df = houseDF.apply(fillColumns, axis=0)
      return filled_df
  </code></pre>
  
  <p>It simply skips all the NaNs in front, fills in the gaps (defined by groups of NaNs between real values), and doesn't fill in NaNs on the end.</p>
  
  <p>Is there a cleaner way to do this, or a built-in pandas function I'm unaware of?</p>