title: How to filter out matching results from prometheus query
body: <p>How to exclude two sets of metrics from prometheus query results? Here's sudo code for what I want to do:</p>
  <pre><code>kubelet_volume_stats_available_bytes | remove(
     kubelet_volume_stats_available_bytes{namespace=&quot;ignore-this&quot;},
     kubelet_volume_stats_available_bytes{namespace=&quot;default&quot;, pvc=&quot;cache&quot;}
  )
  </code></pre>
  <p>Some background - we have prometheus alert that fires when volume is predicted to be full in 4 days. It is great alert for most part. Unfortunately there are two systems where full disk is fine. One system is running in separate namespace, which is trivial to exclude, another system is running in default kubernetes namespace, so for that I want to exclude exactly one metric (namespace=default, pvc=cache).</p>
  <p>Here's full query:</p>
  <pre><code>(
      kubelet_volume_stats_available_bytes{job=&quot;kubelet&quot;, namespace!=&quot;ignore-this&quot;, metrics_path=&quot;/metrics&quot;}
      /
      kubelet_volume_stats_capacity_bytes{job=&quot;kubelet&quot;, namespace!=&quot;ignore-this&quot;, metrics_path=&quot;/metrics&quot;}
  ) &lt; 0.15
  and
  predict_linear(
      kubelet_volume_stats_available_bytes{job=&quot;kubelet&quot;, namespace!=&quot;ignore-this&quot;, metrics_path=&quot;/metrics&quot;}[6h], 4 * 24 * 3600
  )
  &lt; 0
  
  </code></pre>
  <p>How to ignore metrics with these labels <code>namespace=default, pvc=cache</code>?</p>