title: How to calculate containers' cpu usage in kubernetes with prometheus as monitoring?
body: <p>I want to calculate the cpu usage of all pods in a kubernetes cluster. I found two metrics in prometheus may be useful:</p>
  
  <pre><code>container_cpu_usage_seconds_total: Cumulative cpu time consumed per cpu in seconds.
  process_cpu_seconds_total: Total user and system CPU time spent in seconds.
  
  Cpu Usage of all pods = increment per second of sum(container_cpu_usage_seconds_total{id="/"})/increment per second of sum(process_cpu_seconds_total)
  </code></pre>
  
  <p>However, I found every second's increment of <code>container_cpu_usage{id="/"}</code> larger than the increment of <code>sum(process_cpu_seconds_total)</code>. So the usage may be larger than 1...</p>