title: rook-ceph provides PV also when there's no more capacity
body: <p>I'm using rook-ceph in kubernetes.
  I deployed rook-ceph-operator and rook-ceph-cluster Helm charts.</p>
  <p>I have 5 worker nodes with 2 OSD each one. Each OSD has 100GB. So in total, I have 5*200GB = 1TB raw storage. In CephBlockPool I set <code>replicated=3 failureDomain=host</code>.</p>
  <p>I created a POD with a PVC which ask for 500Gi and it works.
  Then I can create more PODs with PVC &gt; 500Gi and it works also.
  Basically I can create hundreds PVC and I have hundreds of PV Bounded, but the TOTAL space is more than 1TB, which is the maximum.</p>
  <p>What I expected? If I create PVs, and they exceeds 1TB, they SHOULD NOT be created. I don't want to use thin provisioning.</p>
  <p>Last thing, my REAL storage should be 1TB/3 because I have a replica of 3.</p>
  <p>I tried using <code>ResourceQuotas</code> with <code>requests.storage</code> and it works but the problem is that it's applied per namespace. And I have a lot of namespaces. I need to limit the PCV requests to all the cluster.</p>
  <p>My question is: How can I limit the storage requests in k8s to avoid request more than the maximum storage capacity for ALL the cluster?</p>