title: Create a cluster without exceeding Quotas
body: <p>When trying to create a dataproc cluster, I get the "Quota exceed" error. </p>
  
  <p>ERROR: (gcloud.beta.dataproc.clusters.create) INVALID_ARGUMENT: Insufficient 'DISKS_TOTAL_GB' quota. Requested 3000.0, available 2048.0.</p>
  
  <p>I have changed the machine types and also reduced number of workers to 2. Further, if I specify master and worker boot disk size I get an unrecognized argument error</p>
  
  <p>I am using GCP free tier. I am trying to follow the steps from Google codelab - <a href="https://codelabs.developers.google.com/codelabs/pyspark-bigquery/index.html?index=..%2F..index#5" rel="nofollow noreferrer">https://codelabs.developers.google.com/codelabs/pyspark-bigquery/index.html?index=..%2F..index#5</a></p>
  
  <p>I have enabled three APIs for this GCP project - Compute Engine, Data Proc and Big Query.</p>
  
  <p>I have already set the machine types as below:
  worker-machine-type as n1-standard-2
  master-machine-type as n1-standard-2</p>
  
  <p>First Attempt</p>
  
  <pre><code>gcloud beta dataproc clusters create ${CLUSTER_NAME} \
   --zone=${ZONE} \
   --worker-machine-type n1-standard-8 \
   --num-workers 4 \
   --image-version 1.4-debian9 \
   --initialization-actions gs://dataproc-initialization-actions/python/pip-install.sh \
   --metadata 'PIP_PACKAGES=google-cloud-storage' \
   --optional-components=ANACONDA \
   --enable-component-gateway 
  </code></pre>
  
  <p>Second Attempt: helped remove some quota errors</p>
  
  <pre><code> gcloud beta dataproc clusters create ${CLUSTER_NAME} \
       --zone=${ZONE} \
       --worker-machine-type n1-standard-2 \
       --master-machine-type n1-standard-2 \
       --num-workers 2 \
       --image-version 1.4-debian9 \
       --initialization-actions gs://dataproc-initialization-actions/python/pip-install.sh \
       --metadata 'PIP_PACKAGES=google-cloud-storage' \
       --optional-components=ANACONDA \
       --enable-component-gateway
  </code></pre>
  
  <p>Third Attempt</p>
  
  <pre><code>gcloud beta dataproc clusters create ${CLUSTER_NAME} \
       --zone=${ZONE} \
       --master-machine-type=n1-standard-2 \
       --master-boot-disk-size=500GB \ 
       --worker-machine-type=n1-standard-2 \
       --worker-boot-disk-size=500GB \ 
       --num-workers=2 \
       --image-version=1.4-debian9 \
       --initialization-actions=gs://dataproc-initialization-actions/python/pip-install.sh \
       --metadata='PIP_PACKAGES=google-cloud-storage' \
       --optional-components=ANACONDA \
       --enable-component-gateway
  </code></pre>
  
  <p>I expected the above command to create the cluster instead I get unrecognized arguments message right after the parameter - master-boot-disk-size=500GB (see the error message below).</p>
  
  <p>gcloud beta dataproc clusters create ${CLUSTER_NAME} \</p>
  
  <blockquote>
  <pre><code> --zone=${ZONE} \
   --master-machine-type=n1-standard-2 \
   --master-boot-disk-size=500GB \
  </code></pre>
    
    <p>ERROR: (gcloud.beta.dataproc.clusters.create) unrecognized arguments:</p>
  </blockquote>
  
  <p><a href="https://i.stack.imgur.com/IA5DF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IA5DF.png" alt="enter image description here"></a></p>