title: Spark / PySpark - GMM Clustering returning a perfect equiprobability and only 1 cluster
body: <p>I try to apply a GMM clustering algorithm (as in <a href="https://spark.apache.org/docs/latest/ml-clustering.html" rel="nofollow noreferrer">https://spark.apache.org/docs/latest/ml-clustering.html</a>) on a given DataFrame as following :</p>
  
  <pre><code>vector.show(1)
  </code></pre>
  
  <p>ID   | Features</p>
  
  <p>33.0 | [0.0,1.0,27043.0,....]</p>
  
  <pre><code>type(vector)
  </code></pre>
  
  <p>pyspark.sql.dataframe.DataFrame</p>
  
  <pre><code>type(vector.select('features'))
  </code></pre>
  
  <p>pyspark.sql.dataframe.DataFrame</p>
  
  <pre><code>vector.printSchema()
  </code></pre>
  
  <p>root</p>
  
  <p>|-- id: double (nullable = true)</p>
  
  <p>|-- features: vector (nullable = true)</p>
  
  <p>Then I tried the following code to create the clusters :</p>
  
  <pre><code>from pyspark.ml.clustering import GaussianMixture
  gmm = GaussianMixture().setK(5).setSeed(538009335).setFeaturesCol("features")
  gmm_model = gmm.fit(vector)
  gmm_model.gaussiansDF.show()
  gmm_predictions = gmm_model.transform(vector)
  gmm_predictions.show()
  </code></pre>
  
  <p>This works without any bugs or troubles but the algorithm finally returns the same mean and covariance for all clusters and assign every row/ID to the same cluster 0 (probabilities being always 0.2 for whatever cluster ([0.2,0.2,0,2,0.2,0.2])).</p>
  
  <p>Would you know why it gives me such results back please ? </p>
  
  <p>NB : The data are not responsible for this "bad" clustering : having tried Kmeans with Scikit-learn and PySpark, I get a "realistic" clustering with Scikit-learn.</p>
  
  <p>Thank you in advance for your help.</p>
  
  <p>Best regards</p>