I have a set of points which I can fit a Gaussian model on them using Maximum likelihood estimation. but this estimation is weak and I want to improve it.  I want to fit a mixture of Gaussian on these points so I get better results but I don't know how ! I searched the internet for several days and I only found out that if I have multiple Gaussian distribution, adding them together with some weights will result in mixture of Gaussian. But in my example I only have a single distribution and I don't have any other Gaussian to get the mixture model.  How can I fit a mixture of Gaussians on my observed data ?  my data are in 2D space but for simplicity you can solve it for 1D.
I want to implement the EM algorithm manually and then compare it to the results of the normalmixEM of mixtools package. Of course, I would be happy if they both lead to the same results. The main reference is Geoffrey McLachlan (2000), .  I have a mixture density of two Gaussians, in general form, the log-likelihood is given by (McLachlan page 48):      $$ \log L_c(\Psi) = \sum_{i=1}^g \sum_{j=1}^n z_{ij}\{\log \pi_i + \log f_i(y_i;\theta_i)\}. $$ The $z_{ij}$ are $1$, if the observation was from the $i$th component density, otherwise $0$. The $f_i$ is the density of the normal distribution. The $\pi$ is the mixture proportion, so $\pi_1$ is the probability, that an observation is from the first Gaussian distribution and $\pi_2$ is the probability, that an observation is from the second Gaussian distribution.   The E step is now, calculation of the conditional expectation:      $$ Q(\Psi;\Psi^{(0)}) = E_{\Psi(0)}\{\log L_c(|\Psi)|y\}. $$ which leads, after a few derivations to the result (page 49):      \begin{align} \tau_i(y_j;\Psi^{(k)}) &amp;= \frac{\pi_i^{(k)}f_i(y_j;\theta_i^{(k)}}{f(y_j;\Psi^{(k)}}  \\[8pt]                        &amp;= \frac{\pi_i^{(k)}f_i(y_j;\theta_i^{(k)}}{\sum_{h=1}^g \pi_h^{(k)}f_h(y_j;\theta_h^{(k)})} \end{align} in the case of two Gaussians (page 82):      $$ \tau_i(y_j;\Psi) = \frac{\pi_i \phi(y_j;\mu_i,\Sigma_i)}{\sum_{h=1}^g \pi_h\phi(y_j; \mu_h,\Sigma_h)} $$ The M step is now the maximization of Q (page 49):      $$ Q(\Psi;\Psi^{(k)}) = \sum_{i=1}^g\sum_{j=1}^n\tau_i(y_j;\Psi^{(k)})\{\log \pi_i + \log f_i(y_j;\theta_i)\}. $$ This leads to (in the case of two Gaussians) (page 82):      \begin{align} \mu_i^{(k+1)} &amp;= \frac{\sum_{j=1}^n \tau_{ij}^{(k)}y_j}{\sum_{j=1}^n \tau_{ij}^{(k)}}  \\[8pt] \Sigma_i^{(k+1)} &amp;= \frac{\sum_{j=1}^n \tau_{ij}^{(k)}(y_j - \mu_i^{(k+1)})(y_j - \mu_i^{(k+1)})^T}{\sum_{j=1}^n \tau_{ij}^{(k)}} \end{align} and we know that (p. 50)      $$ \pi_i^{(k+1)} = \frac{\sum_{j=1}^n \tau_i(y_j;\Psi^{(k)})}{n}\qquad (i = 1, \ldots, g). $$ We repeat the E, M steps until $L(\Psi^{(k+1)})-L(\Psi^{(k)})$ is small.        I tried to write a R code (data can be found ).    # EM algorithm manually # dat is the data  # initial values pi1       &lt;-  0.5 pi2       &lt;-  0.5 mu1       &lt;- -0.01 mu2       &lt;-  0.01 sigma1    &lt;-  0.01 sigma2    &lt;-  0.02 loglik[1] &lt;-  0 loglik[2] &lt;- sum(pi1*(log(pi1) + log(dnorm(dat,mu1,sigma1)))) +               sum(pi2*(log(pi2) + log(dnorm(dat,mu2,sigma2))))  tau1 &lt;- 0 tau2 &lt;- 0 k    &lt;- 1  # loop while(abs(loglik[k+1]-loglik[k]) &gt;= 0.00001) {    # E step   tau1 &lt;- pi1*dnorm(dat,mean=mu1,sd=sigma1)/(pi1*dnorm(x,mean=mu1,sd=sigma1) +            pi2*dnorm(dat,mean=mu2,sd=sigma2))   tau2 &lt;- pi2*dnorm(dat,mean=mu2,sd=sigma2)/(pi1*dnorm(x,mean=mu1,sd=sigma1) +            pi2*dnorm(dat,mean=mu2,sd=sigma2))    # M step   pi1 &lt;- sum(tau1)/length(dat)   pi2 &lt;- sum(tau2)/length(dat)    mu1 &lt;- sum(tau1*x)/sum(tau1)   mu2 &lt;- sum(tau2*x)/sum(tau2)    sigma1 &lt;- sum(tau1*(x-mu1)^2)/sum(tau1)   sigma2 &lt;- sum(tau2*(x-mu2)^2)/sum(tau2)    loglik[k] &lt;- sum(tau1*(log(pi1) + log(dnorm(x,mu1,sigma1)))) +                 sum(tau2*(log(pi2) + log(dnorm(x,mu2,sigma2))))   k         &lt;- k+1 }   # compare library(mixtools) gm &lt;- normalmixEM(x, k=2, lambda=c(0.5,0.5), mu=c(-0.01,0.01), sigma=c(0.01,0.02)) gm$lambda gm$mu gm$sigma  gm$loglik   The algorithm is not working, since some observations have the likelihood of zero and the log of this is -Inf. Where is my mistake?
Why does the SUID bit have no effect when applied to directories in Linux? I clearly see that I can set it with 'chmod u+s'. But why does it not have "any" effect. What is the reason for implementing a way to SET it but not implementing any effect? What is the integral of 0? I asked a question a couple of days ago. I didn't receive any answers to the question, save for a few comments.  Once the bounty period was up I placed a bounty on the question as the concern at the time was rather urgent.  I got some answers to question eventually -- as I had expected. But requirements have changed since then and the content of the question has no longer become relevant.  What can or should I do with this question? Should I award the bounty anyway, regardless of whether the question still has it's relevance? How is canonicity of derivative works determined for Star Wars? For all types of derivative works, namely:   Movies Novels Comic books Other written works (including RPG materials) Cartoons Video games Derivative movies (such as the infamous Ewok movies and even more infamous )   According to what rules does something become "canon" and who is responsible for the evaluation of a published work? How to write the Fibonacci Sequence? The following code helps me validate an XML file with an XSD schema.  XmlReaderSettings settings = new XmlReaderSettings(); settings.Schemas.Add(null, xsdFilePath); settings.ValidationType = ValidationType.Schema; settings.ValidationEventHandler += new System.Xml.Schema.ValidationEventHandler(settings_ValidationEventHandler); XmlDocument document = new XmlDocument(); document.Load(xmlFilePath); XmlReader rdr = XmlReader.Create(new StringReader(document.InnerXml), settings);  while (rdr.Read()) {  } isValid = true;   The ValidationEventHandler also tells me what the errors are, but doesn't tell me on 'where' or 'on which line' they are located. Is there any way to get the line number where the XML fails to be validated? U.s. boiler / Burnham boiler alterations i have a 203ncl-tei2 boiler installed in danvers Massachusetts. Regular elevation. This boiler also has a failed tejurland (universal control) powerventer motor on it that is only humming, not spinning. Looked for a dead animal, or nest and checked for free spinning of the squirrel cage it was fine. Its installed 3 feet from outside wall on the verticle, (installed wrong also has 2 seperate 90° bends after the venter, not my issue but may have caused the motor to fail). Ontop of the boiler we have a factory automatic damper assy.   So the 2nd part of the problem is I noticed the tei in the model number and assumed this ment i should expect to see a tejurland "universal" power venter installed and i do. I assume because its a universal control unit thats where they went wrong. This unit is atleast 200$ cheaper here than the model burnham uses.  I call burnham and ask what the original power venter model number is and they tell me what some pros may already know, (i didnt), that the tei2 is actually (t)taco, (el)elivation, (2) 2 is 2,000 ft  .Note:he was not.100% sure about the el. He was definetly sure PV would be in the model number and it was not tejurland. There is no chimney on this house, never was. Power venter wall thimble outside is very old definetly older than the 1 year old motor that failed. Burnham also said this is not a boiler that can be converted to a powerventer which is sometging i do not understand. He said if it does not have pv in the model number it should not have one. I have in the past converted a boiler with a factory reccomended power venter after a customer lost the chimney. I called and got lucky they had a option for it. I do not remember manufacturer but do remember also seeing it in the install manual as an addendum or seperate manual.  Is this a fact?   Anyone have experience with this?   What are the ramifications of an elevated boiler installed at sea level?   If i can not find another awnser, i will havemto tell customer they need a new boiler or i will have to walk away.   Why do they not just make an orifice kit for elevated boilers to work properly at higher elevations? Is it because they have to add another section to the boiler so it can put out the same  b.t.u.?  I know there is a reason. Seems like smaller orifice would fix the air fuel ratio along with a regulator adjustment. Its to simple of an awnser knowing that manufacturers make a complete different boiler for higher elevations.   ! xcode app crashing on launch with breakpoint Express $x^8-x$ as a product of irreducibles in $\Bbb Z_2[x]$ Express $x^8-x$ as a product of irreducibles in $\Bbb Z_2[x]$.  Work so far:  $$x^8 - x = x(x^7 - 1) = x(x - 1)(x^6+x^5+x^4+x^3+x^2+x+1)$$  From here, I think the Zeros of an Irreducible over a splitting field Theorem would be the method. I am unsure how to proceed The Time That 2 Masses Will Collide Due To Newtonian Gravity My friend and I have been wracking our heads with this one for the past 3 hours... We have 2 point masses, $m$ and $M$ in a perfect world separated by radius r.  Starting from rest, they both begin to accelerate towards each other. So we have the gravitational force between them as:  $$F_g ~=~ G\frac{Mm}{r^2}$$  How do we find out at what time they will collide?  What we're having trouble with is this function being a function of $r$, but I have suspected it as actually a function of $t$ due to the units of $G$ being $N(m/kg)^2$.  I've tried taking a number of integrals, which haven't really yielded anything useful.  Any guidance?  No, this is not an actual homework problem, we're just 2 math/physics/computer people who are very bored at work :) How SELECT column does not list in GROUP BY clause The query:  select year,sum(mark),sum(maxmark)  from table1  group by parameterno   yields an error message:     Column 'table1.year' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause. Problem with a sequence with multiple integrals How far can/should one press philosophical doubt? Should we keep on questioning until nothing is left to question or is there a point on which we need to stand (which we often tend to do)? Descartes used 'I think' as this fixed point where the skepticism abates, there may be others. But what is a rational way to find one, if any? How is this question addressed in modern philosophy? Setting up a bicycle for someone with restricted knee articulation My father, now in his 70's, had a left knee replacement several years ago. He has limited movement in that knee - probably about 85 degrees on a good day. His knee can extend fully, but cannot bend as much as it should. As a result, he cannot pedal a bike he cannot get his left leg over the top of the pedal stroke.   He is desperate to get back onto a bike, he knows he won't be ripping up the road and winning races like the old days, but wants the freedom to go for a slow easy rides.    Does anyone have suggestions as to a solution to the problem. We don't mind spending money if we know the solution will work, but he can't afford to throw big money at wild ideas that may or may not work. If we find a solution, a new bike is likely.   We have access several mountain bikes in a range of sizes to trail ideas, and we have all the tools and skills to build bikes up from parts and rearrange things as needed.    EDIT: Thanks everyone - all great suggestions. What am am going to do is got tto the LBS and get hold of a cheap set of short cranks for around $30- probably from a kids bike, in the 120-150 size range, and see if he can ride that. Depending how he goes, I'll try a standard right crank and a short left crank. If all else fails, we will go for the Power Cranks   UPDATE: In case anyone else has similar problems. Got some 152mm cranks for $30. Set an old  mountain bike up with the seat as far back as possible. The old man is able to ride reasonably comfortably, now has a smile from ear to ear - any suggestions how to remove it - doesn't fit with the grumpy old man image :)   Has got me wondering how many people with minor knee problems would benefit from short(er) cranks....... Is it proper to use "the" before the name of a government organization? What is the relationship between a chi squared test and test of equal proportions? Suppose that I have three populations with four, mutually exclusive characteristics.  I take random samples from each population and construct a crosstab or frequency table for the characteristics that I am measuring.  Am I correct in saying that:   If I wanted to test whether there is any relationship between the populations and the characteristics (e.g. whether one population has a higher frequency of one of the characteristics), I should run a chi-squared test and see whether the result is significant. If the chi-squared test is significant, it only shows me that there is some relationship between the populations and characteristics, but not how they are related. Furthermore, not all of the characteristics need to be related to the population.  For example, if the different populations have significantly different distributions of characteristics A and B, but not of C and D, then the chi-squared test may still come back as significant. If I wanted to measure whether or not a specific characteristic is affected by the population, then I can run a test for equal proportions (I have seen this called a z-test, or as prop.test() in R) on just that characteristic.   In other words, is it appropriate to use the prop.test() to more accurately determine the nature of a relationship between two sets of categories when the chi-squared test says that there is a significant relationship? CS0120: An object reference is required for the nonstatic field, method, or property 'foo' Do you really want to create a new tag? Probability distribution in the coupon collector's problem I'm trying to solve the well known Coupon Collector's Problem by explicitly finding the probability distribution (so far all the methods I read involve using some sort of trick). However, I'm not having much luck getting anywhere as combinatorics is not something I'm particularly good at.   The Coupon Collector's Problem is stated as:  There are $m$ different kinds of coupons to be collected from boxes. Assuming each type of coupon is equally likely to be found per box, what's the expected amount of boxes one has to buy to collect all types of coupons?  What I'm attempting:  Let $N$ be the random variable associated with the number of boxes one has to buy to find all coupons. Then $P(N=n)=\frac{|A_n|}{|\Omega _n|}$, where $A_n$ is the set of all outcomes such that all types of coupons are observed in $n$ buys, and $\Omega _n$ is the set of all the possible outcomes in $n$ buys. I think $|\Omega _n| = m^n$, but I'm not even sure about that anymore, as all my attempts so far led to garbage probabilities that either diverged or didn't sum up to 1. If you make a search for something that returns more than 10,000 items in result. For example, search for  return 161,338 results on MSE. But if you select a page where 10,000+ result should be shown:        In the edge case  shows zero results:        But this is actual for any page where per_page * page_number is more than 10,000.  Though this does not happen for Stack Overflow in English. It looks like all other sites are affected.  Related bug report on ruSO.meta: