I have a dataset with approximately 4000 rows and 150 columns. I want to predict the values of a single column (= target).  The data is on cities (demography, social, economic, ... indicators). A lot of these are highly correlated, so I want to do a PCA - Principal Component Analysis.   The problem is, that ~40% of the values are missing.  My current approach is: Remove target indicator and do PCA with mean/median imputation of missing values. Select x principal components (PC). Append target indicator to these PC. Use PC as predictors for the target variable and try common regression techniques, e.g. knn, linear regression, random forest etc.  With this approach, I'm getting quite good results. My metric is RMSE% - root mean squared relative prediction error. I tried this for all columns in the dataset, the RMSE% is between 0.5% and 8% (depending on the column). These errors are for values I actually know, NOT imputed values.  So, here's my problem: I'm not sure how much my data is distorted by replacing the missing values with the column mean/median. Is there any other way of imputing the missing values with minimal effect on the PCA results?
I used the prcomp() function to perform a PCA (principal component analysis) in R. However, there's a bug in that function such that the na.action parameter does not work. ; two users there offered two different ways of dealing with NA values. However, the problem with both solutions is that when there is an NA value, that row is dropped and not considered in the PCA analysis. My real data set is a matrix of 100 x 100 and I do not want to lose a whole row just because it contains a single NA value.  The following example shows that the prcomp() function does not return any principal components for row 5 as it contains a NA value.  d       &lt;- data.frame(V1 = sample(1:100, 10), V2 = sample(1:100, 10),                        V3 = sample(1:100, 10)) result  &lt;- prcomp(d, center = TRUE, scale = TRUE, na.action = na.omit) result$x                                # $ d$V1[5] &lt;- NA                           # $ result  &lt;- prcomp(~V1+V2, data=d, center = TRUE, scale = TRUE, na.action = na.omit) result$x   I was wondering if I can set the NA values to a specific numerical value when center and scale are set to TRUE so that the prcomp() function works and does not remove rows containing NA's, but also does not influence the outcome of the PCA analysis.  I thought about replacing NA values with the median value across a single column, or with a value very close to 0. However, I am not sure how that influences the PCA analysis.  Can anybody think of a good way of solving that problem?
Why the definition of limit have a strict inequality rather than not? My question is why the definition of limit of sequence is for $\epsilon &gt;0$, there exist $N$ st for $n\ge N$ that $|x_n-L|&lt;\epsilon$ instead of $|x_n-L|\le \epsilon$ Core Data NSPredicate with to-Many Relationship Does the following limit exist? $$\lim_{x \to 0} x\sin\left(\frac{1}{x}\right)$$  If yes, then I have to find it; if no, then I need to give reason why. I couldn’t figure out how to prove this formally, can someone please help me out. "With what [...]?" or "What [...] with?" Making a comparison with Who/Whom I now have a question about the use of what with prepositions in questions. I'll explain by example:  These two sentences are correct, one is more formal than the other:  Who/Whom     Formal: With whom are you doing that? Informal: Who are you doing that with?   But if I substitute Who/Whom with What, the "formal" one sounds a bit awkward and I'm not sure if it's correct or not:  What     With what are you doing that? What are you doing that with?   So, are those last two questions grammatically correct? Is one more formal than the other? On the importance of the i.i.d. assumption in statistical learning Is it possible to skip releases during an upgrade? Possible Duplicate:          Is it possible to upgrade a 10.04 lucid installation to 11.04 Natty, without upgrading to 10.10 first? What about other versions? Where do I use delegates? What are some real world places that call for delegates?  I'm curious what situations or patterns are present where this method is the best solution.  No code required. Propagate all arguments in a bash shell script If $[G' : G'']\leq p^2$, then $G'$ is abelian. Problem : Let $G$ be a p-group and $G'$ denote the commutator subgroup of $G$. If $[G' : G'']\leq p^2$, then $G'$ is abelian.  It is easy to prove it for the case of $[G' : G'']=1$ since G is solvable. But it is too hard to prove the other cases. Please give me any advice or solutions. Thanks in advance. I recently printed a paper that asks to prove the "amazing" claim that for all $a_1,a_2,\dots$  $$\sum_{k=1}^\infty\frac{a_1a_2\cdots a_{k-1}}{(x+a_1)\cdots(x+a_k)}=\frac{1}{x}$$  and thus (probably) that  $$\zeta(3)=\frac{5}{2}\sum_{n=1}^\infty {2n\choose n}^{-1}\frac{(-1)^{n-1}}{n^3}$$  Since the paper gives no information on $a_n$, should it be possible to prove that the relation holds for any "context-reasonable" $a_1$? For example, letting $a_n=1$ gives  $$\sum_{k=1}^\infty\frac{1}{(x+1)^k}=\frac{1}{x}$$  which is true.  The article is How to properly render smoke on a transparent background? When rendering volumes with Film > Transparent enabled, the alpha appears to be the wrong type (premultiplied/straight). In the 3D view it works as expected:    This is a known .  Is there any way to work around this? I would like to create a md5 checksum list for all files in a directory.  I want to cat filename | md5sum &gt; ouptput.txt. I want to do this in 1 step for all files in my directory.   Any assistance would be great. How to calculate polygon areas and perimeters using QGIS? Using QGIS, if I load a polygon layer and make a new column, how do I fill that column with the:   area of each polygon and another column with length of the perimeter of each polygon? Will tcpdump see packets that are being dropped by iptables? Group Membership Test for Permutations Can we have the table name as "option" in MySQL? I am very, very new to MYSQL.I tried to create a table named "option". My SQL Query is :  create table option(   id int not null primary key auto_increment,  choice varchar(30)  )  While executing this query it shows the following error  Error Code : 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'option( id int not null primary key auto_increment, choice varchar(30) )' at line 1 (0 ms taken)  If I try with the table name as "choice" it is working.  can we have the table name as "option" in mysql?   thanks I was having trouble playing the game on my android so I downloaded it to my wife's phone (Galaxy S6). I read some posts on how to get it running on my android and it worked but, my level 7 account is on my wife's phone.  How do I get it back to my android? When I sign out and back in it doesn't ask for any sign in info. I've tried deleting the app's data and even uninstalling and re-installing, but it still doesn't ask for sign-in info. I am unable to understand what exactly is critical value.My question here is why do we fail to reject a null hypothesis when the test statistic is within the range of critical value.  What i am trying to ask is what is the property of the critical value that makes the hypothesized value to fall within the range implied by the critical value.  Thanks in advance How can you tell if if a meal is thawed? I am planning to place a meal (salmon + rice) in a refrigerator after having frozen it. How can I tell when it's thawed? How do I hide Thunderbird from the launcher while running? I have a question for you all.  It's possible to "hide" thunderbird from the unity application bar when thunderbird is running?  For my is unhelpfull see thunderbird on the "mail" tray ( near shutdown button) and the unity "running" application.  If i close thunder with the close button, it don't notify me new email.