I'm building a logistic regression model to predict if a patient has cancer based on 9 features. Having built learning curves and increasing my regularization parameter to reduce over fitting, which caused my accuracy to increase from 82% to 86%, I'm stuck on options of how to improve it further.  I'm using a training set of 69 samples, a validation set of 24 samples and a test set of 24 samples. (60%,20%,20% split) (I'm doing this on octave)  When I drew my initial learning curve, I saw that the validation error is actually lower than the training error. Then according to  , as the issue lies in overfitting, I increased the regularization parameter (lambda), which caused the learning curve to show characteristics of high bias, and the accuracy on the Test Set increased to 86%.   Learning curve code which causes validation error to be less than training error;  %costlogreg is a function to calculate logistic regression cost %linearRegcost is a function to calculate the training cost and validation cost %sigmoid2 is just a simple logistic function %listJ is a matrix of zeros  listJ = zeros(69,3);  for i = 1:69  initial_theta = zeros(size(Xadjust,2) + 1,1); %Xtrain, Ytrain is the training set Xtrain = Xadjust([1:i],:); Ytrain = Y([1:i],:);  %function handle for costlogreg costfunction = @(t)costlogreg(t,Xtrain,Ytrain,lambda);  %obtains theta depending on training set options = optimset('Gradobj','on','MaxIter',50); [theta] = fmincg(costfunction, initial_theta,options);  %calculates the cost on the training set and adds it to a list [Jtrain,~] = linearRegCost(theta,Xtrain,Ytrain); listJ(i,1) = i; listJ(i,2) = Jtrain;  %obtains Jcv using earlier found theta [Jval,~] = linearRegCost(theta,Xval,Yval); listJ(i,3) = Jval;  end   Code to choose the best lambda value which gives higher accuracy on test set  %lambda set is a set of regularization parameters to test %listlambda is a matrix of zeros to collect the accuracy on test set for each lambda used listlambda = zeros(size(lambdaset,2),4);  for i = 1:size(lambdaset,2)  initial_theta = zeros(size(Xadjust,2) + 1,1); costfunction = @(t)costlogreg(t,Xtrain,Ytrain,lambdaset(1,i)); options = optimset('Gradobj','on','MaxIter',50); [theta] = fmincg(costfunction, initial_theta,options); [Jtrain,~] = linearRegCost(theta,Xtrain,Ytrain); listlambda(i,1) = lambdaset(1,i); listlambda(i,2) = Jtrain; [Jval,~] = linearRegCost(theta,Xval,Yval); listlambda(i,3) = Jval; [h] = sigmoid2(Xtest,theta); list = zeros(size(h,1),1); idx = find(h &gt;= 0.5); list(idx) = 1; accuracy = mean(double(list == Ytest)); listlambda(i,4) = accuracy; endfor  %figures the row with the highest accuracy on the test set [~,maxidx] = (max(listlambda(:,4))); bestlambda = listlambda(maxidx,1);
Imagine a standard machine-learning scenario:     You are confronted with a large multivariate dataset and you have a   pretty blurry understanding of it. What you need to do is to make   predictions about some variable based on what you have. As usual, you   clean the data, look at descriptive statistics, run some models,   cross-validate them etc., but after several attempts, going back and   forth and trying multiple models nothing seems to work and your   results are miserable. You can spend hours, days, or weeks on such a   problem...   The question is: when to stop? How do you know that your data actually is hopeless and all the fancy models wouldn't do you any more good than predicting the average outcome for all cases or some other trivial solution?  Of course, this is a forecastability issue, but as far as I know, it is hard to assess forecastability for multivariate data before trying something on it. Or am I wrong?   Disclaimer: this question was inspired by this one  that did not attract much attention. It would be nice to have detailed answer to such question for reference.
Can I paste plain text by default? I regularly copy and paste text between spreadsheets, emails, browser windows, etc. I can't think of a single time when I've wanted to keep the formatting from the source text.  I already know about the following workarounds:   Use “Edit” → “Paste Special” → “Unformatted text” Paste to Notepad and copy from there Install a program that takes over the clipboard   What I want is to tell Windows to just do this by default.  Is this possible? Would real Moody have taught 4th year students about Unforgiveable curses? Would real Moody have taught 4th year students about Unforgiveable curses the way fake Barty Crouch Moody did?  In other words, did Barty teach about them (leaving aside his hands-on teaching approach) just to satisfy his Death Eater wishes, or did he teach them as part of his cover as Moody impersonator because Moody would have taught about them? I started thinking about the number of associative (binary) operations on a set with $n$ elements today.  Looking online I found  which indicates only $113$ of the possible $19,683$ operations on a three-element set satisfy association.  So, $50\%$ of binary operations on a two-element set satisfy association, and less than $0.6\%$ of all binary operations on a three element set satisfy association.  For an $n$-element set, where $n$ denotes a natural number, is the ratio $R_{n}=A_{n}/B_{n}$, of the number of associative binary operations $A_{n}$ to the number of all binary operations $B_{n}$, in general small?  I don't mean the following questions as equivalent, but since they seem more concretely answerable in principle, does   $$ \lim_{n \to +\infty} \frac{A_{n}}{B_{n}} = 0 ? $$       Also, is $F : n \to R_{n}$ a monotonically decreasing function?  Some Background of the Question: In his Linear Algebra Problem Book 1995 on p. 6 Paul Halmos writes "The commonly accepted attitudes toward the commutative law and the associative law are different.  Many real life operations fail to commute; the mathematical community has learned to live with that fact and even to enjoy it.  Violations of the associative law, on the other hand, are usually considered by specialists only."  For all I know, Halmos might only have written that to motivate the study of Linear Algebra and doesn't quite literally mean what he appears to say.  But, if he means what he appears to say, and if $F : n \to R_{n}$ is monotonically decreasing, or $R_{n}$ is generally small, I think there's something amiss with what Halmos says, since non-associative operations seem so common that one may as well enjoy them. Is there is any reason to make the permissions on an overridden C++ virtual function different from the base class?  Is there any danger in doing so?  For example:  class base {     public:         virtual int foo(double) = 0; }  class child : public base {     private:         virtual int foo(double); }   The  says that it is a bad idea, but doesn't say why.  I have seen this idiom in some code and I believe that the author was attempting to make the class final, based on an assumption that it is not possible to override a private member function.  However,  shows an example of overriding private functions.  Of course  recommends against doing so.  My concrete questions:   Is there any technical problem with using a different permission for virtual methods in derived classes vs base class? Is there any legitimate reason to do so? Solving cubic equations with another method. I want to solve cubic equations with some pattern or formula. My maths teacher told me that if we want to solve cubic equations we have to find any one value for the equation which satisfy it and then divide the main equation by it. But I find that method little useless . Is there any other simplier or good method to solve it. You can explain it by an example. Among any three consecutive positive integers one is a multiple of 3 How to distinguish between American Indians and Indian Indians in native English (language) parlance? How to distinguish between American Indians and Indian Indians in native English (language) parlance?  Can I say Indian Indian to say Indian from Asia compared to the Native Americans? How to solve $x^x=100$? $x^x = 100$.   I have no clue on how to solve this. If you guys have, please show me your solution as well. I have two Date values. I need to find number of weeks between them. I want to do it Apex Class. Not looking for formula fields.  So far I tried:  Integer numberofweeks = (projectStartDate - opp.Project_Start_Date__c)/7;    I receive this error:     Date arithmetic expressions must use Integer or Long arguments How does commands like ls know what its stdout is?  It seems ls is operating different depending on what the target stdout is. For example if I do:  ls /home/matt/tmp    the result is:  a.txt b.txt c.txt   However if I do  ls /home/matt/tmp | cat   the result is (i.e. new line per result):  a.txt b.txt c.txt   The process is passed a file descriptor 1 for stdout right? How does it determine how to format the result? Does the file descriptor reveal information? Shortcuts in Objective-C to concatenate NSStrings Are there any shortcuts to (stringByAppendingString:) string concatenation in Objective-C, or shortcuts for working with NSString in general?  For example, I'd like to make:  NSString *myString = @"This"; NSString *test = [myString stringByAppendingString:@" is just a test"];   something more like:  string myString = "This"; string test = myString + " is just a test"; Make a table span multiple pages I have a table that takes up a full page, but I want it to start on a page that already has some text and then continue onto the next page (i.e. it's ok if the table gets broken up and spans two pages). It is a simple two column table:  \begin{table}[h] \centering \begin{tabular}{| p{.20\textwidth} | p{.80\textwidth} |}  \hline foo &amp; bar \\ \hline  foo &amp; bar \\ \hline  ... \end{tabular} \end{table}   I guess I need longtable? How do I use it? In linear regression, each predicted value is assumed to have been picked from a normal distribution of possible values. See below.  But why is each predicted value assumed to have come from a normal distribution? How does linear regression use this assumption? What if possible values are not normally distributed? I found the following question about compactness:     A set $S\subseteq \mathbb{R}^n$ has the following property:   if a sequence of $S$ has exactly one accumulation point in $S$, then it converges in $S$. Prove that $S$ is compact.    We know that in $\mathbb{R}^n$ compact sets correspond to closed and bounded sets.   (1) I am trying to prove that $S$ is closed. The condition in the question says if a sequence of $S$ has an accumulation point in $S$ then it converges in $S$. I think instead the condition should be if a sequence of $S$ has an accumulation point( in $\mathbb{R}^n$, not necessarily in $S$ )then it converges in $S$.  Am I correct?  (2) How to prove that $S$ is bounded?  P.S. I have made an edit (there is exactly one accumulation point).  Source of the question:  of this problem set. I know that the usual ordering of $\mathbb R$ is not a well-ordering but is there an uncountable $S\subset \mathbb R$ such that S is well-ordered by $&lt;_\mathbb R$?   Intuitively I'd say there is no such set but intuitively I'd also say there is no well-ordered uncountable set at all, which is obviously wrong. I still struggle to grasp the idea of an uncountable, well-ordered set. Where should I start in tracking down Firefox malware How to automate entry of external column data in BCS? I have a document library, Resumes, that has a large number of resumes in it. There is a regular column on that library, called EmployeeID, that is already populated.  I have an external content type ( ECT ) that is mapped to a table that lists EmployeeID and several other columns of information. I added this ECT to the library as an External Column.    I am looking for a good way to automatically go through all items in the document library, and copy the value from the "regular" EmployeeID column into the External EmployeeID column and have it update the other columns from the ECT. Any techniques are useful (PowerShell, Object Model, Workflow?) When we have tried this in the past with workflows the setting of the External Column did not report failure, but didn't actually trigger a lookup of the other data and the value didn't seem to actually get saved.  [Update] I have been working on this and am getting closer. I am using PowerShell to access the list and set the external column ID field to the value I want it to be.  The problem is that it doesn't seem to be triggering a lookup of the external data when that field is set.  my "list" (a document library) has a few normal fields and a set of external columns mapped to an ECT "EmplData".  When you use the GUI to add or edit the properties of an item, it works. When I use PowerShell to do the following:  $spList = $spWeb.Lists["Resumes"] $spListItem = $spList.GetItemById(1) $spListItem["EmplData_ID"] = "00300"    $spListItem.Update()   Sometime the external ID field gets updated when I the refresh the web page, other times it says "(blank)". In all cases it does not load in the other fields.  Is there some Update method that updates the rest of the elements?  I gather from research that these are called "secondary fields", is the method to update the secondary fields based on the external item ID available in PowerShell cmdlets someplace?  [Update] I am finding that if I use:  $spListItem["EmplData"] = "000300"   instead of $spListItem["EmplData_ID"] = "00300" it gets around the problem of the column showing "(blank)".   I still need to figure how to make it update the other external columns (also known as "secondary columns" I think) Implementing the decorator pattern in Python Error message on Terminal launch This is a follow-up to this question:   Hans Bethe wrote a paper in 1944, "Theory of Diffraction by Small Holes," Phys. Rev. 66, 163. I don't have access to the paper, but from descriptions online it sounds like he proved the following. Suppose a plane wave impinges on an absorbing sheet, and there is a hole in the sheet of diameter $d$, with $d&lt;\lambda$. Let $P_0$ be the power incident on the hole, and $P$ the power diffracted through the hole. Then the transmission is $T=P/P_0=(d/\lambda)^4$.  As a practical application, I think this explains why microwaves don't leak strongly through the metal grille in the front of a microwave oven.  Questions:   Does the thickness of the sheet matter? From references online, it sounds like the hole is treated as a waveguide...? This seems to relate to cutoff frequencies of waveguides, etc.? It seems to me that Huygens' principle would give $T=1$ for $d\ll\lambda$, since in this limit the wavelets are in phase. Why is Huygens' principle invalid here? Is this related to question #1? Is there a simple argument for the proportionality to $(d/\lambda)^4$? Or if not, how does one prove this using the gory details of Bessel functions, etc.?