i am using nested cross validation in a unbalanced dataset, my question is how how to perform nested cross validation to the feature importance? i should use feature selector in inner loop of cv or out? the features importance found in cv will be the same in the final model?
I have recently been reading a lot on this site (@Aniko, @Dikran Marsupial, @Erik) and elsewhere about the problem of overfitting occuring with cross validation - (Smialowski et al 2010 Bioinformatics, Hastie, Elements of statistical learning).  The suggestion is that any supervised feature selection (using correlation with class labels) performed outside of the model performance estimation using cross validation (or other model estimating method such as bootstrapping) may result in overfitting.   This seems unintuitive to me - surely if you select a feature set and then evaluate your model using only the selected features using cross validation, then you are getting an unbiased estimate of generalized model performance on those features (this assumes the sample under study are representive of the populatation)?  With this procedure one cannot of course claim an optimal feature set but can one report the performance of the selected feature set on unseen data as valid?  I accept that selecting features based on the entire data set may resuts in some data leakage between test and train sets. But if the feature set is static after initial selection, and no other tuning is being done, surely it is valid to report the cross-validated performance metrics?   In my case I have 56 features and 259 cases and so #cases > #features. The features are derived from sensor data.  Apologies if my question seems derivative but this seems an important point to clarify.  Edit: On implementing feature selection within cross validation on the data set detailed above (thanks to the answers below), I can confirm that selecting features prior to cross-validation in this data set introduced a significant bias. This bias/overfitting was greatest when doing so for a 3-class formulation, compared to as 2-class formulation.  I think the fact that I used stepwise regression for feature selection increased this overfitting; for comparison purposes, on a different but related data set I compared a sequential forward feature selection routine performed prior to cross-validation against results I had previously obtained with feature selection within CV. The results between both methods did not differ dramatically. This may mean that stepwise regression is more prone to overfitting than sequential FS or may be a quirk of this data set.
What is $\sqrt{i}$? If $i=\sqrt{-1}$, is $\large\sqrt{i}$ imaginary?     Is it used or considered often in mathematics? How is it notated? Row colour gaps in tabularx with \aboverulesep and \{} \documentclass[]{article} \usepackage[table,dvipsnames]{xcolor} \usepackage{multicol,booktabs,tabularx}  % Table settings \renewcommand{\aboverulesep}{1pt} \renewcommand{\belowrulesep}{1pt}  \begin{document}     \begin{tabularx}{\textwidth}{@{}X l@{}}         This is Header 1 &amp; This is Header 2 \\         \toprule         \rowcolor{Apricot}         This is Text 1 &amp; This is Text 2 \\\midrule         This is Text 1 &amp; This is Text 2 \\\midrule         This is Text 1 &amp; This is Text 2 \\         \bottomrule     \end{tabularx} \end{document}   I have the above table MWE using tabularx, and I am trying to colour the row from the table. However, due to the use of my \aboverulesep and \belowrulesep, I now have gaps above and below my table row that is not coloured.   Also, because of the use of my @{} on either side of the table to 'remove' the excess padding on the sides of the table, the row colour is being applied in those padding as well.   Qn 1: How can I colour the aforementioned gaps (arising due to the \aboverulesep) in, while maintaining the rule separation?   Qn 2: How to remove the colour from the two sides of the table?     EDIT  This is in response to Zarko's answer. I want to maintain the use of @{} on either side to remove the padding. If I were to adapt this into your answer, like so:  % @Zarko's answer \documentclass{article} \usepackage[table,dvipsnames]{xcolor} \usepackage{booktabs, cellspace, tabularx}  % Table settings \renewcommand{\aboverulesep}{0pt} \renewcommand{\belowrulesep}{0pt} \setlength\cellspacetoplimit{5pt} \setlength\cellspacebottomlimit{5pt}  \begin{document}     \begin{tabularx}{\textwidth}{@{}SX Sl@{}}% &lt;-- S is append for activate additional vertical space          This is Header 1 &amp; This is Header 2 \\         \toprule         \rowcolor{Apricot}         This is Text 1 &amp; This is Text 2 \\\midrule         This is Text 1 &amp; This is Text 2 \\\midrule         This is Text 1 &amp; This is Text 2 \\         \bottomrule     \end{tabularx} \end{document}   I obtain this:   and so my second question (Qn2) is still unresolved.. For Qn 1, I prefer to use the solution by @Skillmon with \renewcommand{\arraystretch}{1.15} as I need not add a new package. Can't replace block with a Command Block I am making a Sell Shop, but I need to replace some things using /setblock.  For one of the replacements, I am using the command:  /setblock 579 62 1053 minecraft:command_block 0 replace {Command:"testfor @p {Inventory:[{id:"minecraft:log",Count:32b}]}",TrackOutput:0}  But it won't work! When I try it using a Command Block, I get this message:     [19:14:38]   Data tag parsing failed: Unexpected token 'm' at: minecraft:log",Count:32b}]}",TrackOutput:0   Is there any way I can fix this? This question led to a new package:      I'm trying to make some good-looking numbers in a circle, using the simple command:  \textcircled{1}   However, the circle is misaligned vertically with the number, and look a bit sloppy:    Any suggestions how can I adjust the vertical alignment so that the numbers look more centered w.r.t to the circles? Or, if that's not an option, what other ways to achieve the same results are possible?  I will use those only for numbers, and in very few places, so manual adjustment per glyph is an option.  I'm using pdfLaTeX with Computer Modern, if that matters. Can I update an early Airport Time Capsule? How can I add metadata to NTFS files/folders? I want to tag different file types (i.e. .pdf, .epub, .iso, .bin, folders,..) using the same descriptive fields. For example i would like a metadata field "type" which would be "eBook" on pdf- and epub-files, "CD-Image" on iso- and bin-files.  I read about Alternate Data Streams (ADS) to make this possible. Does anyone know a good program for Windows 7 to tag different files and search for them? It is important for me, that the metadata is NOT stored in a separate database. I move the files a lot and need to stay flexible (ADSs 'stick' to the files).  Any ideas? What is the intuition behind the unit normal vector being the derivative of the unit tangent vector? I've seen the math, but... It just doesn't make sense to me. How is the slope going to point perpendicular to the vector that is clearly a straight line not going in that direction? Which one is correct? (Maybe both are correct.)     He passed away on the morning of March 5.   Or      He passed away in the morning of March 5. Why is rabies incurable? I'm still not sure about the mechanics that lead to rabies being incurable. I know that it can be treated before any symptoms show up, but why is it that once symptoms show the person is a dead man walking? I have installed Google Drive on Windows Server 2008 R2. When I login to the server, Google Drive starts automatically and synchrnoizes files, pulling changes down from the cloud. All good so far.  However, I would like the sync to run even if I am not logged in. Is there any way to start the sync process automatically, maybe as a scheduled task?  I have other jobs on the server which distribute the files by FTP so I would like the server copies of the files to be up to date. I've just started to learn about the tensor product and I want to show: $$(\mathbb{Z}/m\mathbb{Z}) \otimes_\mathbb{Z} (\mathbb{Z} / n \mathbb{Z}) \cong \mathbb{Z}/ \gcd(m,n)\mathbb{Z}.$$  Can you tell me if my proof is right:  $\mathbb{Z}/m\mathbb{Z}$ and $\mathbb{Z} / n \mathbb{Z}$ are both finite free $\mathbb{Z}$-modules with the basis consisting of one single element $\{ 1 \}$. So $(\mathbb{Z}/m\mathbb{Z}) \otimes_\mathbb{Z} (\mathbb{Z} / n \mathbb{Z})$ has the basis $\{ 1 \otimes 1 \}$.  Therefore, any element in $(\mathbb{Z}/m\mathbb{Z}) \otimes_\mathbb{Z} (\mathbb{Z} / n \mathbb{Z})$  is of the form $(ab) 1 \otimes 1$ and any element in $\mathbb{Z}/ \gcd(m,n)\mathbb{Z}$ is of the form $k 1 = k$ where $k \in \{ 0, \dots , \gcd(n,m) \}$.  I would like to construct an isomorphism that maps $ab$ to some $k$. Let this map be $ab (1 \otimes 1) \mapsto ab \bmod \gcd(n,m)$.  This is a homomorphism between modules: it maps $0$ to $0$ because it maps the empty sum to the empty sum. It also fulfills $f(a + b) = f(a) + f(b)$ because there is only one element, $a = 1$.   It is surjective. So all I need to show is that it is injective. But that is clear too because if $ab \equiv 0 \bmod \gcd(m,n)$  then both $a \equiv 0 \bmod n$ and $b \equiv 0 \bmod m$ so the kernel is trivial.  Many thanks for your help!! Prove that ${\sqrt {n} }^{\sqrt {n+1}} > {\sqrt {n+1}}^{\sqrt {n}}$. Wrong numeration in toc longer then one page I tried anything I could find with \cleardoublepage and \newpage and so on; mostly they use \addcontentsline. I don't need to use it as listof=totoc already puts it into the toc so even if I use \addcontentsline instead I get the same result.  \documentclass[     fontsize=11pt,     DIV10,     ngerman,     paper=a4,      twoside=false,     titlepage,     parskip=half,     listof=totoc,     bibliography=totoc,     index=totoc,     captions=tableheading,     foottotes=multiple,     final ]{scrartcl}  \usepackage[     automark,     headsepline,     ilines ]{scrpage2}  \usepackage[ngerman]{babel} \usepackage[babel,german=quotes]{csquotes} \usepackage[dvips]{graphicx}  \begin{document}  \ofoot{\pagemark} \pagenumbering{Roman} \clearpage \tableofcontents \clearpage \listoffigures \clearpage \pagenumbering{arabic} \section{Test} \label{sec:Test}  Test  \subsection{Test} \label{sec:Test}  \begin{figure}[H]  \centering \caption{Testfig} \label{fig:testfig} \end{figure}  \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test} \subsection{Test} \label{sec:Test}  \end{document} Proving that if $A$ is a $8\times 8$ matrix over $\mathbb{R}$ and $A^3=A$, then $A$ is diagonalizable. There was the following sentence in an article titled, “Like, Degrading the Language? No Way” in New York Times (April 5), in which the author says Americans are moving backward on language:     “(Like the use of ‘like’) the use of “totally” mines the same vein.   “He’s totally going to call you” does not mean “He is going to call   you in a total fashion.” It has a more specific meaning, although only   handled subconsciously by speakers, as so much of language is. “He’s   totally going to call you” contains an implication: that someone has   said otherwise or that the chances of it may seem slim at first glance   but in fact aren’t. As with “like,” “totally” tracks and nods to the   opinions of others. It’s totally civilized.”      Although the author explains “totally going to call you” contains an implication of someone having said otherwise, I’m not still clear with the meaning of “He’s totally going to call you.”   What does it mean? He’s totally going to call you “what”?   How does “He’s totally going to call you.” differ from “He’s going to call you.”? Is this expression (use of totally) ubiquitous today as the author asserts? Oneboxing is a handy feature in chat for quickly previewing links, but the list of supported sites is very small, and occasionally boxes for sites such as Wikipedia can break and fetch any image/text.  Instead of having to request additional sites to be added to the onebox list (, , , , , and of course ), we could let the sites themselves handle the oneboxing with some &lt;meta&gt; tags, such as:  &lt;meta name="onebox-title" content="The Onebox Title"/&gt; &lt;meta name="onebox-image" content="/image.png"/&gt; &lt;meta name="onebox-summary" content="This is the summary for a site's onebox"/&gt;   Then when a link is placed on its own in chat, the oneboxing spider would look for these tags and, if they are found, show the appropriate data.  Support for these tags would certainly be bad (for lack of any real better term) at first, though most users of SO chat could easily add it to their personal / company's sites (should they be in such a position) and writing friendly suggestions to other sites to implement them isn't exactly a gargantuan effort.  This would also present an opportunity for currently oneboxed sites to set in stone what should be shown, such as each Wikipedia article setting the main image to be oneboxed instead of SO's spider picking one willy-nilly.  Finally, sites implementing these tags would lay the foundation for other sites and services to develop similar features to the onebox, without having to write per-site scrapers. How do I audit my reputation? If I am concerned that my reputation score is incorrect, how can I audit it, or get a report of a detailed breakdown of my reputation? Convert particle system to animated meshes I want to make someone a present by subscribing for him to a mathematical journal. (It should appear every month or every two months). The contents should be comprehensible to an undergraduate in mathematics, and be about a broad selection of topics (for example not only analysis).   I expect the content to be something like proposing interesting problems and then giving solutions with some references for further study. Are there some famous journals you could suggest me? How to make Table of Contents manually I'm trying to make table of contents manually, I mean I set the page of each of sections one by one. Here's my target:  and I'm using book class here.  Here's the code that I found   \documentclass[oneside]{book} \renewcommand\contentsname{CONTENTS} \renewcommand\listtablename{LIST OF TABLES} \renewcommand\listfigurename{LIST OF FIGURES} \addtocontents{toc}{{\bfseries Contents\hfill Page No.\bigskip\par}} \addtocontents{lot}{{\bfseries No.\hfill Title\hfill\hfill Page No.\bigskip\par}} \addtocontents{lof}{{\bfseries No.\hfill Title\hfill\hfill Page No.\bigskip\par}}  \usepackage[titles]{tocloft} \usepackage{xcolor,xpatch} \renewcommand\cftbeforechapskip{0ex} \renewcommand\cftchapfont{\mdseries} \renewcommand\cftchappagefont{\mdseries} \renewcommand\cftchappresnum{Chapter~} \renewcommand\cftchapaftersnum{. } \newlength\tocindent \settowidth\tocindent{\cftchapfont\cftchappresnum9\cftchapaftersnum} \edef\cftchapnumwidth{\the\tocindent} \edef\cftsecindent{\the\tocindent} \advance\tocindent2.3em \edef\cftsubsecindent{\the\tocindent} \advance\tocindent3.2em \edef\cftsubsubsecindent{\the\tocindent} \renewcommand\cftsecdotsep{\cftnodots} \renewcommand\cftsubsecdotsep{\cftnodots} \renewcommand\cftsubsubsecdotsep{\cftnodots} \newcommand\tocmainmatter {\renewcommand\cftchappagefont{\color{white}}% } \xapptocmd\mainmatter{\addtocontents{toc}{\protect\tocmainmatter}}{}{}   \newif\ifschaptertoc \makeatletter \renewcommand\@makeschapterhead[1]% {{\parindent \z@ \centering     \normalfont     \interlinepenalty\@M     \Large \bfseries  #1\par\nobreak     \vskip 20\p@ }% \@mkboth{\MakeUppercase{#1}}{\MakeUppercase{#1}}% \ifschaptertoc \addcontentsline{toc}{chapter}{#1}% \fi } \makeatother  \usepackage{blindtext}  \begin{document} \frontmatter  \tableofcontents \schaptertoctrue  \chapter*{ACKNOWLEDGEMENTS} \blindtext  \chapter*{ABSTRACT} \blindtext  \listoftables  \listoffigures  \chapter*{ABBREVIATIONS} \Blindtext  \chapter*{NOTATIONS} \Blindtext  \mainmatter  \chapter{INTRODUCTION} \blindtext  \section{General Background} \blindtext  \section{Objective} \blindtext \subsection{Objective 1} \blindtext \subsection{Objective 2} \blindtext \subsection{Objective 3} \blindtext  \section{Scheme} \blindtext \end{document}