I know that one can approximate expectations of a function with respect to a pdf as such $$ \mathbb{E}_{p(x)}[\phi(x)] = \int \phi(x) p(x) dx \approx \frac{1}{N}\sum_{i=1}^N \phi(x^{(i)}) \qquad\qquad \text{where } x^{(i)} \sim p(x) $$ How can I approximate a normalizing constant in a similar way using Monte Carlo? $$ \mathcal{Z} = \int \widetilde{p}(x) dx \approx \,\, ? $$
This is a recurring question (see ,  and ), but I have a different spin.  Suppose I have a bunch of samples from a generic MCMC sampler. For each sample $\theta$, I know the value of the log likelihood $\log f(\textbf{x} | \theta)$ and of the log prior $\log f(\theta)$. If it helps, I also know the value of the log likelihood per data point, $\log f(x_i | \theta)$ (this information helps with certain methods, such as WAIC and PSIS-LOO).  I want to obtain a (crude) estimate of the marginal likelihood, just with the samples that I have, and possibly a few other function evaluations (but without rerunning an ad hoc MCMC).  First of all, let's clear the table. We all know that the harmonic estimator is the . Let's move on. If you are doing Gibbs sampling with priors and posteriors in closed form, you can use ; but I am not sure how to generalize outside of those cases. There are also methods that require you to modify the sampling procedure (such as via ), but I am not interested in that here.  The approach I am thinking of consists of approximating the underlying distribution with a parametric (or nonparametric) shape $g(\theta)$, and then figuring out the normalization constant $Z$ as a 1-D optimization problem (i.e., the $Z$ that minimizes some error between $Z g(\theta)$ and $f(\textbf{x}|\theta) f(\theta)$, evaluated on the samples). In the simplest case, suppose that the posterior is roughly multivariate normal, I can fit $g(\theta)$ as a multivariate normal and get something similar to a Laplace approximation (I might want to use a few additional function evaluations to refine the position of the mode). However, I could use as $g(\theta)$ a more flexible family such as a variational mixture of multivariate $t$ distributions.  I appreciate that this method works only if $Z g(\theta)$ is a reasonable approximation to $f(\textbf{x}|\theta) f(\theta)$, but any reason or cautionary tale of why it would be very unwise to do it? Any reading that you would recommend?  The fully nonparametric approach uses some nonparametric family, such as a Gaussian process (GP), to approximate $\log f(\textbf{x}|\theta) + \log f(\theta)$ (or some other nonlinear transformation thereof, such as the square root), and  to implicitly integrate over the underlying target (see  and ). This seems to be an interesting alternative approach, but analogous in spirit (also, note that GPs would be unwieldy in my case).
Change Data Type Varchar To Varbinary(max) In SQL Server Do you say IN one of the computers or ON one of the computers? Which is right? How to use z-index in svg elements? RSA encryption with private key and decryption with a public key When using the RSA cryptosystem, does it still work if you instead encrypt with the private key and decrypt with the public key? What about in the case of using RSA for sender authentication? There is a prime between $n$ and $n^2$, without Bertrand How to understand the definition of vector and tensor? Physics texts like to define vector as something that transform like a vector and tensor as something that transform like a tensor, which is different from the definition in math books. I am having difficulty understanding this kind of definition. To me, it goes like this:   First we have a collection of bases or coordinate systems (do these represent reference frames?) and the transformations between them. A vector/tensor is an assignment of an array of numbers to each basis, and these arrays are related to each other by coordinate transformations.   I am wondering how coordinate systems and transformations are specified.    Do we need all possible coordinate systems and transformations to define a vector space or just a few of them? How to define the concept of basis and coordinate transformation without completing the very notion of vector? I read (or do I say "have read"?) many rules for when to use the present perfect. I found them complex and hard to understand (or do I say "have found"?). I am finding it hard to apply these rules in real sentences.  Is there a simple and clear way to explain the difference between past simple and present perfect? If there is a simple explanation, please tell it! If there is no simple explanation, please tell why it is so complicated. Maybe if I knew why there is no simple explanation, that would help me understand what work I need to do in order to learn the difference. Is ¬¬(¬¬P → P) provable in intuitionistic logic? Can druids Wild Shape into creatures other than beasts? Under Wild Shape it says "assume the shape of a beast" you have seen before. Does this mean they can only transform into creatures that have the beast type, such as bear, wolf, crocodile, etc.? Or can they turn into things with other types, like dragon, monstrosity, humanoid, fiend, etc.? What do they do with all the plates and cups in Star Trek? How to reset Canon 70D WiFi? Counting surjective functions The following time-frequency correspondence illustration is in .    Also a gif animation version (if the animation doesn't show, please open the following image in a new window):    This picture has so much explanatory power, and I would like to replicate it in TikZ for future use.  Here is what I came up with:  \documentclass{minimal} \usepackage{tikz}  \begin{document} \begin{tikzpicture} [x={(1cm,0.5cm)},z={(0cm,0.5cm)},y={(1cm,-0.2cm)}] \draw[-&gt;,thick,blue!90] (0,6.5,0) -- (6.2,6.5,0) node[right] {Frequency}; \draw[-&gt;,thick,red!90] (0,0,0) -- (0,6.5,0) node[below] {Time}; \draw[-&gt;,thick] (0,0,0) -- (0,0,2) node[above] {Magnitude}; \foreach \x in {0.5,1.5,2.5,3.5,4.5,5.5}{   \draw[blue!50] (\x,0,0)   \foreach \y in {0,0.02,...,6.28}{     -- ({\x},{\y},{sin(\x*\y*(157))/sqrt(2*\x)})   };   \draw[blue!90, thick] (\x,6.5,0) -- (\x,6.5,1/\x); }  \end{tikzpicture} \end{document}   This is the output so far.    I have two questions:   How to produce that red superposed sine wave of all the blue sine waves? I don't know if there is a sum function or I have to use loop yet again? How to make the camera projection in TikZ more similar to the perspective in that wikipedia illustration?   Any suggestion and tweaking of the parameters I used in the sample drawing are welcome as well.  Thanks in advance!    Update 1:  Here is a new version using tikz, more readable than the first one. Yet the superposition of the sine waves are done manually...I still don't know how to use foreach to produce a sum.  \begin{tikzpicture}[x={(1cm,0.5cm)},z={(0cm,0.5cm)},y={(1cm,-0.2cm)}] \draw[-&gt;,thick,black!70] (0,6.5,0) -- (6.2,6.5,0) node[right] {Frequency}; \draw[-&gt;,thick,black!70] (0,0,0) -- (0,6.5,0) node[below right] {Time}; \draw[-&gt;,thick] (0,0,0) -- (0,0,2) node[above] {Magnitude}; \foreach \y in {0.5,1.5,...,5.5}{ \draw [cyan!50, domain=0:2*pi,samples=200,smooth]   plot (\y,\x, {sin(4*\y*\x r)/\y }); \draw[blue, ultra thick] (\y,6.5,0) -- (\y,6.5,1/\y); } \draw [red, thick, domain=0:2*pi,samples=200,smooth]  plot (0,\x, {sin(4*0.5*\x r)/0.5 + sin(4*1.5*\x r)/1.5 + sin(4*2.5*\x r)/2.5 + sin(4*3.5*\x r)/3.5 + sin(4*4.5*\x r)/4.5 + sin(4*5.5*\x r)/5.5} ); \end{tikzpicture}   The result is as follows: I noticed that the reputation from the main site carried over to meta on Gaming (at least the first upvote), I guess that doesn't work the other way around, right?  Is it then impossible to have a meta reputation higher than the site reputation? How to add a class to a given element? I have an element that already has a class: &lt;div class=&quot;someclass&quot;&gt;     &lt;img ... id=&quot;image1&quot; name=&quot;image1&quot; /&gt; &lt;/div&gt;  Now, I want to create a JavaScript function that will add a class to the div (not replace, but add). How can I do that? Silencing "Your disk is almost full" notification After upgrading to macOS Sierra, I'm getting the notification that "Your disk is almost full. Save space by optimizing storage.":    The options appear to be to store my files in iCloud, automatically delete files, or manually delete files:    My problem/irritation is that I have 80GB free of my 440GB volume.  The question: is there a way to silence this notification in a (semi-)permanent fashion, or to change the threshold it uses for the notification? How do I declare and initialize an array in Java? Whether and how to thank a girlfriend or boyfriend in the acknowledgements of thesis? How is it received by the academia if someone thanks to his/her girlfriend/boyfriend in the acknowledgements of the thesis?  I have seen writers including his/her fiancee in the acknowledgements, however I don't know if including an informal relationship in the thesis will be received well.  If it is acceptable, how should I refer to my girlfriend? Do the 46 human chromosomes form a single unbroken DNA helix?  Or is it rather that a human's genome consists of 46 disconnected helices?    If it is the former, does the common numbering scheme for the chromosomes have any correlation to their actual ordering in the one large strand?  If is the latter, is there a convention on how the chromosomes are ordered in genomic datasets?  Also, is there a clear understanding of how sister chromosomes "find" each other in Meiosis I?  Generally, during periods when Mitosis/Meiosis are not occurring,  what's a good physical picture for how the chromosomes are physically arranged (e.g. a bowl of 46 spaghetti noodles, or maybe the sister chromosomes always stay close together, etc)  thanks! Context: I prepare my scientific documents using LaTeX and compile to a PDF. I often need to seek comments on drafts from collaborators who do not use LaTeX. Most of these collaborators use Windows OS. Assume also that the collaborator does not need to edit the document.  They only need to be able to add comments to the document. Most would be familiar with the commenting system in MS Word, for example. I'd like to be able to give the collaborator some clear instructions about what software and system they should use to comment on the draft.  This should involve free software and an easy to use interface.  Question: What is a good strategy for getting comments on draft documents when the collaborator does not know LaTeX?  Initial thoughts: I know Adobe professional allows you to add comments to a PDF.  However, some collaborators don't have this software and it costs money.  I could send the raw LaTeX to the collaborator. However, given all the markup, the collaborator may find LaTeX source a bit mysterious.  UPDATE: After posting I noticed  to a similar question. Along with a number of other good tips (such as printing and getting paper comments), he mentions jarnal, xournal, and gournal as free cross-platform PDF annotators. I'd be curious to know whether experts have found them adequate for the above mentioned purpose and whether any of them are to be preferred. Others mention FoXIt. And yet others discuss the option of exporting to MS Word or Open Office and using the reviewing system within these programs.