It is already common to do something "like"**(see asterisks below) z-standardization of the outputs of one neural network layer before passing it to the next.  z-standardization would transform the columns of $H_{\ell}W_{\ell} + \beta_{\ell}$ (where $\ell$ denotes a layer and $H$ denotes a "hidden" matrix containing the values of hidden neurons, or the input data) to have 0 mean and unit standard deviation.    **In reality, batchnorm is used, which incorporates learnable weights to the standardization function to help the model "undo" or "modify" the deterministic nature of z-scaling.  (1) I observe that z-scoring is a nonlinear function in $W_{\ell}$, because we must compute the sample standard deviation of $H_{\ell}W_{\ell} + \beta_{\ell}$, which involves a square root.  It follows that batch-norm will be nonlinear in the previous layer's weights as well.  (2) Thus, if we do batch norm, we do not need to use a common activation function such as ReLU or tanh to prevent the whole "stack" (composition) of affine layers from collapsing into one affine layer.  Meanwhile, the community generally believes batch norm is "good".  Thus, why not just use batch norm between layers and free ourselves from choosing between ReLU, ELU, etc. etc.?  ^^ that is the purpose of my question  what follows is an observation of why my question might also be useful:  (3) Then, we may observe all the questions online about whether batch norm should be used before or after the activation function.  But couldn't we just use batch norm and avoid this question?  Insights and/or references on this specific topic would be much appreciated!  Thanks
I read this great answer about how ReLu could approximate non-linear functions. However, if ReLu can approximate such functions, why cant linear activation function do the same? If they can, why linear activation functions are strictly prohibited in NN?
I am currently learning how to use TikZ/PGF to make some figures for a project report, and so I have a question:     Is there is a practical way of making a particular object into a node?   The object I have in mind is shown in the Figure below. I want the object to consist in the big circle, the two circular arcs, the label and anchors at north, south, east and west. The dotted lines are simply there to indicate that I am going to have paths going out of these points, which is why I need the anchors there.     I include the source, in the case that it is useful to anyone.  \begin{tikzpicture} \draw[help lines, gray, dotted] (-2,-2) grid (2,2);  \begin{scope}[shift={ (0,0) } ]     \begin{scope}[rotate=90]         \draw[thick] (-1,0) arc[start angle=-90, delta angle=90, %                               x radius=1, y radius=1] -- (0,1) %NW                      (0,-1) arc[start angle=180, delta angle=-90, %                                    x radius=1, y radius=1] -- (1,0); %SE         \draw[gray] (0,0) circle (1);     \end{scope} % Anchor replacements     \node(N1)       at (0,1)    {};     \node(S1)       at (0,-1)   {};     \node(E1)       at (1,0)    {};     \node(W1)       at (-1,0)   {}; % Label     \node(Crossing number) at (0,0) {1}; \end{scope}  \path[thick, gray, dotted] [ out=90,   in=270]     (N1.center) edge (0,2) [ out=180,  in=0]       (W1.center) edge (-2,0) [ out=270,  in=90]      (S1.center) edge (0,-2) [ out=360,  in=180]     (E1.center) edge (2,0); \end{tikzpicture}   PS. I have had a look at the Manual, but I could not find a description of how to do this. Why was Arcanine with the Legendary Birds in Veridian City in Pokémon Origins? In Pokémon Origins, Red sees a image with the three legendary birds and Arcanine.     Why was Arcanine there? I mean, he's 'The Legendary Pokemon' but not a legendary Pokémon. In , there is a comment by  on the answer that Arcanine was originally a legendary Pokémon:     "...Nintendo first gave Zapdos, Articuno and Arcanine as the legendary trio. But they removed Arcanine as it would have been awkward. Two birds and a dog, huh!..."   This still does not add up because Pokémon Origins was created a while after they removed Arcanine. Any thoughts? Prove that, for $p > 3$, $(\frac 3p) = 1$ when $p \equiv 1,11 \pmod{12}$ and $(\frac 3p) = -1$ when $p \equiv 5,7 \pmod{12}$ $(\frac 3p)$ is the Legendre symbol here, not a fraction, if that wasn't clear.  This is what I have so far:  We know by Gauss's Lemma that $(\frac qp) = (-1)^v$ where $$v = \#\left\{1 \le a \le \frac{p-1}{2} \mid aq \equiv k_a \pmod{12}),\space -p/2 &lt; k_a &lt; 0\right\}$$  And we also know that multiples of $(\dfrac{p-1}{2}) \times 3$ must be between $0$ and $p/2$, between $p/2$ and $p$, or between $p$ and $3p/2$.   Representing $p$ as $12k + r$ seems like the next step, but I'm not sure where to go from there. Show that $d_2$ defined by $d_2(x,y)=\frac{|x-y|}{1+|x-y|}$ is a metric I am pretty sure that this is such a stupid, stupid question, but how do you prove that $d_2(x,y)={|x-y|\over {1+|x-y|}}$ satisfying the third condition to be a metric, which is the triangle inequality.   I know that I should really be able to do this, but I just can't, I keep seeing a contradiction. Please help me. Bisecting K-means using Dynamic Time Warping I'm trying to cluster time series of different length and I came up to an idea to use DTW as a similarity measure, which seems to be adequate, but the thing is, I cannot use it with K-means, since it's hard to define centroids based on time series which can have different length/phase. So I was thinking about Hierarchical clustering, since it seems appropriate to combine with DTW, but it's not scalable. So my next thought is to try with bisecting k-means that seems scalable, since it is based on K-means step repetitions. My idea is next, by steps:   Take two signals as initial centroids (maybe two signals that have smallest similarity, calculated using DTW) Assign all signals to two initial centroids Repeat the procedure on the biggest cluster   In this way I could use DTW as distance measure, that could be useful since my data may be shifted, skewed, and avoid calculating centroids. At the end I could take one signal from each cluster that is the most similar with others in cluster (some kind of centroid/medioid).  What do you think about this approach and about the scalability? PC started having screen glitches after update I'm not sure what's caused it. I haven't changed any system settings. The only thing I've done that may have broken something is updating my software. My update was: upgrade linux-generic-hwe-20.04:amd64 5.8.0.48.54~20.04.32 5.8.0.49.55~20.04.33 upgrade linux-image-generic-hwe-20.04:amd64 5.8.0.48.54~20.04.32  upgrade linux-headers-generic-hwe-20.04:amd64 5.8.0.48.54~20.04.32  upgrade linux-image-extra-virtual:amd64 5.4.0.70.73 5.4.0.71.74 upgrade linux-image-generic:amd64 5.4.0.70.73 5.4.0.71.74 upgrade linux-libc-dev:amd64 5.4.0-70.78 5.4.0-71.79 upgrade libseccomp2:amd64 2.4.3-1ubuntu3.20.04.3 2.5.1-1ubuntu1~20.04.1 upgrade nomachine:amd64 7.3.2-3 7.4.1-3```  Removing nomachine did not solve the problem. It was far worse before, I'm not sure how to describe exactly, but my entire screen's UI was glitching out. So I rebooted and that stopped, but now the terminal text looks like this:  I haven't changed any terminal settings. This seems to be occurring in the entire OS. Here is another example:  Not sure how to go about troubleshooting this? I don't have the time to reinstall the OS, nor do I think I should have to because this is just ridiculous and shouldn't happen. I'm not aware of anything I've done to make the system unstable. Edit Earlier I also tried booting the PC. It failed and got stuck on some command in a terminal (didn't manage to take a photo of what it said). So I rebooted into recovery mode with both available kernels. I was getting strange glitches like this:  That didn't work. So I rebooted the PC normally, and now I somehow logged in fine. Now I'm getting popups saying to report a system error. It says this:  Environment  Ubuntu 20.04.2 LTS GNOME 3.36.8  sudo lshw -class display   *-display        description: VGA compatible controller        product: Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller        vendor: Intel Corporation        physical id: 2        bus info: pci@0000:00:02.0        version: 06        width: 64 bits        clock: 33MHz        capabilities: msi pm vga_controller bus_master cap_list rom        configuration: driver=i915 latency=0        resources: irq:33 memory:f7800000-f7bfffff memory:e0000000-efffffff ioport:f000(size=64) memory:c0000-dffff  I also did , which passed with 0 errors. Let me know if more details are required. How to include audio file in PDF What is the module/method used to get the current time? I'm trying to change the numbers in problems I write for examples I use in my class as well as assignments. However the problems are still inspired by the textbook. Are the problems sufficiently different from the textbook problems to not have copyright issues? There is also an issue of theorems and definitions. Authors of the calculus textbook certainly did not come up with these things. Is it OK to just copy them down in my lecture notes?  By the way I do share my lecture notes on Piazza and the course I'm teaching is multivariable calculus. What resolution would be needed for a billboard advertisement? Are there systems in classical mechanics that are provably non-deterministic? Solving functional differential equation $f'(x)=2f(2x)-f(x)$ Show that there is at least a nonzero function $f$, differentiable on $[0,+\infty)$, satisfying $$f'(x)=2f(2x)-f(x) \qquad \forall x&gt;0 $$   $$M_n:=\int_{0}^{\infty}x^nf(x)dx&lt;\infty \qquad \forall n\in  \mathbb{N} $$   My best idea so far is to assume that the solution is a power series, i.e.  $$ f(x)=\sum_{n=0}^{\infty}a_nx^n\qquad \forall x&gt;0$$ Then the equation becomes $$ \sum_{n=0}^{\infty}na_nx^{n-1}=2\sum_{n=0}^{\infty}a_n2^nx^n-\sum_{n=0}^{\infty}a_nx^n$$ equating all the coefficients of the same degree I get $$na_n=(2^{n}-1)a_{n-1}\qquad \forall n\geq 1$$ So setting $a_0=1$, I get $$a_{n}=\frac{1}{n!}\prod_{k=1}^{n}(2^k-1) \qquad \forall n$$ But does the power series actually converge? Using Hadamard's formula, and that $2^{k}-1\geq 2^{k-1}$, $$ |a_n|^{1/n}\geq\frac{1}{(n!)^{1/n}}\left[2^{n(n-1)/2}\right]^{1/n}\sim\frac{e}{n(2\pi n)^{1/2n}}2^{(n-1)/2}\to \infty$$ so the radius of converge of the series is $0$, so it doesn't actually define a solution on $[0,+\infty)$. Some compound words are written without hyphens (nonaggression, nonbeliever), some with hyphens (well-intentioned), and others with spaces (post office).  Is there a rule or good guide as to which option should be used? Is $7$ the only prime followed by a cube? Examples of a non-commutative division ring Let $f(z)$ be an entire function such that $$|f(z)|&lt;\frac{1}{|\text{Im}(z)|},\qquad z\in\Bbb C-\Bbb R.$$ The question asked me to prove that $f(z)=0$. At least looking at it, it really seems to have an application of Liouville's theorem lurking around somewhere, but I haven't found it.   My thoughts first led me to think about doing this by contradiction and using Picard's little theorem. So, I've considered a strip containing the real axis (say of width $2$ for simplicity). This will imply the complement maps into the unit disk, which implies that this strip has to map almost everywhere else on the complex plane. Now, on the imaginary axis, I know that $f(z)$ will vanish as $z\to\infty$, and this seems like it might be useful. I'm sort of under the impression $f(z)$ might have an essential singularity at $\infty$, which would mean $f(1/z)$ has an essential singularity at $0$. Either way, it of course can't have a pole at $\infty$ because of $f(z)$ vanishing on the imaginary axis, and if it is a removable singularity, it must be $0$, which still gives the solution. Therefore, I think it must have something to do with $f(z)$ having an essential singularity at $\infty$.   I was hoping to combine this with the above inequality and deduce a contradiction, but I haven't thought of one yet.   Any hints or suggestions? I'm studying for a prelim. exam, so this isn't homework.  Update: So, in the spirit of searching methods using the essential singularity at $z=0$, we have $$\left|f\left(\frac{1}{z}\right)\right|\leq\frac{1}{\left|\text{Im}\left(\frac{1}{z}\right)\right|}=\frac{|z|^2}{|y|}.$$ Thus, for $|y|&gt;1$, we have $$\left|f\left(\frac{1}{z}\right)\right|&lt;|y|\left|f\left(\frac{1}{z}\right)\right|&lt;|z|^2.$$ Thus, if I can show that $|f(1/z)|$ is a polynomial, we'll know it can't have an essential singularity at the origin, concluding my proof. Limit of $s_n = \int\limits_0^1 \frac{nx^{n-1}}{1+x} dx$ as $n \to \infty$ How does the admissions process work for Ph.D. programs in the US, particularly for weak or borderline students? I know the "textbook" estimate of the standard error of a proportion is $SE=\sqrt{\frac{p(1-p)}{n}}$, but does this hold up when the data are weighted? has a duration of instantaneous. However, it also produces an ongoing effect, and the spell's text describes when this effect ends (emphasis added):  At the end of every 30 days, the creature can repeat its saving throw against this spell. If it succeeds on its saving throw, the spell ends. The spell can also be ended by greater restoration, heal, or wish.  The fact that the text refers to the spell ending, as opposed to just &quot;the effect&quot; or something similar, implies that it produces an ongoing magical spell effect, which can be dispelled using Dispel Magic. However, it has also been argued that any effects that persist beyond a spell's stated duration cannot be considered magical (with the classic example being fires started by a Fireball spell). Applying this rule to Feeblemind, it shouldn't have any ongoing effects since its duration is instantaneous. So, which is it? Are the ongoing effects of Feeblemind considered magical and therefore susceptible to effects like Dispel Magic, or are are they simply the lingering result of the initial shattering of the creature's intellect and personality? (Note that the final sentence quoted above does not by itself rule out Dispel Magic and other ways of ending the spell. It gives additional options for ending the spell, but it doesn't say these are the only options.)