I wanted to see if a neural network could learn the identity function using the MNIST handwritten dataset.  Here is the full code  import keras from keras.datasets import mnist from keras.models import Sequential from keras.layers import Dense from keras.optimizers import RMSprop  batch_size = 128 epochs = 20 (x_train, y_train), (x_test, y_test) = mnist.load_data() x_train = x_train.reshape(60000, 784) x_test = x_test.reshape(10000, 784)  model = Sequential() model.add(Dense(784, activation='relu', input_shape=(784,))) model.add(Dense(784, activation='relu')) model.add(Dense(784, activation='relu')) model.add(Dense(784, activation='relu')) model.summary() model.compile(loss='mean_squared_error',             optimizer=RMSprop(),             metrics=['mean_absolute_percentage_error'])  history = model.fit(x_train, x_train,                     batch_size=batch_size,                     epochs=epochs,                     verbose=1,                     validation_data=(x_test, x_test)) score = model.evaluate(x_test, x_test, verbose=0) print('Test loss:', score[0]) print('Test MAPE:', score[1])   and the output  **4 dense layers** Epoch 20/20 60000/60000 [==============================] - 50s 840us/step - loss: 456.7581 - mean_absolute_percentage_error: 351097677.7045 - val_loss: 523.7151 - val_mean_absolute_percentage_error: 504905991.0656 Test loss: 523.7150838867187 Test MAPE: 504905988.5056   What I can't quite get my head around is why training cannot find the perfect solution to the problem and why it takes so long to even get close to it? Even with one dense layer the exact solution cannot be found:  **1 dense layer** Epoch 20/20 60000/60000 [==============================] - 16s 268us/step - loss: 180.6187 - mean_absolute_percentage_error: 209296481.2373 - val_loss: 167.9543 - val_mean_absolute_percentage_error: 192590419.9936 Test loss: 167.954341796875 Test MAPE: 192590420.1984   Conceptually I can see that there is a solution space (not just the exact identity function) as it is likely there are some pixels that have the same value as each other in all the images which could be swapped in the training set at no loss (0's around the edge for example). With the knowledge that this is a local minimum can I learn anything from this to guide me out rather than playing with hyperparameters until I find something better?
I'm training a neural network but the training loss doesn't decrease. How can I fix this?  I'm not asking about overfitting or regularization. I'm asking about how to solve the problem where my network's performance doesn't improve on the training set.    This question is intentionally general so that other questions about how to train a neural network can be closed as a duplicate of this one, with the attitude that "if you give a man a fish you feed him for a day, but if you teach a man to fish, you can feed him for the rest of his life." See this Meta thread for a discussion:   If your neural network does not generalize well, see:
I am trying to create a map in both Leaflet and OpenLayers 3 which displays the  layer EMAP8 from the WMS http://wms.nlsc.gov.tw/wms. I can't get it to work. I can open the layer in QGIS, but when I try to use it in Leaflet like this:  var overlay_taiwanemapen0 = L.tileLayer.wms('http://wms.nlsc.gov.tw/wms', {     layers: 'EMAP8',     format: 'image/png',     transparent: true,     continuousWorld : true, });   or in OpenLayers 3 like this:  var lyr_taiwanemapen = new ol.layer.Tile({     source: new ol.source.TileWMS(({         url: "http://wms.nlsc.gov.tw/wms",         params: {"LAYERS": "EMAP8", "TILED": "true"},     })),     title: "taiwane-map(en)", });   I get nothing displayed. I can see layers being requested, and data being transferred, but nothing on the map.  What am I doing wrong? DXF file not to scale in Blender When I open the DXF file in LibreCAD, then it shows the correct size of an object, but when I import that file in Blender, then the size of that object becomes very large. Using WCS layer in QGIS for processing gives FAILURE: Missing destination I have a GeoServer serving raster dem data out via WCS and can add the layer easily to QGIS (3.2) but when I try to use it as the input for an operation (e.g. HillShade) I get an error.  Processing algorithm… Algorithm 'Hillshade' starting… Input parameters: { 'ALTITUDE' : 45, 'AZIMUTH' : 315, 'BAND' : 1, 'COMBINED' : False, 'COMPUTE_EDGES' : False, 'INPUT' : 'dpiMode=7&amp;identifier=training:panorama&amp;url=http://localhost:8080/geoserver/ows?version%3D1.1.1', 'MULTIDIRECTIONAL' : False, 'OPTIONS' : '', 'OUTPUT' : 'C:/Users/ishare/AppData/Local/Temp/2/processing_868cc25180f24d9f8e193bc0cfedb6d3/7cf48f64d8b54ee694c3ee93ed17e903/OUTPUT.tif', 'SCALE' : 1, 'ZEVENBERGEN' : False, 'Z_FACTOR' : 5 }  GDAL command: gdaldem hillshade dpiMode=7&amp;identifier=training:panorama&amp;url=http://localhost:8080/geoserver/ows?version%3D1.1.1 C:/Users/ishare/AppData/Local/Temp/2/processing_868cc25180f24d9f8e193bc0cfedb6d3/7cf48f64d8b54ee694c3ee93ed17e903/OUTPUT.tif -of GTiff -b 1 -z 5.0 -s 1.0 -az 315.0 -alt 45.0 GDAL command output:   FAILURE: Missing destination.   Clearly the missing destination is a red herring, and it is failing to understand the WCS layer as an input. But GDAL is built with WCS support included so shouldn't this work?  Do I need to do something else or start working out where to hack the QGIS code? Calculate the radius of a circle given the chord length and height of a segment I have a (circular) segment of known height and known chord length. Is is possible to determine the radius of the circle?  Any help much appreciated. dpkg: error: dpkg status database is locked by another process I am currently learning electromagnetism and I am getting really annoyed using a physical hand to accomplish the right hand rule. I get it, i've seen diagrams, and i've solved problems with it but still somehow I mess up because I bend a finger or twist my wrist a strange way. I am tired of using my hand.  So my question is can I find an entirely mathematical basis for this?  Lets say for any given diagram I define the positive $x$ axis as going to the right, the positive $y$ axis as going up and the positive $z$ axis as going into the page. Then I just did unit vectors for everything to determine the missing component?  As I understand it $\vec{F} = q (\vec{V}\times\vec{B})$ and with that definition I am confident I could find out the direction of the electric field given the velocity of a particle of charge and the magnetic field but how can I re-arrange this so I can solve for other variables? For example what does $\vec{B}$ equal in terms of a cross product involving $\vec{V}$ and $\vec{F}$? A probability theory question about independent coin tosses by two players Say Bob tosses his $n+1$ fair coins and Alice tosses her $n$ fair coins. Lets assume independent coin tosses. Now after all the $2n+1$ coin tosses one wants to know the probability that Bob has gotten more heads than Alice.    The way I thought of it is this : if Bob gets $0$ heads then there is no way he can get more heads than Alice. Otherwise the number of heads Bob can get which allows him to win is anything in the set $\{1,2,\dots,n+1\}$. And if Bob gets $x$ heads then the number of heads that Alice can get is anything in the set $\{0,1,2,..,x-1\}$. So\begin{align}P(\text{Bob gets more heads than Alice})&amp;= \sum_{x=1}^{n+1} \sum_{y=0}^{x-1} P( \text{Bob gets x heads }\cap \text{Alice gets y heads }) \\[0.2cm]&amp;= \sum_{x=1}^{n+1} \sum_{y=0}^{x-1} \left(C^{n+1}_x \frac{1}{2}^{x} \frac{1}{2}^{n+1-x}\right)\left( C^n_y \frac{1}{2}^y \frac {1}{2}^{n-y}\right)\\[0.2cm]&amp; = \sum_{x=1}^{n+1} \sum_{y=0}^{x-1} \frac{C^{n+1}_x C^n_y}{2^{2n+1}}\end{align}   How does one simplify this?      Apparently the answer is $\frac{1}{2}$ by an argument which looks like this, Since Bob tosses one more coin that Alice, it is impossible that they toss both the same number of heads and the same number of tails. So Bob tosses either more heads than Alice or more tails than Alice (but not both). Since the coins are fair, these events are equally likely by symmetry, so both events have probability 1/2. Show that $\lim_{x \to +\infty}\left(f(x)+f'(x)\right)=0 \Rightarrow \lim_{x \to +\infty} f(x)=0$ How to show that $\lim_{x \to +\infty}(f(x)+f'(x))=0 $ implies $\lim_{x \to +\infty} f(x)=0$? English equivalent for the Persian idiom "send someone out in pursuit of black chickpeas" "To send somebody out in pursuit of some black " is a Persian idiom that implies 'to make or ask someone to run an errand so that you be able to have/ buy some time in order to deal with your private affairs in their absence', like having or continuing a (secret) conversation with someone else.   According to its etymology, since 'black garbanzos/chickpeas' were rarely available in the markets, when someone wanted to buy them, they had to search many stores for a long time, and most of time they couldn't find them at all.   Today, its meaning has changed somehow, and it doesn't necessarily mean that we ask someone to run an impossible errand, but it means we use that errand as an excuse to send them away for a while just to have time to do something in their absence.   Example scenario:   Suppose your classmate has come to your house and you are talking about some girly issues in the sitting room. Now you want to tell her a big secret about one of your classmates, but not in the presence of your younger sister who is sitting there too. So you ask her to go to the market and buy some snacks for you. In this way, you will have some time to continue your conversation until she is back. So you might say to your friend:     Well, I sent my sister out in pursuit of some black chickpeas.  I'm sure that she won't be back for another 15 minutes, so we can talk more freely now."   And your sister, before leaving the house, might sarcastically say to you:     Okay, but I know this is just an excuse, and you are just sending me in pursuit of some black chickpeas!!! so don't think that you can fool me!   Is there any idiom, expression or proverb that conveys the same meaning or connotation in English (i.e. to make or ask someone to do an errand just in order to buy time so that you can deal with some of your private affairs)?   I have found to be/go on  but I'm not sure if I can use it or not. What can I expect from a Skype interview for PhD admissions? How can I create launchers on my desktop? Don't show pending suggested edits to me after I run out of votes Homotopy equivalence of universal cover As part of am exam question (Q21F ), I'm trying to prove that if $X$ and $Y$ are path-connected, locally path-connected spaces with universal covers $\widetilde{X}$ and $\widetilde{Y}$, respectively, then if $X \simeq Y$ then $\widetilde{X} \simeq \widetilde{Y}$.  My attempt might be correct, but it seems too complicated and amongst the jumble I may have made some incorrect assumptions. So what I'm looking for is (a) verification or correction; and (b) ideas as to how I could simplify my argument.    My argument:  Let $p:\widetilde{X} \to X$ and $q:\widetilde{Y} \to Y$ be the covering maps, and let $X \overset{f}{\underset{g}{\leftrightarrows}} Y$ be a homotopy equivalence.  Fix a point $\widetilde{x} \in \widetilde{X}$, let $x=p(x) \in X$ and $y=f(x) \in Y$, and fix $\widetilde{y} \in q^{-1}(\{y\}) \subseteq \widetilde{Y}$. Also let $x' = g(y) \in X$ and fix $\widetilde{x'} \in p^{-1}(\{x'\}) \subseteq \widetilde{X}$.  Define a map $\widetilde{f} : \widetilde{X} \to \widetilde{Y}$ as follows. For $\widetilde{z} \in \widetilde{X}$ let $\widetilde{u}:[0,1] \to \widetilde{X}$ be a path from $\widetilde{x}$ to $\widetilde{z}$. Let $z=p(z)$ so that $u=p\widetilde{u}$ is a path in $X$ from $x$ to $z$. Let $v=fu$, so that $v$ is a path in $Y$ from $y$ to $f(z)$. Lift $v$ to a path $\widetilde{v}$ in $\widetilde{Y}$ with $\widetilde{v}(0) = \widetilde{y}$. Define $\widetilde{f}(\widetilde{z}) = \widetilde{v}(1)$.  Notice that $q\widetilde{v}=v$ so that $q\widetilde{f} = fp$.  Define $\widetilde{g}:Y \to X$ analogously: For $\widetilde{z} \in \widetilde{Y}$ let $\widetilde{a}:[0,1] \to \widetilde{Y}$ be a path from $\widetilde{y}$ to $\widetilde{z}$. Let $z=q(z)$ so that $a=q\widetilde{a}$ is a path in $Y$ from $y$ to $z$. Let $b=fa$, so that $b$ is a path in $X$ from $x'$ to $g(z)$. Lift $b$ to a path $\widetilde{b}$ in $\widetilde{X}$ with $\widetilde{b}(0) = \widetilde{x'}$. Define $\widetilde{g}(\widetilde{z}) = \widetilde{b}(1)$.  Likewise, notice that $p\widetilde{g}=gq$.  Claim: $\widetilde{X} \overset{\widetilde{f}}{\underset{\widetilde{g}}{\leftrightarrows}} \widetilde{Y}$ is a homotopy equivalence.  We have $p\widetilde{g}\widetilde{f} = gq\widetilde{f}=gfp \simeq p$ and $q\widetilde{f}\widetilde{g} = fp\widetilde{g} = fgq \simeq q$.  This shows that $\widetilde{g}\widetilde{f}$ and $\widetilde{f}\widetilde{g}$ are covering translations (deck transformations) and are therefore homotopic to the respective identity maps. So we have a homotopy equivalence.    Any comments would be appreciated. In a batch file I need to extract a month, day, year from the date command.  So I used the following, which essentially parses the Date command to extract its sub strings into a variable:  set Day=%Date:~3,2% set Mth=%Date:~0,2% set Yr=%Date:~6,4%   This is all great, but if I deploy this batch file to a machine with a different regional/country settings, it fails because month, day and year are in different locations.   How can I extract month, day and year regardless of the date format? Adverb sentence query He is arguably the best actor of his generation   In the above sentence the adverb is placed just after "be" Can we use adverb after "be"? If we can use then please explain properly because I have studied that adverb can be used before adjective and adverb . Complicated FOR loop nesting. Having trouble cleaning up legacy code How are Pokestops chosen? After playing the game for a little while, I've noticed that Pokestops are chosen seemingly at random locations on the map. A few of them are even in inaccessible places; an abandoned church building, inside of an airforce base, etc. How are these locations chosen as Pokestops? Why do we use the Lagrangian and Hamiltonian instead of other related functions? There are 4 main functions in mechanics. $L(q,\dot{q},t)$, $H(p,q,t)$, $K(\dot{p},\dot{q},t)$, $G(p,\dot{p},t)$. First two are Lagrangian and Hamiltonian. Second two are some kind of analogical to first two, but we don't use them. Every where they write that using them can cause problems. But why? How do I run .sh scripts? What is ISCAS85 benchmark? What is ISCAS85 benchmark? I find it in a , is there any simulator for it?