{"action":"closed","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/7364","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/7364/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/7364/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/7364/events","html_url":"https://github.com/fchollet/keras/issues/7364","id":243578985,"number":7364,"title":"Sequence to sequence LSTM model for composing music failing","user":{"login":"santiagobenoit","id":19509047,"avatar_url":"https://avatars3.githubusercontent.com/u/19509047?v=4","gravatar_id":"","url":"https://api.github.com/users/santiagobenoit","html_url":"https://github.com/santiagobenoit","followers_url":"https://api.github.com/users/santiagobenoit/followers","following_url":"https://api.github.com/users/santiagobenoit/following{/other_user}","gists_url":"https://api.github.com/users/santiagobenoit/gists{/gist_id}","starred_url":"https://api.github.com/users/santiagobenoit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/santiagobenoit/subscriptions","organizations_url":"https://api.github.com/users/santiagobenoit/orgs","repos_url":"https://api.github.com/users/santiagobenoit/repos","events_url":"https://api.github.com/users/santiagobenoit/events{/privacy}","received_events_url":"https://api.github.com/users/santiagobenoit/received_events","type":"User","site_admin":false},"labels":[{"id":611595073,"url":"https://api.github.com/repos/fchollet/keras/labels/stale","name":"stale","color":"ffffff","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2017-07-18T02:07:38Z","updated_at":"2017-11-15T04:52:03Z","closed_at":"2017-11-15T04:52:03Z","author_association":"NONE","body":"Hi all,\r\nI'm trying to train a sequence to sequence model to predict the next note (or full MIDI event if I add more functionality later) in a series of notes. So far, nothing has worked, the loss converges to around -0.4783 and the prediction is always all 4s padded with 0s on the left. Here's the code:\r\n\r\n```\r\nimport tensorflow as tf\r\nimport numpy as np\r\nimport mido\r\nfrom keras.preprocessing.text import Tokenizer\r\nfrom keras.preprocessing.sequence import pad_sequences\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Dense,Activation,Dropout,Embedding,SpatialDropout1D\r\nfrom keras.layers import LSTM\r\n\r\ndef shift(seq, n):\r\n    n = n % len(seq)\r\n    return seq[n:] + seq[:n]\r\n\r\ntrain = mido.MidiFile(\"training_data/train.mid\")\r\ntest = mido.MidiFile(\"training_data/test.mid\")\r\nseq = \"cdefgab\"*100 #test sequence; will replace later with notes read from MIDI events\r\n\r\ntk = Tokenizer(num_words=2000, lower=True, split=\" \")\r\ntk.fit_on_texts(seq)\r\nx = tk.texts_to_sequences(seq)\r\n\r\n#shifing to left\r\ny = shift(x,1)\r\nprint(x)\r\nprint(\"\\n\")\r\nprint(y)\r\n\r\n#padding sequence\r\nmax_len = 100\r\nmax_features=len(tk.word_counts)\r\nX = pad_sequences(x, maxlen=max_len)\r\nY = pad_sequences(y, maxlen=max_len)\r\nprint(X)\r\nprint(\"\\n\")\r\nprint(Y)\r\n\r\n#lstm model\r\nmodel = Sequential()\r\nmodel.add(Embedding(max_features + 1, 128, input_length=max_len))\r\nmodel.add(SpatialDropout1D(0.2))\r\nmodel.add(LSTM(128, recurrent_dropout=0.2, dropout=0.2))\r\nmodel.add(Dense(max_len))\r\nmodel.add(Activation('relu'))\r\nmodel.compile(loss='binary_crossentropy', optimizer='rmsprop')\r\nmodel.summary()\r\n\r\nmodel.fit(X, Y, batch_size=200, epochs=10)\r\n\r\nmodel.predict(X, batch_size=200)\r\n```\r\nTraining:\r\n```\r\nEpoch 1/10\r\n700/700 [==============================] - 1s - loss: 0.1733     \r\nEpoch 2/10\r\n700/700 [==============================] - 0s - loss: -0.0066    \r\nEpoch 3/10\r\n700/700 [==============================] - 0s - loss: -0.4747     \r\nEpoch 4/10\r\n700/700 [==============================] - 0s - loss: -0.4782     \r\nEpoch 5/10\r\n700/700 [==============================] - 0s - loss: -0.4783     \r\nEpoch 6/10\r\n700/700 [==============================] - 0s - loss: -0.4783     \r\nEpoch 7/10\r\n700/700 [==============================] - 0s - loss: -0.4783     \r\nEpoch 8/10\r\n700/700 [==============================] - 0s - loss: -0.4783     \r\nEpoch 9/10\r\n700/700 [==============================] - 0s - loss: -0.4783     \r\nEpoch 10/10\r\n700/700 [==============================] - 0s - loss: -0.4783\r\n```\r\nPrediction:\r\n```\r\narray([[ 0.        ,  0.        ,  0.        , ...,  0.        ,\r\n         0.        ,  4.26653194],\r\n       [ 0.        ,  0.        ,  0.        , ...,  0.        ,\r\n         0.        ,  4.26872206],\r\n       [ 0.        ,  0.        ,  0.        , ...,  0.        ,\r\n         0.        ,  4.26919842],\r\n       ..., \r\n       [ 0.        ,  0.        ,  0.        , ...,  0.        ,\r\n         0.        ,  4.27029228],\r\n       [ 0.        ,  0.        ,  0.        , ...,  0.        ,\r\n         0.        ,  4.26764679],\r\n       [ 0.        ,  0.        ,  0.        , ...,  0.        ,\r\n         0.        ,  4.26126671]], dtype=float32)\r\n```\r\nAs you might be able to tell, I am new to machine learning. Can someone help point me in the right direction?\r\n\r\n- [ √] Check that you are up-to-date with the master branch of Keras. You can update with:\r\npip install git+git://github.com/fchollet/keras.git --upgrade --no-deps\r\n\r\n- [ √] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found [here](https://www.tensorflow.org/get_started/os_setup).\r\n\r\n- [ √] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:\r\npip install git+git://github.com/Theano/Theano.git --upgrade --no-deps\r\n\r\n- [ √] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).\r\n"}}
{"actor":{"display_login":"stale"}}