{"action":"created","issue":{"url":"https://api.github.com/repos/jcjohnson/neural-style/issues/425","repository_url":"https://api.github.com/repos/jcjohnson/neural-style","labels_url":"https://api.github.com/repos/jcjohnson/neural-style/issues/425/labels{/name}","comments_url":"https://api.github.com/repos/jcjohnson/neural-style/issues/425/comments","events_url":"https://api.github.com/repos/jcjohnson/neural-style/issues/425/events","html_url":"https://github.com/jcjohnson/neural-style/issues/425","id":261956185,"number":425,"title":"FP16 version of neural style","user":{"login":"michaelhuang74","id":23154573,"avatar_url":"https://avatars2.githubusercontent.com/u/23154573?v=4","gravatar_id":"","url":"https://api.github.com/users/michaelhuang74","html_url":"https://github.com/michaelhuang74","followers_url":"https://api.github.com/users/michaelhuang74/followers","following_url":"https://api.github.com/users/michaelhuang74/following{/other_user}","gists_url":"https://api.github.com/users/michaelhuang74/gists{/gist_id}","starred_url":"https://api.github.com/users/michaelhuang74/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/michaelhuang74/subscriptions","organizations_url":"https://api.github.com/users/michaelhuang74/orgs","repos_url":"https://api.github.com/users/michaelhuang74/repos","events_url":"https://api.github.com/users/michaelhuang74/events{/privacy}","received_events_url":"https://api.github.com/users/michaelhuang74/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2017-10-01T23:42:31Z","updated_at":"2017-10-02T09:35:19Z","closed_at":null,"author_association":"NONE","body":"I am trying to carry out neural style in half precision, i.e., 16-bit floating point.\r\nI modified the code and it was posted on https://github.com/michaelhuang74/FP16-Neural-Style/blob/master/neural_style_FP16.lua\r\n\r\n(1) The above version doesn't work with lbfgs optimizer. \r\n(2) When I use the adam optimizer , it does run on Titan Xp. FP16 performance on Titan Xp is supposed to 1/64 of FP32 performance. But the current implementation is only a bit slower than FP32 version. In addition, it has NaN (not a number) issue. Follows are the outputs.\r\n\r\nth neural_style.lua -style_image style/vangogh.jpg -content_image inputimage/man_face.jpg -style_weight 20 -content_weight 1 -image_size 800 -style_scale 1   -style_layers relu1_2,relu2_2,relu3_2,relu4_2,relu5_2 -output_image outputimage/man_face.vangogh.sw20.cw1.sl12-22-32-42-52.c800s800.1000it.adam.cudnn.half.jpg -num_iterations 1000 -save_iter 100 -backend cudnn -cudnn_autotune -optimizer adam\r\n\r\n\r\nRunning optimization with ADAM\r\nIteration 100 / 1000\r\n  Content 1 loss: nan\r\n  Style 1 loss: nan\r\n  Style 2 loss: nan\r\n  Style 3 loss: nan\r\n  Style 4 loss: nan\r\n  Style 5 loss: nan\r\n  Total loss: nan\r\nIteration 200 / 1000\r\n  Content 1 loss: nan\r\n  Style 1 loss: nan\r\n  Style 2 loss: nan\r\n  Style 3 loss: nan\r\n  Style 4 loss: nan\r\n  Style 5 loss: nan\r\n  Total loss: nan\r\n\r\nI changed the epsilon from the default 1e-8 to 1e-4 or 1e-2, still facing the nan (not a number) output.\r\n\r\nAny idea how to modify the code to make the FP16 work? \r\n"},"comment":{"url":"https://api.github.com/repos/jcjohnson/neural-style/issues/comments/333485976","html_url":"https://github.com/jcjohnson/neural-style/issues/425#issuecomment-333485976","issue_url":"https://api.github.com/repos/jcjohnson/neural-style/issues/425","id":333485976,"user":{"login":"htoyryla","id":15064373,"avatar_url":"https://avatars0.githubusercontent.com/u/15064373?v=4","gravatar_id":"","url":"https://api.github.com/users/htoyryla","html_url":"https://github.com/htoyryla","followers_url":"https://api.github.com/users/htoyryla/followers","following_url":"https://api.github.com/users/htoyryla/following{/other_user}","gists_url":"https://api.github.com/users/htoyryla/gists{/gist_id}","starred_url":"https://api.github.com/users/htoyryla/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/htoyryla/subscriptions","organizations_url":"https://api.github.com/users/htoyryla/orgs","repos_url":"https://api.github.com/users/htoyryla/repos","events_url":"https://api.github.com/users/htoyryla/events{/privacy}","received_events_url":"https://api.github.com/users/htoyryla/received_events","type":"User","site_admin":false},"created_at":"2017-10-02T09:35:19Z","updated_at":"2017-10-02T09:35:19Z","author_association":"NONE","body":"Comment removed... could not reproduce the results."}}
{"actor":{"display_login":"htoyryla"}}