{"action":"created","issue":{"url":"https://api.github.com/repos/minimagick/minimagick/issues/423","repository_url":"https://api.github.com/repos/minimagick/minimagick","labels_url":"https://api.github.com/repos/minimagick/minimagick/issues/423/labels{/name}","comments_url":"https://api.github.com/repos/minimagick/minimagick/issues/423/comments","events_url":"https://api.github.com/repos/minimagick/minimagick/issues/423/events","html_url":"https://github.com/minimagick/minimagick/issues/423","id":243781869,"number":423,"title":"A JSON::ParserError occurred","user":{"login":"PeterMozesMerl","id":6610365,"avatar_url":"https://avatars0.githubusercontent.com/u/6610365?v=4","gravatar_id":"","url":"https://api.github.com/users/PeterMozesMerl","html_url":"https://github.com/PeterMozesMerl","followers_url":"https://api.github.com/users/PeterMozesMerl/followers","following_url":"https://api.github.com/users/PeterMozesMerl/following{/other_user}","gists_url":"https://api.github.com/users/PeterMozesMerl/gists{/gist_id}","starred_url":"https://api.github.com/users/PeterMozesMerl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PeterMozesMerl/subscriptions","organizations_url":"https://api.github.com/users/PeterMozesMerl/orgs","repos_url":"https://api.github.com/users/PeterMozesMerl/repos","events_url":"https://api.github.com/users/PeterMozesMerl/events{/privacy}","received_events_url":"https://api.github.com/users/PeterMozesMerl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-07-18T16:49:58Z","updated_at":"2017-07-20T21:58:54Z","closed_at":null,"body":"I started receiving emails of JSON::ParserError exceptions once in a while when the user uploads an image. \r\n\r\nI don’t have the specific images. I save every image caused an ImageMagick::ThisOrThat exception (and they were all user errors or corrupt images).\r\n\r\nBut the JSON::ParserError is not handled there. I will include it but it will take time and the exception doesn’t happen often.\r\n\r\nBased on my emails, it has been occuring since 1st March, which was few days after I deployed a change that maximizes the quality of the uploaded image.\r\n\r\nThe backtrace attach is all I have (sent by ExceptionNotification). It’s strange if I call JSON.parse on the string you see in the backtrace, it gets parsed. At least the one I tried did.\r\n\r\nI would attach the specific image if I could. I don’t deploy each day, and the exception happened only 33 times of a few ten million of page views. It might take a time to find and save a specific image for debugging.\r\n\r\nI can see a strange character in the \"comment\" field on GitHub. But when I copy it to irb, it’s a space.\r\n\r\nThe number is almost always 743. It was 419 two times, 409 two times, 784 two times. That’s all.\r\n\r\n```ruby\r\ntmp_img.quality(maximum_value) if original_img.data['quality'].blank? || original_img.data['quality'] > maximum_value\r\n ```\r\n\r\n```\r\nA JSON::ParserError occurred in images#create:\r\n\r\n  743: unexpected token at '{\r\n  \"image\": {\r\n    \"name\": \"/tmp/mini_magick20170718-69587-1qd9hbh.jpeg[0]\",\r\n    \"format\": \"JPEG\",\r\n    \"formatDescription\": \"Joint Photographic Experts Group JFIF format\",\r\n    \"mimeType\": \"image/jpeg\",\r\n    \"class\": \"DirectClass\",\r\n    \"geometry\": {\r\n      \"width\": 1280,\r\n      \"height\": 960,\r\n      \"x\": 0,\r\n      \"y\": 0\r\n    },\r\n    \"resolution\": {\r\n      \"x\": \"72\",\r\n      \"y\": \"72\"\r\n    },\r\n    \"printSize\": {\r\n      \"x\": \"17.777777777777778567\",\r\n      \"y\": \"13.333333333333333925\"\r\n    },\r\n    \"units\": \"PixelsPerInch\",\r\n    \"type\": \"TrueColor\",\r\n    \"endianess\": \"Undefined\",\r\n    \"colorspace\": \"sRGB\",\r\n    \"depth\": 8,\r\n    \"baseDepth\": 8,\r\n    \"channelDepth\": {\r\n      \"red\": 8,\r\n      \"green\": 8,\r\n      \"blue\": 8\r\n    },\r\n    \"pixels\": 1228800,\r\n    \"imageStatistics\": {\r\n      \"all\": {\r\n        \"min\": \"46\",\r\n        \"max\": \"255\",\r\n        \"mean\": \"180.931\",\r\n        \"standardDeviation\": \"52.6511\",\r\n        \"kurtosis\": \"-1.0767\",\r\n        \"skewness\": \"-0.00895097\"\r\n      }\r\n    },\r\n    \"channelStatistics\": {\r\n      \"red\": {\r\n        \"min\": \"46\",\r\n        \"max\": \"255\",\r\n        \"mean\": \"181.173\",\r\n        \"standardDeviation\": \"55.9718\",\r\n        \"kurtosis\": \"-1.03302\",\r\n        \"skewness\": \"-0.230587\"\r\n      },\r\n      \"green\": {\r\n        \"min\": \"58\",\r\n        \"max\": \"255\",\r\n        \"mean\": \"175.331\",\r\n        \"standardDeviation\": \"52.5951\",\r\n        \"kurtosis\": \"-1.1273\",\r\n        \"skewness\": \"0.216887\"\r\n      },\r\n      \"blue\": {\r\n        \"min\": \"56\",\r\n        \"max\": \"255\",\r\n        \"mean\": \"186.289\",\r\n        \"standardDeviation\": \"48.5512\",\r\n        \"kurtosis\": \"-1.14997\",\r\n        \"skewness\": \"0.0986408\"\r\n      }\r\n    },\r\n    \"renderingIntent\": \"Perceptual\",\r\n    \"gamma\": 0.454545,\r\n    \"chromaticity\": {\r\n      \"redPrimary\": {\r\n        \"x\": 0.64,\r\n        \"y\": 0.33\r\n      },\r\n      \"greenPrimary\": {\r\n        \"x\": 0.3,\r\n        \"y\": 0.6\r\n      },\r\n      \"bluePrimary\": {\r\n        \"x\": 0.15,\r\n        \"y\": 0.06\r\n      },\r\n      \"whitePrimary\": {\r\n        \"x\": 0.3127,\r\n        \"y\": 0.329\r\n      }\r\n    },\r\n    \"backgroundColor\": \"#FFFFFF\",\r\n    \"borderColor\": \"#DFDFDF\",\r\n    \"matteColor\": \"#BDBDBD\",\r\n    \"transparentColor\": \"#000000\",\r\n    \"interlace\": \"None\",\r\n    \"intensity\": \"Undefined\",\r\n    \"compose\": \"Over\",\r\n    \"pageGeometry\": {\r\n      \"width\": 1280,\r\n      \"height\": 960,\r\n      \"x\": 0,\r\n      \"y\": 0\r\n    },\r\n    \"dispose\": \"Undefined\",\r\n    \"iterations\": 0,\r\n    \"compression\": \"JPEG\",\r\n    \"quality\": 95,\r\n    \"orientation\": \"TopLeft\",\r\n    \"properties\": {\r\n      \"comment\": \"\u0005\",\r\n      \"date:create\": \"2017-07-18T15:08:37+02:00\",\r\n      \"date:modify\": \"2017-07-18T15:08:37+02:00\",\r\n      \"exif:ColorSpace\": \"1\",\r\n      \"exif:ComponentsConfiguration\": \"1, 2, 3, 0\",\r\n      \"exif:DateTime\": \"2014:04:21 17:42:32\",\r\n      \"exif:DateTimeDigitized\": \"2014:04:21 17:42:32\",\r\n      \"exif:DateTimeOriginal\": \"2014:04:21 17:42:32\",\r\n      \"exif:ExifImageLength\": \"960\",\r\n      \"exif:ExifImageWidth\": \"1280\",\r\n      \"exif:ExifOffset\": \"236\",\r\n      \"exif:ExifVersion\": \"48, 50, 50, 48\",\r\n      \"exif:ExposureMode\": \"0\",\r\n      \"exif:ExposureProgram\": \"3\",\r\n      \"exif:ExposureTime\": \"1/14\",\r\n      \"exif:Flash\": \"0\",\r\n      \"exif:FlashPixVersion\": \"48, 49, 48, 48\",\r\n      \"exif:FNumber\": \"26/10\",\r\n      \"exif:FocalLength\": \"355/100\",\r\n      \"exif:GPSInfo\": \"596\",\r\n      \"exif:GPSVersionID\": \"2, 2, 0, 0\",\r\n      \"exif:ImageLength\": \"960\",\r\n      \"exif:ImageWidth\": \"1280\",\r\n      \"exif:InteroperabilityOffset\": \"566\",\r\n      \"exif:ISOSpeedRatings\": \"200\",\r\n      \"exif:Make\": \"SAMSUNG\",\r\n      \"exif:MaxApertureValue\": \"300/100\",\r\n      \"exif:MeteringMode\": \"3\",\r\n      \"exif:Model\": \"GT-S5830i\",\r\n      \"exif:Orientation\": \"1\",\r\n      \"exif:ResolutionUnit\": \"2\",\r\n      \"exif:SceneCaptureType\": \"0\",\r\n      \"exif:SceneType\": \"1\",\r\n      \"exif:Software\": \"S5830iXXLD3\",\r\n      \"exif:thumbnail:Compression\": \"6\",\r\n      \"exif:thumbnail:InteroperabilityIndex\": \"R98\",\r\n      \"exif:thumbnail:InteroperabilityVersion\": \"48, 49, 48, 48\",\r\n      \"exif:thumbnail:JPEGInterchangeFormat\": \"668\",\r\n      \"exif:thumbnail:JPEGInterchangeFormatLength\": \"17705\",\r\n      \"exif:thumbnail:ResolutionUnit\": \"2\",\r\n      \"exif:WhiteBalance\": \"0\",\r\n      \"exif:XResolution\": \"72/1\",\r\n      \"exif:YCbCrPositioning\": \"1\",\r\n      \"exif:YResolution\": \"72/1\",\r\n      \"jpeg:colorspace\": \"2\",\r\n      \"jpeg:sampling-factor\": \"2x1,1x1,1x1\",\r\n      \"signature\": \"9d6f15dee6a144e3667bdb5e4ba4b32fcfa90868042819a982df687c9b8f549f\"\r\n    },\r\n    \"profiles\": {\r\n      \"exif\": {\r\n        \"length\": \"18379\"\r\n      }\r\n    },\r\n    \"artifacts\": {\r\n      \"filename\": \"/tmp/mini_magick20170718-69587-1qd9hbh.jpeg[0]\"\r\n    },\r\n    \"tainted\": false,\r\n    \"filesize\": \"332360B\",\r\n    \"numberPixels\": \"1228800\",\r\n    \"pixelsPerSecond\": \"52428800B\",\r\n    \"userTime\": \"0.023u\",\r\n    \"elapsedTime\": \"0:01.023\",\r\n    \"version\": \"ImageMagick 6.9.8-9 Q16 amd64 2017-07-13 http://www.imagemagick.org\"\r\n  }\r\n}'\r\n  app/models/image.rb:302:in `maximize_quality'\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/minimagick/minimagick/issues/comments/316842030","html_url":"https://github.com/minimagick/minimagick/issues/423#issuecomment-316842030","issue_url":"https://api.github.com/repos/minimagick/minimagick/issues/423","id":316842030,"user":{"login":"janko-m","id":795488,"avatar_url":"https://avatars2.githubusercontent.com/u/795488?v=4","gravatar_id":"","url":"https://api.github.com/users/janko-m","html_url":"https://github.com/janko-m","followers_url":"https://api.github.com/users/janko-m/followers","following_url":"https://api.github.com/users/janko-m/following{/other_user}","gists_url":"https://api.github.com/users/janko-m/gists{/gist_id}","starred_url":"https://api.github.com/users/janko-m/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/janko-m/subscriptions","organizations_url":"https://api.github.com/users/janko-m/orgs","repos_url":"https://api.github.com/users/janko-m/repos","events_url":"https://api.github.com/users/janko-m/events{/privacy}","received_events_url":"https://api.github.com/users/janko-m/received_events","type":"User","site_admin":false},"created_at":"2017-07-20T21:58:54Z","updated_at":"2017-07-20T21:58:54Z","body":"Thank you for the detailed info! This is really strange, because MiniMagick also uses `JSON.parse` to parse the JSON, so I don't get why MiniMagick fails but the manual `JSON.parse` succeeds. Note that there was already a report of a `JSON::ParserError` in https://github.com/minimagick/minimagick/issues/410, but in that case not even manual `JSON.parse` failed.\r\n\r\nOne idea where the issue could be is string encoding. MiniMagick always returns output from ImageMagick commands in binary encoding (`ASCII-8BIT`), so it might be that the JSON parsing is failing for binary encoding, but when you copy the JSON string from the error message and paste it into a Ruby file for debugging, it will now be in UTF-8 encoding because that's Ruby's default encoding for strings. I tried to `.force_encoding(Encoding::BINARY)` the JSON you provided, but it still works. However, perhaps the original JSON string retrieved from the command has some different characters then when it's converted to UTF-8 and back.\r\n\r\nWe can test whether that's the case if you manage to obtain one image that errors, or you can rescue `JSON::ParserError` and `File.binwrite(\"tmp/data.json\", MiniMagick::Tool::Convert.new { |c| c << path << \"json:\" })`, and afterwards you can manually read it and try to parse it, first with `JSON.parse(File.binread(\"tmp/data.json\"))` and then with `JSON.parse(File.binread(\"tmp/data.json\").force_encoding(Encoding::UTF_8))` or just `JSON.parse(File.read(\"tmp/data.json\"))`.\r\n\r\nIf the bug does turn out to be caused by encoding, I think a good solution would be for MiniMagick to `force_encoding(Encoding::UTF_8)` the return JSON string. That command should never fail, because it doesn't modify the bytes (unlike `String#encode`), it rather just makes Ruby treat it with a different encoding."}}
{"actor":{"display_login":"janko-m"}}