{"action":"created","issue":{"url":"https://api.github.com/repos/Unidata/netcdf-c/issues/657","repository_url":"https://api.github.com/repos/Unidata/netcdf-c","labels_url":"https://api.github.com/repos/Unidata/netcdf-c/issues/657/labels{/name}","comments_url":"https://api.github.com/repos/Unidata/netcdf-c/issues/657/comments","events_url":"https://api.github.com/repos/Unidata/netcdf-c/issues/657/events","html_url":"https://github.com/Unidata/netcdf-c/issues/657","id":274640147,"number":657,"title":"Invalid argument error with libnetcdf 4.5.0 but not 4.4.1","user":{"login":"spencerkclark","id":6628425,"avatar_url":"https://avatars0.githubusercontent.com/u/6628425?v=4","gravatar_id":"","url":"https://api.github.com/users/spencerkclark","html_url":"https://github.com/spencerkclark","followers_url":"https://api.github.com/users/spencerkclark/followers","following_url":"https://api.github.com/users/spencerkclark/following{/other_user}","gists_url":"https://api.github.com/users/spencerkclark/gists{/gist_id}","starred_url":"https://api.github.com/users/spencerkclark/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spencerkclark/subscriptions","organizations_url":"https://api.github.com/users/spencerkclark/orgs","repos_url":"https://api.github.com/users/spencerkclark/repos","events_url":"https://api.github.com/users/spencerkclark/events{/privacy}","received_events_url":"https://api.github.com/users/spencerkclark/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"WardF","id":4324946,"avatar_url":"https://avatars3.githubusercontent.com/u/4324946?v=4","gravatar_id":"","url":"https://api.github.com/users/WardF","html_url":"https://github.com/WardF","followers_url":"https://api.github.com/users/WardF/followers","following_url":"https://api.github.com/users/WardF/following{/other_user}","gists_url":"https://api.github.com/users/WardF/gists{/gist_id}","starred_url":"https://api.github.com/users/WardF/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WardF/subscriptions","organizations_url":"https://api.github.com/users/WardF/orgs","repos_url":"https://api.github.com/users/WardF/repos","events_url":"https://api.github.com/users/WardF/events{/privacy}","received_events_url":"https://api.github.com/users/WardF/received_events","type":"User","site_admin":false},"assignees":[{"login":"WardF","id":4324946,"avatar_url":"https://avatars3.githubusercontent.com/u/4324946?v=4","gravatar_id":"","url":"https://api.github.com/users/WardF","html_url":"https://github.com/WardF","followers_url":"https://api.github.com/users/WardF/followers","following_url":"https://api.github.com/users/WardF/following{/other_user}","gists_url":"https://api.github.com/users/WardF/gists{/gist_id}","starred_url":"https://api.github.com/users/WardF/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WardF/subscriptions","organizations_url":"https://api.github.com/users/WardF/orgs","repos_url":"https://api.github.com/users/WardF/repos","events_url":"https://api.github.com/users/WardF/events{/privacy}","received_events_url":"https://api.github.com/users/WardF/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/Unidata/netcdf-c/milestones/9","html_url":"https://github.com/Unidata/netcdf-c/milestone/9","labels_url":"https://api.github.com/repos/Unidata/netcdf-c/milestones/9/labels","id":2410164,"number":9,"title":"4.5.1","description":"Issues related to the 4.4.3 release of netCDF-C","creator":{"login":"WardF","id":4324946,"avatar_url":"https://avatars3.githubusercontent.com/u/4324946?v=4","gravatar_id":"","url":"https://api.github.com/users/WardF","html_url":"https://github.com/WardF","followers_url":"https://api.github.com/users/WardF/followers","following_url":"https://api.github.com/users/WardF/following{/other_user}","gists_url":"https://api.github.com/users/WardF/gists{/gist_id}","starred_url":"https://api.github.com/users/WardF/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WardF/subscriptions","organizations_url":"https://api.github.com/users/WardF/orgs","repos_url":"https://api.github.com/users/WardF/repos","events_url":"https://api.github.com/users/WardF/events{/privacy}","received_events_url":"https://api.github.com/users/WardF/received_events","type":"User","site_admin":false},"open_issues":19,"closed_issues":24,"state":"open","created_at":"2017-03-24T21:33:15Z","updated_at":"2017-11-17T00:16:18Z","due_on":null,"closed_at":null},"comments":10,"created_at":"2017-11-16T19:44:45Z","updated_at":"2017-11-17T16:42:31Z","closed_at":null,"author_association":"NONE","body":"Per the recommendation of @dopplershift (cc: @spencerahill), I am migrating an issue (Unidata/netcdf4-python#742) initially posted in netcdf4-python over to here.  \r\n\r\n### Environment Information\r\n\r\n- I have reproduced this behavior on my local machine (using Mac OS version 10.13)\r\n- This behavior was first identified with our test suite run using Travis CI (see [here](https://travis-ci.org/spencerahill/aospy/jobs/302309349#L6) for the platform information)\r\n- On both platforms `libnetcdf` was installed from the conda-forge channel (e.g. `conda install -c conda-forge libnetcdf`).\r\n\r\n### Summary of Issue\r\n\r\nSince `libnetcdf` version 4.5.0 appeared on conda-forge, we have encountered some puzzling errors in our CI test suite.  Namely, some files that we used to be able to open with xarray using netCDF4 now lead to errors like the following:\r\n```\r\naospy/test/test_calc_basic.py:60: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\naospy/test/test_calc_basic.py:35: in _test_files_and_attrs\r\n    _test_output_attrs(calc, dtype_out)\r\naospy/test/test_calc_basic.py:18: in _test_output_attrs\r\n    data = xr.open_dataset(calc.path_out[dtype_out])\r\n../../../miniconda/envs/test_env/lib/python3.6/site-packages/xarray/backends/api.py:282: in open_dataset\r\n    autoclose=autoclose)\r\n../../../miniconda/envs/test_env/lib/python3.6/site-packages/xarray/backends/netCDF4_.py:210: in __init__\r\n    self.ds = opener()\r\n../../../miniconda/envs/test_env/lib/python3.6/site-packages/xarray/backends/netCDF4_.py:185: in _open_netcdf4_group\r\n    ds = nc4.Dataset(filename, mode=mode, **kwargs)\r\nnetCDF4/_netCDF4.pyx:2015: in netCDF4._netCDF4.Dataset.__init__\r\n    ???\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n>   ???\r\nE   OSError: [Errno -36] NetCDF: Invalid argument: b'/home/travis/build/spencerahill/aospy/aospy/test/data/objects/test-files/example_proj/example_model/example_run/condensation_rain/condensation_rain.ann.av.from_monthly_ts.example_model.example_run.0004-0006.nc'\r\n```\r\nIn doing some investigation offline, it seems that the command line tool `ncdump` provided by `libnetcdf` version 4.5.0 is also unable to open these files.  E.g. I get the following response:\r\n```\r\n$ ncdump sphum.ann.reg.av.vert_int.from_monthly_ts_sigma.example_model.example_run.0006.nc\r\nncdump: sphum.ann.reg.av.vert_int.from_monthly_ts_sigma.example_model.example_run.0006.nc: NetCDF: Invalid argument\r\n```\r\nHowever, using `libnetcdf` version 4.4.1.1, `ncdump` is able to handle the file (and using version 4.4.1.1 also enables downstream tools like netcdf4-python to work):\r\n```\r\n$ ncdump -h sphum.ann.reg.av.vert_int.from_monthly_ts_sigma.example_model.example_run.0006.nc\r\nnetcdf sphum.ann.reg.av.vert_int.from_monthly_ts_sigma.example_model.example_run.0006 {\r\nvariables:\r\n\tdouble raw_data_start_date ;\r\n\t\traw_data_start_date:units = \"days since 1673-01-01\" ;\r\n\t\traw_data_start_date:long_name = \"time axis boundaries\" ;\r\n\t\traw_data_start_date:calendar = \"NOLEAP\" ;\r\n\t\traw_data_start_date:_FillValue = 1.e+20 ;\r\n\tint subset_end_date ;\r\n\t\tsubset_end_date:units = \"days since 1678-01-31 00:00:00\" ;\r\n\t\tsubset_end_date:calendar = \"proleptic_gregorian\" ;\r\n\tdouble globe ;\r\n\t\tglobe:units = \"(vertical integral of quantity with unspecified units)\" ;\r\n\t\tglobe:_FillValue = NaN ;\r\n\t\tglobe:description = \"specific humidity\" ;\r\n\tdouble raw_data_end_date ;\r\n\t\traw_data_end_date:units = \"days since 1673-01-01\" ;\r\n\t\traw_data_end_date:long_name = \"time axis boundaries\" ;\r\n\t\traw_data_end_date:calendar = \"NOLEAP\" ;\r\n\t\traw_data_end_date:_FillValue = 1.e+20 ;\r\n\tint subset_start_date ;\r\n\t\tsubset_start_date:units = \"days since 1678-01-01 00:00:00\" ;\r\n\t\tsubset_start_date:calendar = \"proleptic_gregorian\" ;\r\n\r\n// global attributes:\r\n\t\t:coordinates = \"raw_data_start_date subset_end_date raw_data_end_date subset_start_date\" ;\r\n}\r\n```\r\nI have posted a [link to this file as an example](http://www.princeton.edu/~skclark/sphum.ann.reg.av.vert_int.from_monthly_ts_sigma.example_model.example_run.0006.nc).  It would be helpful to know if you can reproduce the issues we've encountered here with files like the one provided, and what about them led to this problem.  Thanks in advance for your help!\r\n\r\n### Steps to reproduce the behavior\r\n\r\nUsing the latest version of `libnetcdf` from the conda-forge channel, [download the linked file](http://www.princeton.edu/~skclark/sphum.ann.reg.av.vert_int.from_monthly_ts_sigma.example_model.example_run.0006.nc) and try to run `ncdump`."},"comment":{"url":"https://api.github.com/repos/Unidata/netcdf-c/issues/comments/345296151","html_url":"https://github.com/Unidata/netcdf-c/issues/657#issuecomment-345296151","issue_url":"https://api.github.com/repos/Unidata/netcdf-c/issues/657","id":345296151,"user":{"login":"wkliao","id":1634098,"avatar_url":"https://avatars2.githubusercontent.com/u/1634098?v=4","gravatar_id":"","url":"https://api.github.com/users/wkliao","html_url":"https://github.com/wkliao","followers_url":"https://api.github.com/users/wkliao/followers","following_url":"https://api.github.com/users/wkliao/following{/other_user}","gists_url":"https://api.github.com/users/wkliao/gists{/gist_id}","starred_url":"https://api.github.com/users/wkliao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wkliao/subscriptions","organizations_url":"https://api.github.com/users/wkliao/orgs","repos_url":"https://api.github.com/users/wkliao/repos","events_url":"https://api.github.com/users/wkliao/events{/privacy}","received_events_url":"https://api.github.com/users/wkliao/received_events","type":"User","site_admin":false},"created_at":"2017-11-17T16:42:31Z","updated_at":"2017-11-17T16:42:31Z","author_association":"CONTRIBUTOR","body":"Forgot to mention that there is a PnetCDF utility program named ncmpivalid. It can be used to validate a CDF file and report the exact location that fails to conform with the file specification.\r\nhttps://trac.mcs.anl.gov/projects/parallel-netcdf/browser/tags/v1-9-0-pre1/src/utils/ncmpivalid"}}
{"actor":{"display_login":"wkliao"}}