{"action":"created","issue":{"url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/issues/147","repository_url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis","labels_url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/issues/147/labels{/name}","comments_url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/issues/147/comments","events_url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/issues/147/events","html_url":"https://github.com/MPAS-Dev/MPAS-Analysis/pull/147","id":215834325,"number":147,"title":"Add Remapper class to do remapping internally in python","user":{"login":"xylar","id":4179064,"avatar_url":"https://avatars3.githubusercontent.com/u/4179064?v=3","gravatar_id":"","url":"https://api.github.com/users/xylar","html_url":"https://github.com/xylar","followers_url":"https://api.github.com/users/xylar/followers","following_url":"https://api.github.com/users/xylar/following{/other_user}","gists_url":"https://api.github.com/users/xylar/gists{/gist_id}","starred_url":"https://api.github.com/users/xylar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xylar/subscriptions","organizations_url":"https://api.github.com/users/xylar/orgs","repos_url":"https://api.github.com/users/xylar/repos","events_url":"https://api.github.com/users/xylar/events{/privacy}","received_events_url":"https://api.github.com/users/xylar/received_events","type":"User","site_admin":false},"labels":[{"id":396974348,"url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/labels/enhancement","name":"enhancement","color":"84b6eb","default":true},{"id":560292568,"url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/labels/priority","name":"priority","color":"ffb405","default":false}],"state":"open","locked":false,"assignee":{"login":"milenaveneziani","id":8237579,"avatar_url":"https://avatars2.githubusercontent.com/u/8237579?v=3","gravatar_id":"","url":"https://api.github.com/users/milenaveneziani","html_url":"https://github.com/milenaveneziani","followers_url":"https://api.github.com/users/milenaveneziani/followers","following_url":"https://api.github.com/users/milenaveneziani/following{/other_user}","gists_url":"https://api.github.com/users/milenaveneziani/gists{/gist_id}","starred_url":"https://api.github.com/users/milenaveneziani/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/milenaveneziani/subscriptions","organizations_url":"https://api.github.com/users/milenaveneziani/orgs","repos_url":"https://api.github.com/users/milenaveneziani/repos","events_url":"https://api.github.com/users/milenaveneziani/events{/privacy}","received_events_url":"https://api.github.com/users/milenaveneziani/received_events","type":"User","site_admin":false},"assignees":[{"login":"milenaveneziani","id":8237579,"avatar_url":"https://avatars2.githubusercontent.com/u/8237579?v=3","gravatar_id":"","url":"https://api.github.com/users/milenaveneziani","html_url":"https://github.com/milenaveneziani","followers_url":"https://api.github.com/users/milenaveneziani/followers","following_url":"https://api.github.com/users/milenaveneziani/following{/other_user}","gists_url":"https://api.github.com/users/milenaveneziani/gists{/gist_id}","starred_url":"https://api.github.com/users/milenaveneziani/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/milenaveneziani/subscriptions","organizations_url":"https://api.github.com/users/milenaveneziani/orgs","repos_url":"https://api.github.com/users/milenaveneziani/repos","events_url":"https://api.github.com/users/milenaveneziani/events{/privacy}","received_events_url":"https://api.github.com/users/milenaveneziani/received_events","type":"User","site_admin":false}],"milestone":null,"comments":20,"created_at":"2017-03-21T18:21:46Z","updated_at":"2017-04-26T13:41:08Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/pulls/147","html_url":"https://github.com/MPAS-Dev/MPAS-Analysis/pull/147","diff_url":"https://github.com/MPAS-Dev/MPAS-Analysis/pull/147.diff","patch_url":"https://github.com/MPAS-Dev/MPAS-Analysis/pull/147.patch"},"body":"This PR includes some fairly significant reorganization of the interpolation code needed to support grids not currently handled by `ncremap` and to make it easier to introduce new projections in the future.\r\n\r\n  * The code for creating SCRIP files has been reorganized to consist of 3 grid descriptor classes, one for MPAS meshes, one for lat-lon grids and one for general projections supported with the `pyproj` library (notably stereographic grids commonly used for polar observations and modeling).  These classes now live in `shared/grid/grid.py`\r\n  * The interpolation code has been updated so mapping files are created with a source and destination grid descriptor, allowing much more flexibility than before.\r\n  * All interpolation is now done through a `Remapper` class, allowing the option of remapping fields directly in python in addition to  the usual offline remapping using `ncremap`.  A 'Remapper` object can be initialized once (either with an existing mapping file name or creating a new mapping file) and then reused to remap any number of data sets or data arrays between the same source and destination grids.\r\n  * The `climatology` module:\r\n    * has been updated to use the new interpolation capabilities.  Rather than just writing a mapping file, now climatology will produce a remapper between MPAS or observation grids and a comparison grid (using `get_remapper`).  A new function, `remap_and_write_climatology`, has been added to do remapping of a climatology data set either \"online\" in memory or \"offline\" with `ncremap`, depending on a config option.\r\n    * has also been updated to support multiple comparison grids in anticipation of analysis on a polar stereographic grid.  The comparison grid is defined by a `MeshDescriptor` object. Currently only the global lat/lon comparison grid is supported via the `get_lat_lon_comarison_descriptor` function.\r\n * The 2 `modelvsobs` tasks have been updated to use the `get_remapper` and `remap_and_write_climatology` functions.\r\n * Ocean `modelvsobs` has been broken into several helper functions that initialize observations, plot titles, etc. for each field.\r\n * Two new config options, `useNcremap` and `normalizationThreshold` have been introduced in `[climatology]` to control whether `ncreamp` or the `Remapper` are used to do remapping and to determine the minimum weight of a destination cell before it gets masked out as part of renormalization (if the `Remapper` is used)."},"comment":{"url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/issues/comments/297411994","html_url":"https://github.com/MPAS-Dev/MPAS-Analysis/pull/147#issuecomment-297411994","issue_url":"https://api.github.com/repos/MPAS-Dev/MPAS-Analysis/issues/147","id":297411994,"user":{"login":"milenaveneziani","id":8237579,"avatar_url":"https://avatars2.githubusercontent.com/u/8237579?v=3","gravatar_id":"","url":"https://api.github.com/users/milenaveneziani","html_url":"https://github.com/milenaveneziani","followers_url":"https://api.github.com/users/milenaveneziani/followers","following_url":"https://api.github.com/users/milenaveneziani/following{/other_user}","gists_url":"https://api.github.com/users/milenaveneziani/gists{/gist_id}","starred_url":"https://api.github.com/users/milenaveneziani/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/milenaveneziani/subscriptions","organizations_url":"https://api.github.com/users/milenaveneziani/orgs","repos_url":"https://api.github.com/users/milenaveneziani/repos","events_url":"https://api.github.com/users/milenaveneziani/events{/privacy}","received_events_url":"https://api.github.com/users/milenaveneziani/received_events","type":"User","site_admin":false},"created_at":"2017-04-26T13:41:08Z","updated_at":"2017-04-26T13:41:08Z","body":"I tested this on the 18to6 for computing 1-year climatologies on rhea (the 18to6 G-case was run on titan), and timed run_analysis with and without useNcremap. Here is the result:\r\n1) with ncremap it took ~2400 sec (40 min)\r\n2) without ncremap it took more than double the time, ~5580 sec (1.5 hr)\r\nSo, my recommendation would be to set `useNcremap=True` as default and then make sure that the interpolations to polar stereographic grid are handled by python. Does this sound reasonable?"}}
{"actor":{"display_login":"milenaveneziani"}}