{"action":"created","issue":{"url":"https://api.github.com/repos/STScI-JWST/jwst/issues/1565","repository_url":"https://api.github.com/repos/STScI-JWST/jwst","labels_url":"https://api.github.com/repos/STScI-JWST/jwst/issues/1565/labels{/name}","comments_url":"https://api.github.com/repos/STScI-JWST/jwst/issues/1565/comments","events_url":"https://api.github.com/repos/STScI-JWST/jwst/issues/1565/events","html_url":"https://github.com/STScI-JWST/jwst/issues/1565","id":279168335,"number":1565,"title":"extract1d tables in ASDF extension","user":{"login":"hbushouse","id":12860084,"avatar_url":"https://avatars1.githubusercontent.com/u/12860084?v=4","gravatar_id":"","url":"https://api.github.com/users/hbushouse","html_url":"https://github.com/hbushouse","followers_url":"https://api.github.com/users/hbushouse/followers","following_url":"https://api.github.com/users/hbushouse/following{/other_user}","gists_url":"https://api.github.com/users/hbushouse/gists{/gist_id}","starred_url":"https://api.github.com/users/hbushouse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hbushouse/subscriptions","organizations_url":"https://api.github.com/users/hbushouse/orgs","repos_url":"https://api.github.com/users/hbushouse/repos","events_url":"https://api.github.com/users/hbushouse/events{/privacy}","received_events_url":"https://api.github.com/users/hbushouse/received_events","type":"User","site_admin":false},"labels":[{"id":390252307,"url":"https://api.github.com/repos/STScI-JWST/jwst/labels/datamodels","name":"datamodels","color":"d4c5f9","default":false},{"id":390242650,"url":"https://api.github.com/repos/STScI-JWST/jwst/labels/extract_1d","name":"extract_1d","color":"0052cc","default":false}],"state":"closed","locked":false,"assignee":{"login":"philhodge","id":2065676,"avatar_url":"https://avatars2.githubusercontent.com/u/2065676?v=4","gravatar_id":"","url":"https://api.github.com/users/philhodge","html_url":"https://github.com/philhodge","followers_url":"https://api.github.com/users/philhodge/followers","following_url":"https://api.github.com/users/philhodge/following{/other_user}","gists_url":"https://api.github.com/users/philhodge/gists{/gist_id}","starred_url":"https://api.github.com/users/philhodge/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philhodge/subscriptions","organizations_url":"https://api.github.com/users/philhodge/orgs","repos_url":"https://api.github.com/users/philhodge/repos","events_url":"https://api.github.com/users/philhodge/events{/privacy}","received_events_url":"https://api.github.com/users/philhodge/received_events","type":"User","site_admin":false},"assignees":[{"login":"philhodge","id":2065676,"avatar_url":"https://avatars2.githubusercontent.com/u/2065676?v=4","gravatar_id":"","url":"https://api.github.com/users/philhodge","html_url":"https://github.com/philhodge","followers_url":"https://api.github.com/users/philhodge/followers","following_url":"https://api.github.com/users/philhodge/following{/other_user}","gists_url":"https://api.github.com/users/philhodge/gists{/gist_id}","starred_url":"https://api.github.com/users/philhodge/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philhodge/subscriptions","organizations_url":"https://api.github.com/users/philhodge/orgs","repos_url":"https://api.github.com/users/philhodge/repos","events_url":"https://api.github.com/users/philhodge/events{/privacy}","received_events_url":"https://api.github.com/users/philhodge/received_events","type":"User","site_admin":false},{"login":"bernie-simon","id":7916041,"avatar_url":"https://avatars0.githubusercontent.com/u/7916041?v=4","gravatar_id":"","url":"https://api.github.com/users/bernie-simon","html_url":"https://github.com/bernie-simon","followers_url":"https://api.github.com/users/bernie-simon/followers","following_url":"https://api.github.com/users/bernie-simon/following{/other_user}","gists_url":"https://api.github.com/users/bernie-simon/gists{/gist_id}","starred_url":"https://api.github.com/users/bernie-simon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bernie-simon/subscriptions","organizations_url":"https://api.github.com/users/bernie-simon/orgs","repos_url":"https://api.github.com/users/bernie-simon/repos","events_url":"https://api.github.com/users/bernie-simon/events{/privacy}","received_events_url":"https://api.github.com/users/bernie-simon/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/STScI-JWST/jwst/milestones/6","html_url":"https://github.com/STScI-JWST/jwst/milestone/6","labels_url":"https://api.github.com/repos/STScI-JWST/jwst/milestones/6/labels","id":2946419,"number":6,"title":"Build 7.1.1","description":"Patch build. Currently due to I&T 5/25/2018.","creator":{"login":"hbushouse","id":12860084,"avatar_url":"https://avatars1.githubusercontent.com/u/12860084?v=4","gravatar_id":"","url":"https://api.github.com/users/hbushouse","html_url":"https://github.com/hbushouse","followers_url":"https://api.github.com/users/hbushouse/followers","following_url":"https://api.github.com/users/hbushouse/following{/other_user}","gists_url":"https://api.github.com/users/hbushouse/gists{/gist_id}","starred_url":"https://api.github.com/users/hbushouse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hbushouse/subscriptions","organizations_url":"https://api.github.com/users/hbushouse/orgs","repos_url":"https://api.github.com/users/hbushouse/repos","events_url":"https://api.github.com/users/hbushouse/events{/privacy}","received_events_url":"https://api.github.com/users/hbushouse/received_events","type":"User","site_admin":false},"open_issues":6,"closed_issues":12,"state":"open","created_at":"2017-11-29T12:14:10Z","updated_at":"2017-12-11T17:54:45Z","due_on":"2018-05-25T07:00:00Z","closed_at":null},"comments":4,"created_at":"2017-12-04T22:07:36Z","updated_at":"2017-12-11T22:48:07Z","closed_at":"2017-12-06T21:17:34Z","author_association":"CONTRIBUTOR","body":"Jeff Valenti reports:\r\n```\r\nI've been learning about ASDF in the context of our JWST pipeline products. For DMS 7.1, \r\nx1d and x1dints files contain are two copies of each extracted spectrum:\r\n \r\nThere is one EXTRACT1D extension for each spectrum.\r\nIn the ASDF extension, there is a separate tree object and corresponding data block \r\nfor each spectrum.\r\n \r\nFor cases with lots of spectra (e.g., multi-object spectroscopy, time series), this \r\ndoubling of file size can be significant.\r\n```\r\nThere should NOT be a separate copy of each extract1d data table in the ASDF extension. Perhaps this is a bug in datamodels that is accidentally allowing the table to appear in ASDF?"},"comment":{"url":"https://api.github.com/repos/STScI-JWST/jwst/issues/comments/350884132","html_url":"https://github.com/STScI-JWST/jwst/issues/1565#issuecomment-350884132","issue_url":"https://api.github.com/repos/STScI-JWST/jwst/issues/1565","id":350884132,"user":{"login":"bernie-simon","id":7916041,"avatar_url":"https://avatars0.githubusercontent.com/u/7916041?v=4","gravatar_id":"","url":"https://api.github.com/users/bernie-simon","html_url":"https://github.com/bernie-simon","followers_url":"https://api.github.com/users/bernie-simon/followers","following_url":"https://api.github.com/users/bernie-simon/following{/other_user}","gists_url":"https://api.github.com/users/bernie-simon/gists{/gist_id}","starred_url":"https://api.github.com/users/bernie-simon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bernie-simon/subscriptions","organizations_url":"https://api.github.com/users/bernie-simon/orgs","repos_url":"https://api.github.com/users/bernie-simon/repos","events_url":"https://api.github.com/users/bernie-simon/events{/privacy}","received_events_url":"https://api.github.com/users/bernie-simon/received_events","type":"User","site_admin":false},"created_at":"2017-12-11T22:48:07Z","updated_at":"2017-12-11T22:48:07Z","author_association":"CONTRIBUTOR","body":"The new function is _snip_tables in fits_support.py\r\n\r\n```python\r\ndef _snip_tables(tree):\r\n    def _snip_node(node, json_id):\r\n        if isinstance(node, np.ndarray):\r\n            dtype = node.dtype\r\n            if hasattr(dtype, 'names'):\r\n                node = None\r\n        return node\r\n    return treeutil.walk_and_modify(tree, _snip_node)\r\n\r\ndef to_fits(tree, schema, extensions=None):\r\n    hdulist = fits.HDUList()\r\n    hdulist.append(fits.PrimaryHDU())\r\n\r\n    _save_from_schema(hdulist, tree, schema)\r\n    _save_extra_fits(hdulist, tree)\r\n    _save_history(hdulist, tree)\r\n    tree = _snip_tables(tree)\r\n\r\n    asdf = fits_embed.AsdfInFits(hdulist, tree, extensions=extensions)\r\n    return asdf\r\n```"}}
{"actor":{"display_login":"bernie-simon"}}