{"push_id":2066366137,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"1be67241f8612e74eb08d9ea2aa05b1d7fc94b6e","before":"b2b7c898173c72091e50346aaaaf7886a7e6aa4b","commits":[{"sha":"1be67241f8612e74eb08d9ea2aa05b1d7fc94b6e","author":{"name":"Kit Rodolfa","email":"fef5bf6ca48924db59bcac75fc7fded1e3116d52@gmail.com"},"message":"Imputation (Closes #99) (#101)\n\nProvides for a number of options for imputing null values, including mean imputation, constant value imputation, zero-filling, modal value for binaries, null categories for categoricals, and raising an exception when nulls are encountered (if data are expected to be clean).\r\n\r\nA couple of notes:\r\n\r\n    Specifying imputation rules is now required for using collate aggregates\r\n    The aggregation objects now need to be passed a state table with a comprehensive set of entities and dates\r\n    For aggregates, an \"imputed\" column is generated to flag NULL values that were filled; for categoricals, this is handled by including a NULL category\r\n    Because the imputation code needs to both identify columns in the database containing NULL values as well as look up the imputation type from the column name, column name truncation will cause the code to break. This can be fixed by hashing column names and maintaining a lookup of full column names (potentially as well as human-readable versions)\r\n\r\nIn the future, we'll likely want to add additional imputation options (e.g., within-group mean, most frequent category for categoricals, most frequent pattern for categoricals, multiple imputation, etc.), but this will take a bit of extra work. Additionally, we may want to allow for overriding the generation of an imputed flag, for instance, in cases where zero-filling is meaningful (e.g., for count features created from a table of arrests, entities not in the table have no arrests, so 0 is in fact the correct value).","distinct":true,"url":"https://api.github.com/repos/dssg/collate/commits/1be67241f8612e74eb08d9ea2aa05b1d7fc94b6e"}]}
{"actor":{"display_login":"thcrock"}}