{"push_id":1544116700,"size":1,"distinct_size":1,"ref":"refs/heads/too-big","head":"800099f015082412612e74dab138f8bef603711f","before":"b4d6f4e13a0fdb431e3ba2abd6add633a70f5bd6","commits":[{"sha":"800099f015082412612e74dab138f8bef603711f","author":{"name":"Chris Masone","email":"e2448888308b5eeabed3cefb3cc2b30e1820038e@atticlabs.io"},"message":"Gather more info about Bug 3156\n\nThere's some case that causes chunks that compress to more than about\n55k (we think these are quite big, chunks that are many hundreds of K\nin size) not to wind up correctly inserted into tables. It looks like\nthe snappy library believes the buffer we've allocated may not be\nlarge enough, so it allocates its own space and this screws us up.\n\nThis patch changes two things:\n1) The CRC in the NBS format is now the CRC of the _compressed_ data\n2) Such chunks will be manually copied into the table, so they won't\n   be missing anymore\n\nAlso, when the code detects a case where the snappy library decided to\nallocate its own storage, it saves the uncompressed data off to the\nside, so that it can be pushed to durable storage. Such chunks are\nstored on disk or in S3 named like \"<chunk-hash>-errata\", and logging\nis dumped out so we can figure out which tables were supposed to\ncontain these chunks.\n\nTowards #3156","distinct":true,"url":"https://api.github.com/repos/cmasone-attic/noms/commits/800099f015082412612e74dab138f8bef603711f"}]}
{"actor":{"display_login":"cmasone-attic"}}