{"push_id":2050881026,"size":1,"distinct_size":1,"ref":"refs/heads/develop","head":"bb58c5858a486b2548e304560f644bf8c9880d4d","before":"a972c791c53a7baa37f21296a7925279e5a65104","commits":[{"sha":"bb58c5858a486b2548e304560f644bf8c9880d4d","author":{"name":"Glenn Sheasby","email":"f3c8cc65b3d7708b91d8a7f5819a8b289867dc95@palantir.com"},"message":"Feature/sweep rewrite (#2436)\n\n* The big sweep rewrite, part 3: Postgres impl (#2021)\r\n\r\n* The big sweep rewrite, part 3: Postgres impl\r\n\r\n- Implement KeyValueService.getCandidateCellsForSweeping() for Postgres\r\n\r\n- Add AgnosticResultRow.getBoolean() and LightAgnosticResultRow.getArray()\r\n\r\n* Address CR comments\r\n\r\nAlso change the stopping condition in computeNext()\r\n\r\n* Add a release note\r\n\r\n* Add the ORDER BY that I forgot\r\n\r\n* Rewrite a fragile test\r\n\r\n* Join pages based on the number of (cell, ts) pairs\r\n\r\n* Move release note to develop\r\n\r\n* The big sweep rewrite, part 4: Cassandra impl (#2231)\r\n\r\n* The big sweep rewrite, part 4: Cassandra impl\r\n\r\nIterate over cells in a table without ever loading more than a given\r\nnumber at once regardless of how wide the rows are, but still perform\r\nefficiently on narrow tables.\r\n\r\nThis should finally allow us to run sweep safely on\r\nCassandra with decent performance.\r\n\r\n* Address clockfort's comments + checkstyle + add release notes\r\n\r\n* Address code review comments\r\n\r\n* Split up test cases in SingleRowColumnPagerTest\r\n\r\n* Tests for KeyRanges\r\n\r\n* Update locks\r\n\r\n* Return Optional instead of passing in @Output list\r\n\r\n* Update dependency locks\r\n\r\n* Fix TestDataBuilder after merge\r\n\r\n* Make tests green\r\n\r\n- Fix a small bug in CassandraGetCandidateCellsForSweepingImpl\r\n\r\n- Remove an irrelevant test (should_not_oom_when_...) and make\r\n  CassKVSeSweepTaskRunnerIntegrationTest non-parameterized.\r\n  (getRangeOfTimestamps() is not used in the sweep code path anymore)\r\n\r\n- Relax assertions in testSweepBatchesUpToDeleteBatchSize\r\n\r\n* Bring back the old variable name to make checkstyle happy\r\n\r\n* Run ./gradlew generateLock saveLock\r\n\r\n* Reapply changes\r\n\r\n* Fix compile errors\r\n\r\n* Simplify getCandidateCellsForSweeping() (#2439)\r\n\r\n* Simplify getCandidateCellsForSweeping()\r\n\r\nInitially I hoped to take advantage of in-database filtering if we get\r\nto implement the \"transaction table sweeping\" feature. However, seems\r\nlike that wasn't a great decision on my part - unclear when (and if) we\r\nget that feature done, and also how much of improvement we would\r\nactually get. The extra logic makes the code significantly more complex,\r\nso I think we need to back off and give up the idea.\r\n\r\nThis doesn't invalidate the sweep rewrite project. The new impls still\r\nbring significant performance improvements.\r\n\r\n* The big sweep rewrite, part 5: Oracle impl\r\n\r\n- Extract the paging logic from the Postgres impl into a separate class\r\n  named CandidatePagingState so that we could reuse it in the Oracle impl\r\n  (and also test the logic with unit tests)\r\n\r\n- Add a new benchmark fullTableScanOneWideRowThorough that simulates\r\n  sweep of a very wide row. This is important for Oracle - a naive\r\n  implementation would perform extremely poorly in that scenario.\r\n\r\n* A bunch of refactors\r\n\r\n- Make a builder for FullQuery\r\n\r\n- Rename RangeBoundPredicates -> RangePredicateHelper and make it\r\n  append to a FullQuery.Builder instead of returning a string + args.\r\n\r\n- Make DbKvsGetCandidateCellsForSweeping a concrete class with common\r\n  logic. Now, database-specific logic is behind the CellTsPairLoader\r\n  interface.\r\n\r\n- Add full test coverage for CandidatePagingState\r\n\r\n* Tautological tests for RangePredicateHelper\r\n\r\n* Import the \"correct\" assertThat\r\n\r\n* Static classes\r\n\r\n* Refactors\r\n\r\n* CandidatePagingState: extract state-updating methods\r\n\r\nAlso, reorganised class\r\n\r\n* Expand the comment in OracleCellTsPageLoaderFactory\r\n\r\nAlso filter out empty pages\r\n\r\n* Fix release notes\r\n\r\n* Fix release notes for real","distinct":true,"url":"https://api.github.com/repos/palantir/atlasdb/commits/bb58c5858a486b2548e304560f644bf8c9880d4d"}]}
{"actor":{"display_login":"dxiao"}}