{"action":"created","issue":{"url":"https://api.github.com/repos/rust-lang/rust/issues/40858","repository_url":"https://api.github.com/repos/rust-lang/rust","labels_url":"https://api.github.com/repos/rust-lang/rust/issues/40858/labels{/name}","comments_url":"https://api.github.com/repos/rust-lang/rust/issues/40858/comments","events_url":"https://api.github.com/repos/rust-lang/rust/issues/40858/events","html_url":"https://github.com/rust-lang/rust/issues/40858","id":217315454,"number":40858,"title":"Slow compilation on large project when nothing changes","user":{"login":"jonhoo","id":176295,"avatar_url":"https://avatars1.githubusercontent.com/u/176295?v=3","gravatar_id":"","url":"https://api.github.com/users/jonhoo","html_url":"https://github.com/jonhoo","followers_url":"https://api.github.com/users/jonhoo/followers","following_url":"https://api.github.com/users/jonhoo/following{/other_user}","gists_url":"https://api.github.com/users/jonhoo/gists{/gist_id}","starred_url":"https://api.github.com/users/jonhoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonhoo/subscriptions","organizations_url":"https://api.github.com/users/jonhoo/orgs","repos_url":"https://api.github.com/users/jonhoo/repos","events_url":"https://api.github.com/users/jonhoo/events{/privacy}","received_events_url":"https://api.github.com/users/jonhoo/received_events","type":"User","site_admin":false},"labels":[{"id":307747675,"url":"https://api.github.com/repos/rust-lang/rust/labels/A-incr-comp","name":"A-incr-comp","color":"d7e012","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2017-03-27T17:18:16Z","updated_at":"2017-03-28T14:34:25Z","closed_at":null,"body":"I'm working on a [decently large](https://github.com/mit-pdos/distributary/) Rust project, and have lately found compilation times (especially for `cargo test`) to be excruciatingly slow. In particular, from a complete build, followed by a `touch src/lib.rs` takes ~80s with the latest nightly and `CARGO_INCREMENTAL=1`:\r\n\r\n```console\r\n$ env | grep CARGO\r\nCARGO_INCREMENTAL=1\r\n$ git clone https://github.com/mit-pdos/distributary.git\r\n$ git checkout 2fd16356a15ab3b0fbb7dbd9cbc85add984251b7 # HEAD has a workaround\r\n$ cd distributary\r\n$ cargo --version\r\ncargo 0.19.0-nightly (c995e9eb5 2017-03-17)\r\n$ rustc --version\r\nrustc 1.17.0-nightly (7846dbe0c 2017-03-26)\r\n$ cargo clean\r\n$ cargo t --no-run\r\n   Compiling nom v1.2.4\r\n   Compiling unicode-normalization v0.1.4\r\n   Compiling serde v0.9.11\r\n...\r\n   Compiling distributary v0.1.0 (file:///home/jon/dev/projects/distributary)\r\n    Finished dev [unoptimized + debuginfo] target(s) in 362.66 secs\r\n$ cargo t --no-run\r\n    Finished dev [unoptimized + debuginfo] target(s) in 0.1 secs\r\n$ touch src/lib.rs\r\n$ cargo t --no-run\r\n   Compiling distributary v0.1.0 (file:///home/jon/dev/projects/distributary)\r\n    Finished dev [unoptimized + debuginfo] target(s) in 80.84 secs\r\n```\r\n\r\n@mbrubeck suggested on #rust that I try using ld.gold. That helps, but the slowdown is still there:\r\n```console\r\n$ cargo clean\r\n$ env RUSTFLAGS=\"-C link-args=-fuse-ld=gold\" cargo t --no-run\r\n   ...\r\n   Compiling distributary v0.1.0 (file:///home/jon/dev/projects/distributary)\r\n    Finished dev [unoptimized + debuginfo] target(s) in 276.47 secs\r\n$ touch src/lib.rs\r\n$ env RUSTFLAGS=\"-C link-args=-fuse-ld=gold\" cargo t --no-run\r\n   Compiling distributary v0.1.0 (file:///home/jon/dev/projects/distributary)\r\n    Finished dev [unoptimized + debuginfo] target(s) in 45.35 secs\r\n```\r\n\r\nPart of the issue seems to be that `rust` re-links the library and all binaries (of which this particular project has 11) each time, and double-compiles a bunch of things (due to `#[cfg(test)]`). Compiling just the unit tests in `lib` is better, though still somewhat slow, and obviously insufficient (I also want to compile and run `tests/`):\r\n```console\r\n$ touch src/lib.rs\r\n$ env RUSTFLAGS=\"-C link-args=-fuse-ld=gold\" cargo rustc --lib -- --test\r\n   Compiling distributary v0.1.0 (file:///home/jon/dev/projects/distributary)\r\n    Finished dev [unoptimized + debuginfo] target(s) in 11.72 secs\r\n```\r\n\r\n@aidanhs observed that it is actually *necessary* to compile the binaries, because the tests script may depend on them (e.g., [cargo](https://github.com/rust-lang/cargo/blob/master/src/cargo/ops/cargo_rustc/context.rs#L646)), or because the binaries themselves contain tests. I tried using `cargo test --test lib --lib` to avoid compiling binaries, but that doesn't seem to work correctly.\r\n\r\nI'm not entirely sure exactly what the right fix for this is, but 12s compilation time for the core library (and 80s for the full crate) when nothing has changed seems excessive, and just what incremental compile was intended to help with. Having a way to say \"don't compile binaries when testing\" would also be nice, and save a decent chunk of compilation time for projects with many binaries, but I'm guessing that's more of an orthogonal cargo issue?"},"comment":{"url":"https://api.github.com/repos/rust-lang/rust/issues/comments/289789933","html_url":"https://github.com/rust-lang/rust/issues/40858#issuecomment-289789933","issue_url":"https://api.github.com/repos/rust-lang/rust/issues/40858","id":289789933,"user":{"login":"jonhoo","id":176295,"avatar_url":"https://avatars1.githubusercontent.com/u/176295?v=3","gravatar_id":"","url":"https://api.github.com/users/jonhoo","html_url":"https://github.com/jonhoo","followers_url":"https://api.github.com/users/jonhoo/followers","following_url":"https://api.github.com/users/jonhoo/following{/other_user}","gists_url":"https://api.github.com/users/jonhoo/gists{/gist_id}","starred_url":"https://api.github.com/users/jonhoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonhoo/subscriptions","organizations_url":"https://api.github.com/users/jonhoo/orgs","repos_url":"https://api.github.com/users/jonhoo/repos","events_url":"https://api.github.com/users/jonhoo/events{/privacy}","received_events_url":"https://api.github.com/users/jonhoo/received_events","type":"User","site_admin":false},"created_at":"2017-03-28T14:34:25Z","updated_at":"2017-03-28T14:34:25Z","body":"On `HEAD`, where all binaries are hidden behind a feature flag (and thus aren't compiled):\r\n```console\r\n$ env RUSTFLAGS=-Ztime-passes cargo t --no-run > /dev/null\r\n$ touch src/lib.rs\r\n$ env RUSTFLAGS=-Ztime-passes cargo t --no-run\r\n   Compiling distributary v0.1.0 (file:///home/jon/dev/projects/distributary)\r\ntime: 0.053; rss: 63MB\tparsing\r\ntime: 0.000; rss: 63MB\trecursion limit\r\ntime: 0.000; rss: 63MB\tcrate injection\r\ntime: 0.052; rss: 63MB\tparsing\r\ntime: 0.000; rss: 63MB\trecursion limit\r\ntime: 0.000; rss: 63MB\tcrate injection\r\ntime: 0.001; rss: 66MB\tplugin loading\r\ntime: 0.000; rss: 66MB\tplugin registration\r\ntime: 0.001; rss: 66MB\tplugin loading\r\ntime: 0.000; rss: 66MB\tplugin registration\r\ntime: 0.245; rss: 214MB\texpansion\r\ntime: 0.000; rss: 214MB\tmaybe building test harness\r\ntime: 0.003; rss: 214MB\tmaybe creating a macro crate\r\ntime: 0.000; rss: 214MB\tchecking for inline asm in case the target doesn't support it\r\ntime: 0.008; rss: 214MB\tearly lint checks\r\ntime: 0.003; rss: 214MB\tAST validation\r\ntime: 0.039; rss: 222MB\tname resolution\r\ntime: 0.008; rss: 222MB\tcomplete gated feature checking\r\ntime: 0.025; rss: 235MB\tlowering ast -> hir\r\ntime: 0.008; rss: 237MB\tindexing hir\r\ntime: 0.003; rss: 237MB\tattribute checking\r\ntime: 0.001; rss: 232MB\tlanguage item collection\r\ntime: 0.006; rss: 232MB\tlifetime resolution\r\ntime: 0.000; rss: 232MB\tlooking for entry point\r\ntime: 0.000; rss: 232MB\tlooking for plugin registrar\r\ntime: 0.015; rss: 242MB\tregion resolution\r\ntime: 0.003; rss: 242MB\tloop checking\r\ntime: 0.001; rss: 242MB\tstatic item recursion checking\r\ntime: 0.389; rss: 231MB\texpansion\r\ntime: 0.017; rss: 232MB\tmaybe building test harness\r\ntime: 0.003; rss: 232MB\tmaybe creating a macro crate\r\ntime: 0.000; rss: 232MB\tchecking for inline asm in case the target doesn't support it\r\ntime: 0.013; rss: 232MB\tearly lint checks\r\ntime: 0.003; rss: 232MB\tAST validation\r\ntime: 0.089; rss: 243MB\tcompute_incremental_hashes_map\r\ntime: 0.069; rss: 241MB\tname resolution\r\ntime: 0.010; rss: 241MB\tcomplete gated feature checking\r\ntime: 0.043; rss: 256MB\tlowering ast -> hir\r\ntime: 0.020; rss: 257MB\tindexing hir\r\ntime: 0.005; rss: 257MB\tattribute checking\r\ntime: 0.003; rss: 246MB\tlanguage item collection\r\ntime: 0.009; rss: 246MB\tlifetime resolution\r\ntime: 0.000; rss: 246MB\tlooking for entry point\r\ntime: 0.000; rss: 246MB\tlooking for plugin registrar\r\ntime: 0.023; rss: 254MB\tregion resolution\r\ntime: 0.007; rss: 254MB\tloop checking\r\ntime: 0.001; rss: 254MB\tstatic item recursion checking\r\ntime: 0.262; rss: 258MB\tload_dep_graph\r\ntime: 0.005; rss: 249MB\tstability index\r\ntime: 0.012; rss: 250MB\tstability checking\r\ntime: 0.034; rss: 262MB\ttype collecting\r\ntime: 0.003; rss: 262MB\tvariance inference\r\ntime: 0.004; rss: 262MB\timpl wf inference\r\ntime: 0.151; rss: 255MB\tcompute_incremental_hashes_map\r\ntime: 0.068; rss: 281MB\tcoherence checking\r\ntime: 0.071; rss: 283MB\twf checking\r\ntime: 0.056; rss: 284MB\titem-types checking\r\ntime: 0.302; rss: 262MB\tload_dep_graph\r\ntime: 0.006; rss: 262MB\tstability index\r\ntime: 0.016; rss: 264MB\tstability checking\r\ntime: 0.048; rss: 276MB\ttype collecting\r\ntime: 0.007; rss: 276MB\tvariance inference\r\ntime: 0.007; rss: 278MB\timpl wf inference\r\ntime: 0.072; rss: 297MB\tcoherence checking\r\ntime: 0.085; rss: 299MB\twf checking\r\ntime: 0.117; rss: 312MB\titem-types checking\r\ntime: 1.647; rss: 321MB\titem-bodies checking\r\ntime: 0.066; rss: 323MB\tconst checking\r\ntime: 0.027; rss: 323MB\tprivacy checking\r\ntime: 0.010; rss: 323MB\tintrinsic checking\r\ntime: 0.005; rss: 323MB\teffect checking\r\ntime: 0.018; rss: 323MB\tmatch checking\r\ntime: 0.013; rss: 325MB\tliveness checking\r\ntime: 0.049; rss: 325MB\trvalue checking\r\ntime: 0.128; rss: 367MB\tMIR dump\r\n  time: 0.019; rss: 367MB\tSimplifyCfg\r\n  time: 0.029; rss: 367MB\tQualifyAndPromoteConstants\r\n  time: 0.040; rss: 367MB\tTypeckMir\r\n  time: 0.002; rss: 367MB\tSimplifyBranches\r\n  time: 0.010; rss: 367MB\tSimplifyCfg\r\ntime: 0.100; rss: 367MB\tMIR cleanup and validation\r\ntime: 0.159; rss: 369MB\tborrow checking\r\ntime: 0.003; rss: 369MB\treachability checking\r\ntime: 0.014; rss: 369MB\tdeath checking\r\ntime: 0.000; rss: 369MB\tunused lib feature checking\r\ntime: 0.096; rss: 369MB\tlint checking\r\ntime: 0.000; rss: 369MB\tresolving dependency formats\r\n  time: 0.003; rss: 369MB\tNoLandingPads\r\n  time: 0.008; rss: 369MB\tSimplifyCfg\r\n  time: 0.022; rss: 369MB\tEraseRegions\r\n  time: 0.004; rss: 369MB\tAddCallGuards\r\n  time: 0.083; rss: 375MB\tElaborateDrops\r\n  time: 0.002; rss: 375MB\tNoLandingPads\r\n  time: 0.018; rss: 375MB\tSimplifyCfg\r\n  time: 0.000; rss: 375MB\tInline\r\n  time: 0.012; rss: 375MB\tInstCombine\r\n  time: 0.005; rss: 375MB\tDeaggregator\r\n  time: 0.002; rss: 375MB\tCopyPropagation\r\n  time: 0.013; rss: 375MB\tSimplifyLocals\r\n  time: 0.003; rss: 375MB\tAddCallGuards\r\n  time: 0.001; rss: 375MB\tPreTrans\r\ntime: 0.178; rss: 375MB\tMIR optimisations\r\n  time: 0.025; rss: 380MB\twrite metadata\r\ntime: 2.632; rss: 362MB\titem-bodies checking\r\ntime: 0.105; rss: 368MB\tconst checking\r\ntime: 0.052; rss: 368MB\tprivacy checking\r\ntime: 0.016; rss: 369MB\tintrinsic checking\r\ntime: 0.008; rss: 369MB\teffect checking\r\ntime: 0.026; rss: 369MB\tmatch checking\r\ntime: 0.020; rss: 371MB\tliveness checking\r\ntime: 0.068; rss: 371MB\trvalue checking\r\ntime: 0.196; rss: 443MB\tMIR dump\r\n  time: 0.026; rss: 443MB\tSimplifyCfg\r\n  time: 0.042; rss: 443MB\tQualifyAndPromoteConstants\r\n  time: 0.067; rss: 443MB\tTypeckMir\r\n  time: 0.004; rss: 443MB\tSimplifyBranches\r\n  time: 0.015; rss: 443MB\tSimplifyCfg\r\ntime: 0.154; rss: 443MB\tMIR cleanup and validation\r\ntime: 0.266; rss: 443MB\tborrow checking\r\ntime: 0.002; rss: 443MB\treachability checking\r\ntime: 0.015; rss: 443MB\tdeath checking\r\ntime: 0.000; rss: 443MB\tunused lib feature checking\r\ntime: 0.139; rss: 443MB\tlint checking\r\ntime: 0.013; rss: 443MB\tresolving dependency formats\r\n  time: 0.003; rss: 443MB\tNoLandingPads\r\n  time: 0.015; rss: 443MB\tSimplifyCfg\r\n  time: 0.036; rss: 447MB\tEraseRegions\r\n  time: 0.006; rss: 447MB\tAddCallGuards\r\n  time: 0.123; rss: 455MB\tElaborateDrops\r\n  time: 0.003; rss: 455MB\tNoLandingPads\r\n  time: 0.024; rss: 455MB\tSimplifyCfg\r\n  time: 0.000; rss: 455MB\tInline\r\n  time: 0.016; rss: 455MB\tInstCombine\r\n  time: 0.007; rss: 455MB\tDeaggregator\r\n  time: 0.004; rss: 455MB\tCopyPropagation\r\n  time: 0.018; rss: 455MB\tSimplifyLocals\r\n  time: 0.005; rss: 455MB\tAddCallGuards\r\n  time: 0.002; rss: 455MB\tPreTrans\r\ntime: 0.261; rss: 455MB\tMIR optimisations\r\n  time: 0.000; rss: 455MB\twrite metadata\r\n  time: 2.681; rss: 438MB\ttranslation item collection\r\n  time: 0.637; rss: 480MB\tcodegen unit partitioning\r\n  time: 2.946; rss: 529MB\ttranslation item collection\r\n  time: 0.286; rss: 499MB\tinternalize symbols\r\ntime: 5.346; rss: 499MB\ttranslation\r\ntime: 0.000; rss: 499MB\tassert dep graph\r\ntime: 0.477; rss: 489MB\tserialize dep graph\r\n  time: 0.002; rss: 395MB\tcodegen passes [3]\r\ntime: 0.083; rss: 395MB\tLLVM passes\r\ntime: 0.000; rss: 395MB\tserialize work products\r\n  time: 0.685; rss: 554MB\tcodegen unit partitioning\r\ntime: 1.331; rss: 399MB\tlinking\r\ntime: 0.007; rss: 55MB\tparsing\r\ntime: 0.000; rss: 55MB\trecursion limit\r\ntime: 0.000; rss: 55MB\tcrate injection\r\ntime: 0.000; rss: 55MB\tplugin loading\r\ntime: 0.000; rss: 55MB\tplugin registration\r\ntime: 0.151; rss: 196MB\texpansion\r\ntime: 0.004; rss: 198MB\tmaybe building test harness\r\ntime: 0.000; rss: 198MB\tmaybe creating a macro crate\r\ntime: 0.000; rss: 198MB\tchecking for inline asm in case the target doesn't support it\r\ntime: 0.003; rss: 198MB\tearly lint checks\r\ntime: 0.001; rss: 198MB\tAST validation\r\ntime: 0.034; rss: 204MB\tname resolution\r\ntime: 0.001; rss: 204MB\tcomplete gated feature checking\r\ntime: 0.006; rss: 204MB\tlowering ast -> hir\r\ntime: 0.002; rss: 206MB\tindexing hir\r\ntime: 0.000; rss: 206MB\tattribute checking\r\ntime: 0.000; rss: 210MB\tlanguage item collection\r\ntime: 0.001; rss: 210MB\tlifetime resolution\r\ntime: 0.000; rss: 210MB\tlooking for entry point\r\ntime: 0.000; rss: 210MB\tlooking for plugin registrar\r\ntime: 0.002; rss: 210MB\tregion resolution\r\ntime: 0.000; rss: 210MB\tloop checking\r\ntime: 0.000; rss: 210MB\tstatic item recursion checking\r\ntime: 0.015; rss: 210MB\tcompute_incremental_hashes_map\r\ntime: 0.054; rss: 218MB\tload_dep_graph\r\ntime: 0.000; rss: 218MB\tstability index\r\ntime: 0.003; rss: 218MB\tstability checking\r\ntime: 0.003; rss: 218MB\ttype collecting\r\ntime: 0.000; rss: 218MB\tvariance inference\r\ntime: 0.000; rss: 218MB\timpl wf inference\r\ntime: 0.000; rss: 218MB\tcoherence checking\r\ntime: 0.002; rss: 218MB\twf checking\r\ntime: 0.013; rss: 220MB\titem-types checking\r\n  time: 0.337; rss: 557MB\tinternalize symbols\r\ntime: 5.774; rss: 557MB\ttranslation\r\ntime: 0.000; rss: 557MB\tassert dep graph\r\ntime: 0.414; rss: 268MB\titem-bodies checking\r\ntime: 0.011; rss: 268MB\tconst checking\r\ntime: 0.003; rss: 268MB\tprivacy checking\r\ntime: 0.002; rss: 268MB\tintrinsic checking\r\ntime: 0.001; rss: 268MB\teffect checking\r\ntime: 0.003; rss: 268MB\tmatch checking\r\ntime: 0.002; rss: 268MB\tliveness checking\r\ntime: 0.006; rss: 268MB\trvalue checking\r\ntime: 0.028; rss: 276MB\tMIR dump\r\n  time: 0.003; rss: 276MB\tSimplifyCfg\r\n  time: 0.006; rss: 276MB\tQualifyAndPromoteConstants\r\n  time: 0.008; rss: 276MB\tTypeckMir\r\n  time: 0.000; rss: 276MB\tSimplifyBranches\r\n  time: 0.002; rss: 276MB\tSimplifyCfg\r\ntime: 0.020; rss: 276MB\tMIR cleanup and validation\r\ntime: 0.032; rss: 276MB\tborrow checking\r\ntime: 0.000; rss: 276MB\treachability checking\r\ntime: 0.003; rss: 276MB\tdeath checking\r\ntime: 0.000; rss: 276MB\tunused lib feature checking\r\ntime: 0.022; rss: 276MB\tlint checking\r\ntime: 0.014; rss: 276MB\tresolving dependency formats\r\n  time: 0.000; rss: 276MB\tNoLandingPads\r\n  time: 0.002; rss: 276MB\tSimplifyCfg\r\n  time: 0.004; rss: 278MB\tEraseRegions\r\n  time: 0.001; rss: 278MB\tAddCallGuards\r\n  time: 0.020; rss: 280MB\tElaborateDrops\r\n  time: 0.000; rss: 280MB\tNoLandingPads\r\n  time: 0.004; rss: 280MB\tSimplifyCfg\r\n  time: 0.000; rss: 280MB\tInline\r\n  time: 0.003; rss: 280MB\tInstCombine\r\n  time: 0.000; rss: 280MB\tDeaggregator\r\n  time: 0.000; rss: 280MB\tCopyPropagation\r\n  time: 0.002; rss: 280MB\tSimplifyLocals\r\n  time: 0.001; rss: 280MB\tAddCallGuards\r\n  time: 0.000; rss: 280MB\tPreTrans\r\ntime: 0.038; rss: 280MB\tMIR optimisations\r\n  time: 0.000; rss: 280MB\twrite metadata\r\ntime: 0.668; rss: 542MB\tserialize dep graph\r\n  time: 0.003; rss: 406MB\tcodegen passes [9]\r\ntime: 0.081; rss: 401MB\tLLVM passes\r\ntime: 0.000; rss: 401MB\tserialize work products\r\n  time: 0.465; rss: 300MB\ttranslation item collection\r\n  time: 0.100; rss: 304MB\tcodegen unit partitioning\r\n  time: 0.046; rss: 320MB\tinternalize symbols\r\ntime: 1.056; rss: 320MB\ttranslation\r\ntime: 0.000; rss: 320MB\tassert dep graph\r\ntime: 0.110; rss: 324MB\tserialize dep graph\r\n  time: 0.002; rss: 307MB\tcodegen passes [1]\r\ntime: 0.021; rss: 308MB\tLLVM passes\r\ntime: 0.000; rss: 308MB\tserialize work products\r\n  time: 3.697; rss: 401MB\trunning linker\r\ntime: 3.711; rss: 401MB\tlinking\r\n  time: 3.605; rss: 308MB\trunning linker\r\ntime: 3.620; rss: 308MB\tlinking\r\n    Finished dev [unoptimized + debuginfo] target(s) in 16.75 secs\r\n```"}}
{"actor":{"display_login":"jonhoo"}}