{"action":"created","issue":{"url":"https://api.github.com/repos/NixOS/nixpkgs/issues/21943","repository_url":"https://api.github.com/repos/NixOS/nixpkgs","labels_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/21943/labels{/name}","comments_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/21943/comments","events_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/21943/events","html_url":"https://github.com/NixOS/nixpkgs/pull/21943","id":201171482,"number":21943,"title":"[WIP, don't merge] make-disk-image: change to be less VM-centric","user":{"login":"copumpkin","id":2623,"avatar_url":"https://avatars.githubusercontent.com/u/2623?v=3","gravatar_id":"","url":"https://api.github.com/users/copumpkin","html_url":"https://github.com/copumpkin","followers_url":"https://api.github.com/users/copumpkin/followers","following_url":"https://api.github.com/users/copumpkin/following{/other_user}","gists_url":"https://api.github.com/users/copumpkin/gists{/gist_id}","starred_url":"https://api.github.com/users/copumpkin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/copumpkin/subscriptions","organizations_url":"https://api.github.com/users/copumpkin/orgs","repos_url":"https://api.github.com/users/copumpkin/repos","events_url":"https://api.github.com/users/copumpkin/events{/privacy}","received_events_url":"https://api.github.com/users/copumpkin/received_events","type":"User","site_admin":false},"labels":[{"id":60265212,"url":"https://api.github.com/repos/NixOS/nixpkgs/labels/6.topic:%20nixos","name":"6.topic: nixos","color":"fef2c0","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2017-01-17T04:04:48Z","updated_at":"2017-01-17T18:00:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/NixOS/nixpkgs/pulls/21943","html_url":"https://github.com/NixOS/nixpkgs/pull/21943","diff_url":"https://github.com/NixOS/nixpkgs/pull/21943.diff","patch_url":"https://github.com/NixOS/nixpkgs/pull/21943.patch"},"body":"This changes much of the make-disk-image.nix logic (and thus most NixOS image building) to use LKL to set up the target directory structure rather than a Linux VM. The only work we still do in a VM is less IO-heavy stuff that while still time-consuming, is less of the overall load. The goal is to kill more of that stuff, but that will require deeper changes to NixOS activation scripts and switch-to-configuration.pl, and I don't want to bite off too much at once.\r\n\r\nOn my test EC2 instance, the old image building code took about 25 minutes (see #20471), and this takes a little less than a minute. I've been testing it as follows:\r\n\r\n```shell\r\ntime nix-build nixos/ --no-out-link -A config.system.build.amazonImage --arg configuration \"{ imports = [ ./nixos/maintainers/scripts/ec2/amazon-image.nix ]; ec2.hvm = true; }\"\r\n```\r\n\r\nThings I'm unsure about and would appreciate comments on:\r\n\r\n1. Whether this actually produces an image that doesn't have subtle issues. I'm not so much concerned about the filesystem (since I'm using the Linux filesystem drivers) but I did hack up nixos-install.sh and might have missed something.\r\n2. Whether I can kill the last two uses of the VM\r\n3. Whether the `fakechroot` + `nix-env` is actually necessary. If I'm just doing `--set` with no previous generations, is `nix-env` doing anything beyond making two symlinks? Not doing it myself future-proofs this code a bit if the link scheme changes someday, but that also seems unlikely.\r\n\r\nThings I still need to do before I take the WIP marker off:\r\n\r\n1. More testing\r\n2. Add back support for `qcow2`, which I'd probably do as a post-VM conversion with `qemu-img` now that I touch image files directly and LKL doesn't understand anything but a flat image\r\n3. Possibly refactor the `nixos-install.sh` logic to share the core bits and pieces with this work, so we don't risk getting out of sync. I'd however rather do that as a follow-up PR to keep moving parts to a minimum."},"comment":{"url":"https://api.github.com/repos/NixOS/nixpkgs/issues/comments/273247757","html_url":"https://github.com/NixOS/nixpkgs/pull/21943#issuecomment-273247757","issue_url":"https://api.github.com/repos/NixOS/nixpkgs/issues/21943","id":273247757,"user":{"login":"copumpkin","id":2623,"avatar_url":"https://avatars.githubusercontent.com/u/2623?v=3","gravatar_id":"","url":"https://api.github.com/users/copumpkin","html_url":"https://github.com/copumpkin","followers_url":"https://api.github.com/users/copumpkin/followers","following_url":"https://api.github.com/users/copumpkin/following{/other_user}","gists_url":"https://api.github.com/users/copumpkin/gists{/gist_id}","starred_url":"https://api.github.com/users/copumpkin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/copumpkin/subscriptions","organizations_url":"https://api.github.com/users/copumpkin/orgs","repos_url":"https://api.github.com/users/copumpkin/repos","events_url":"https://api.github.com/users/copumpkin/events{/privacy}","received_events_url":"https://api.github.com/users/copumpkin/received_events","type":"User","site_admin":false},"created_at":"2017-01-17T18:00:45Z","updated_at":"2017-01-17T18:00:45Z","body":"@shlevy just suggested that I might be able to use user namespaces to do a proper native `chroot` inside the build. I'll give that a go later."}}
{"actor":{"display_login":"copumpkin"}}