{"action":"created","comment":{"url":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/comments/100526927","pull_request_review_id":21224870,"id":100526927,"diff_hunk":"@@ -27,115 +31,164 @@ repositories' pull requests by using the GitHub Pull Request Builder plugin. It\n will start a batch of tests whenever a new pull request is open in the \"builds\"\n and \"versions\" repositories by an authorized user, when those pull requests are\n updated with new commits or when an authorized user comments the phrase\n-\"start tests\". Builds and Pylint validation can be triggered independently with\n-the phrases \"start build\" and \"start pylint\", respectively.\n-\n-### Release process\n-\n-A Jenkins job will trigger periodically to update the packages' versions, create\n-release notes and execute a build with all packages available, by executing the\n-corresponding commands from the \"builds\" repository scripts. Commits will be\n-created for the \"versions\" and \"open-power-host-os.github.io\" repositories and\n-pushed to the designated GitHub user's repository. Ideally, this should be a\n-\"bot\" user. Pull requests for those commits can be created manually and merged\n-into the main organization/user repository, if desired.\n-\n-Note: The default is to run those periodic builds once a week (every Wednesday\n-at 11 AM).\n+\"start tests\". Currently, tests are: building of all packages, Pylint validation\n+of Python code, yamllint validation of all package metadata YAML files and rpmlint\n+validation of all RPM packages specification files. These tests can be triggered\n+independently with the phrases \"start build\", \"start pylint\", \"start yamllint\"\n+and \"start rpmlint\" respectively.\n+\n+### Periodic builds process\n+\n+A Jenkins job will trigger periodically to check the versions of the packages\n+available in OpenPOWER Host OS GitHub organization, update the corresponding\n+packages' versions in the \"versions\" git repository, create a build with all\n+packages available and create release notes, by executing the corresponding\n+commands from the build scripts in the \"builds\" git repository. Commits will\n+be created for the local \"versions\" and \"open-power-host-os.github.io\" git\n+repositories and pushed to the designated GitHub user's git repository. Ideally,\n+this should be a \"bot\" user. Pull requests for those commits can be created\n+manually and merged into the main organization/user git repository, if desired.\n+\n+Note: The default is to execute those periodic builds once a week (every Wednesday\n+at 11 AM UTC-03:00).\n \n ## Infrastructure setup\n \n If the whole infrastructure setup is desired, you should do the following steps:\n-- Prepare CentOS system\n-- Execute Ansible playbooks\n-- Manual Jenkins steps\n+- Prepare CentOS system(s)\n+- Setup SSH credentials used to access upload server and GitHub from Jenkins slaves\n+- Automatically setup Jenkins master and slave(s) using Ansible playbooks\n+- Manually setup Jenkins via web UI\n  - Create administrative jobs\n- - Create slaves credentials\n- - Create GitHub credentials\n- - Create slave nodes\n- - Create Host OS builds jobs\n+ - Setup SSH credentials used to access Jenkins slaves from Jenkins master\n+ - Create credentials used to access GitHub API from Jenkins\n+ - Create Jenkins slaves in Jenkins web UI\n+ - Create builds jobs\n+\n They are described in detail below.\n \n-### Prepare CentOS system\n+### Prepare CentOS system(s)\n+\n+First, you will need to prepare at least one CentOS system. If the Jenkins\n+instance will be used for testing, one system is enough; production instances\n+should use more systems.  Currently, OpenPOWER Host OS has 1 Jenkins slave for\n+validating the code from GitHub pull requests and 3 Jenkins slaves for builds.\n+\n+KVM virtual machines are recommended, but you could optionally use bare-metal or\n+containers instead. If KVM virtualization is used, for each required system,\n+you will need to create a KVM guest and install a CentOS ISO in it. It is\n+recommended to use virt-install to do it.\n+\n+If you do not have a direct network connection to the system(s), you may\n+setup a SSH proxy in ~/.ssh/config in your workstation to make the SSH\n+proxy transparent. Example:\n \n-First, you'll need to prepare at least one CentOS system. Virtual machines are\n-recommended, but they could also be bare-metal or containers.\n+Host ophostos-builds\n+    HostName <target_server>\n+    ProxyCommand ssh <myuser>@<proxy_server> nc <target_server> 22\n \n-Note: to be able to upload the results of builds to a server and push commits to\n-GitHub automatically, the slaves must have be able to identify the hosts. Add\n-them to the \"known_hosts\" file by executing a simple ssh command from the slaves\n-to the upload server and to github.com. Alternatively, you could disable\n-\"StrictHostKeyChecking\", but that may be a security hazard.\n+After the SSH proxy is configured, simply use the following to connect to the system:\n+`ssh <myuser>@ophost-builds`\n \n-### Execute Ansible playbooks\n+### Setup SSH credentials used to access upload server and GitHub from Jenkins slaves\n \n-There are Ansible playbooks that set up the Jenkins master node and two Jenkins\n-slave nodes, one for Host OS builds and one for validating the code from GitHub\n-pull requests. There could be multiple slave nodes set up for parallel builds.\n-The different roles (master, builds slave and validation slave) could even be\n-set up in the same system, as desired.\n+To automate SSH authentication between 2 hosts, it is necessary to:\n+ - create a SSH key pair if you still do not have one\n+ - add the target host to ~/.ssh/known_hosts in the source host. It is also\n+   possible to do it by executing a simple SSH command from the source host to\n+   the target host. Alternatively, you could disable \"StrictHostKeyChecking\"\n+   for all SSH connections in the source host, but that may be a security hazard.\n+ - add the SSH private key to ~/.ssh/ in the source host\n+ - add the SSH public key to ~/.ssh/authorized_keys file in the target host\n \n-Read the [Ansible instructions](ansible/README.md) for details on how to execute\n+To be able to upload the results of builds to a server and push commits to\n+GitHub automatically, you must automate SSH authentication from Jenkins slaves\n+to those two servers.\n+\n+### Automatically setup Jenkins master and slave(s) using Ansible playbooks\n+\n+There are two Ansible playbooks: one that sets up a Jenkins master node and\n+another one that sets up a Jenkins slave node. Read the\n+[Ansible instructions](ansible/README.md) for details on how to execute\n the playbooks.\n \n-Note: The \"master\" playbook may fail due to Jenkins temporary unresponsiveness.\n-If you see HTTP request errors, try running it again.\n+If you wish to have a single system hosting the entire Jenkins instance, both\n+playbooks can be executed in the same system. Execute first the Jenkins master\n+playbook, stop Jenkins service (systemctl stop jenkins) and then execute the\n+Jenkins slave playbook.\n+\n+Note: The Jenkins playbooks may fail due to network errors. If you see HTTP\n+request errors, try executing them again.\n+\n \n-### Manual Jenkins steps\n+### Manually setup Jenkins via web UI\n+\n+If you do not have a direct network connection to the system(s), you may\n+create a SSH tunnel in your workstation to be able to access the Jenkins\n+web UI, which is accessible via HTTPS. Example:\n+\n+`sudo ssh -o \"ProxyCommand ssh <user>@<proxy_server> -W %h:%p\" -NL 443:localhost:443 root@<jenkins_master>`","path":"README.md","position":181,"original_position":181,"commit_id":"4a808623d484e4a315410c7a616c688afd210173","original_commit_id":"4a808623d484e4a315410c7a616c688afd210173","user":{"login":"olavphibm","id":15910498,"avatar_url":"https://avatars.githubusercontent.com/u/15910498?v=3","gravatar_id":"","url":"https://api.github.com/users/olavphibm","html_url":"https://github.com/olavphibm","followers_url":"https://api.github.com/users/olavphibm/followers","following_url":"https://api.github.com/users/olavphibm/following{/other_user}","gists_url":"https://api.github.com/users/olavphibm/gists{/gist_id}","starred_url":"https://api.github.com/users/olavphibm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olavphibm/subscriptions","organizations_url":"https://api.github.com/users/olavphibm/orgs","repos_url":"https://api.github.com/users/olavphibm/repos","events_url":"https://api.github.com/users/olavphibm/events{/privacy}","received_events_url":"https://api.github.com/users/olavphibm/received_events","type":"User","site_admin":false},"body":"I use a different command and I am not sure what `<proxy_server>` should be filled with. Maybe use a different name or explain in the text.","created_at":"2017-02-10T12:40:22Z","updated_at":"2017-02-10T13:02:25Z","html_url":"https://github.com/open-power-host-os/infrastructure/pull/25#discussion_r100526927","pull_request_url":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25","_links":{"self":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/comments/100526927"},"html":{"href":"https://github.com/open-power-host-os/infrastructure/pull/25#discussion_r100526927"},"pull_request":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25"}}},"pull_request":{"url":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25","id":104870463,"html_url":"https://github.com/open-power-host-os/infrastructure/pull/25","diff_url":"https://github.com/open-power-host-os/infrastructure/pull/25.diff","patch_url":"https://github.com/open-power-host-os/infrastructure/pull/25.patch","issue_url":"https://api.github.com/repos/open-power-host-os/infrastructure/issues/25","number":25,"state":"open","locked":false,"title":"Update Jenkins setup doc","user":{"login":"alanoe","id":6615847,"avatar_url":"https://avatars.githubusercontent.com/u/6615847?v=3","gravatar_id":"","url":"https://api.github.com/users/alanoe","html_url":"https://github.com/alanoe","followers_url":"https://api.github.com/users/alanoe/followers","following_url":"https://api.github.com/users/alanoe/following{/other_user}","gists_url":"https://api.github.com/users/alanoe/gists{/gist_id}","starred_url":"https://api.github.com/users/alanoe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alanoe/subscriptions","organizations_url":"https://api.github.com/users/alanoe/orgs","repos_url":"https://api.github.com/users/alanoe/repos","events_url":"https://api.github.com/users/alanoe/events{/privacy}","received_events_url":"https://api.github.com/users/alanoe/received_events","type":"User","site_admin":false},"body":"","created_at":"2017-02-06T19:59:48Z","updated_at":"2017-02-10T13:02:25Z","closed_at":null,"merged_at":null,"merge_commit_sha":"df7cc38df7affd322586123bad226d99f4de5e5c","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25/commits","review_comments_url":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25/comments","review_comment_url":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/comments{/number}","comments_url":"https://api.github.com/repos/open-power-host-os/infrastructure/issues/25/comments","statuses_url":"https://api.github.com/repos/open-power-host-os/infrastructure/statuses/4a808623d484e4a315410c7a616c688afd210173","head":{"label":"alanoe:ae/update_jenkins_setup_doc","ref":"ae/update_jenkins_setup_doc","sha":"4a808623d484e4a315410c7a616c688afd210173","user":{"login":"alanoe","id":6615847,"avatar_url":"https://avatars.githubusercontent.com/u/6615847?v=3","gravatar_id":"","url":"https://api.github.com/users/alanoe","html_url":"https://github.com/alanoe","followers_url":"https://api.github.com/users/alanoe/followers","following_url":"https://api.github.com/users/alanoe/following{/other_user}","gists_url":"https://api.github.com/users/alanoe/gists{/gist_id}","starred_url":"https://api.github.com/users/alanoe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alanoe/subscriptions","organizations_url":"https://api.github.com/users/alanoe/orgs","repos_url":"https://api.github.com/users/alanoe/repos","events_url":"https://api.github.com/users/alanoe/events{/privacy}","received_events_url":"https://api.github.com/users/alanoe/received_events","type":"User","site_admin":false},"repo":{"id":80141226,"name":"infrastructure","full_name":"alanoe/infrastructure","owner":{"login":"alanoe","id":6615847,"avatar_url":"https://avatars.githubusercontent.com/u/6615847?v=3","gravatar_id":"","url":"https://api.github.com/users/alanoe","html_url":"https://github.com/alanoe","followers_url":"https://api.github.com/users/alanoe/followers","following_url":"https://api.github.com/users/alanoe/following{/other_user}","gists_url":"https://api.github.com/users/alanoe/gists{/gist_id}","starred_url":"https://api.github.com/users/alanoe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alanoe/subscriptions","organizations_url":"https://api.github.com/users/alanoe/orgs","repos_url":"https://api.github.com/users/alanoe/repos","events_url":"https://api.github.com/users/alanoe/events{/privacy}","received_events_url":"https://api.github.com/users/alanoe/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/alanoe/infrastructure","description":"Infrastructure as Code for OpenPOWER Host OS","fork":true,"url":"https://api.github.com/repos/alanoe/infrastructure","forks_url":"https://api.github.com/repos/alanoe/infrastructure/forks","keys_url":"https://api.github.com/repos/alanoe/infrastructure/keys{/key_id}","collaborators_url":"https://api.github.com/repos/alanoe/infrastructure/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/alanoe/infrastructure/teams","hooks_url":"https://api.github.com/repos/alanoe/infrastructure/hooks","issue_events_url":"https://api.github.com/repos/alanoe/infrastructure/issues/events{/number}","events_url":"https://api.github.com/repos/alanoe/infrastructure/events","assignees_url":"https://api.github.com/repos/alanoe/infrastructure/assignees{/user}","branches_url":"https://api.github.com/repos/alanoe/infrastructure/branches{/branch}","tags_url":"https://api.github.com/repos/alanoe/infrastructure/tags","blobs_url":"https://api.github.com/repos/alanoe/infrastructure/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/alanoe/infrastructure/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/alanoe/infrastructure/git/refs{/sha}","trees_url":"https://api.github.com/repos/alanoe/infrastructure/git/trees{/sha}","statuses_url":"https://api.github.com/repos/alanoe/infrastructure/statuses/{sha}","languages_url":"https://api.github.com/repos/alanoe/infrastructure/languages","stargazers_url":"https://api.github.com/repos/alanoe/infrastructure/stargazers","contributors_url":"https://api.github.com/repos/alanoe/infrastructure/contributors","subscribers_url":"https://api.github.com/repos/alanoe/infrastructure/subscribers","subscription_url":"https://api.github.com/repos/alanoe/infrastructure/subscription","commits_url":"https://api.github.com/repos/alanoe/infrastructure/commits{/sha}","git_commits_url":"https://api.github.com/repos/alanoe/infrastructure/git/commits{/sha}","comments_url":"https://api.github.com/repos/alanoe/infrastructure/comments{/number}","issue_comment_url":"https://api.github.com/repos/alanoe/infrastructure/issues/comments{/number}","contents_url":"https://api.github.com/repos/alanoe/infrastructure/contents/{+path}","compare_url":"https://api.github.com/repos/alanoe/infrastructure/compare/{base}...{head}","merges_url":"https://api.github.com/repos/alanoe/infrastructure/merges","archive_url":"https://api.github.com/repos/alanoe/infrastructure/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/alanoe/infrastructure/downloads","issues_url":"https://api.github.com/repos/alanoe/infrastructure/issues{/number}","pulls_url":"https://api.github.com/repos/alanoe/infrastructure/pulls{/number}","milestones_url":"https://api.github.com/repos/alanoe/infrastructure/milestones{/number}","notifications_url":"https://api.github.com/repos/alanoe/infrastructure/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/alanoe/infrastructure/labels{/name}","releases_url":"https://api.github.com/repos/alanoe/infrastructure/releases{/id}","deployments_url":"https://api.github.com/repos/alanoe/infrastructure/deployments","created_at":"2017-01-26T18:15:08Z","updated_at":"2017-01-26T18:15:10Z","pushed_at":"2017-02-09T18:56:20Z","git_url":"git://github.com/alanoe/infrastructure.git","ssh_url":"git@github.com:alanoe/infrastructure.git","clone_url":"https://github.com/alanoe/infrastructure.git","svn_url":"https://github.com/alanoe/infrastructure","homepage":"","size":69,"stargazers_count":0,"watchers_count":0,"language":"Groovy","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"open-power-host-os:master","ref":"master","sha":"f638109764edb915ac4a6b4a9a6985420cdbab68","user":{"login":"open-power-host-os","id":19573150,"avatar_url":"https://avatars.githubusercontent.com/u/19573150?v=3","gravatar_id":"","url":"https://api.github.com/users/open-power-host-os","html_url":"https://github.com/open-power-host-os","followers_url":"https://api.github.com/users/open-power-host-os/followers","following_url":"https://api.github.com/users/open-power-host-os/following{/other_user}","gists_url":"https://api.github.com/users/open-power-host-os/gists{/gist_id}","starred_url":"https://api.github.com/users/open-power-host-os/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/open-power-host-os/subscriptions","organizations_url":"https://api.github.com/users/open-power-host-os/orgs","repos_url":"https://api.github.com/users/open-power-host-os/repos","events_url":"https://api.github.com/users/open-power-host-os/events{/privacy}","received_events_url":"https://api.github.com/users/open-power-host-os/received_events","type":"Organization","site_admin":false},"repo":{"id":75936942,"name":"infrastructure","full_name":"open-power-host-os/infrastructure","owner":{"login":"open-power-host-os","id":19573150,"avatar_url":"https://avatars.githubusercontent.com/u/19573150?v=3","gravatar_id":"","url":"https://api.github.com/users/open-power-host-os","html_url":"https://github.com/open-power-host-os","followers_url":"https://api.github.com/users/open-power-host-os/followers","following_url":"https://api.github.com/users/open-power-host-os/following{/other_user}","gists_url":"https://api.github.com/users/open-power-host-os/gists{/gist_id}","starred_url":"https://api.github.com/users/open-power-host-os/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/open-power-host-os/subscriptions","organizations_url":"https://api.github.com/users/open-power-host-os/orgs","repos_url":"https://api.github.com/users/open-power-host-os/repos","events_url":"https://api.github.com/users/open-power-host-os/events{/privacy}","received_events_url":"https://api.github.com/users/open-power-host-os/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/open-power-host-os/infrastructure","description":"Infrastructure as Code for OpenPOWER Host OS","fork":false,"url":"https://api.github.com/repos/open-power-host-os/infrastructure","forks_url":"https://api.github.com/repos/open-power-host-os/infrastructure/forks","keys_url":"https://api.github.com/repos/open-power-host-os/infrastructure/keys{/key_id}","collaborators_url":"https://api.github.com/repos/open-power-host-os/infrastructure/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/open-power-host-os/infrastructure/teams","hooks_url":"https://api.github.com/repos/open-power-host-os/infrastructure/hooks","issue_events_url":"https://api.github.com/repos/open-power-host-os/infrastructure/issues/events{/number}","events_url":"https://api.github.com/repos/open-power-host-os/infrastructure/events","assignees_url":"https://api.github.com/repos/open-power-host-os/infrastructure/assignees{/user}","branches_url":"https://api.github.com/repos/open-power-host-os/infrastructure/branches{/branch}","tags_url":"https://api.github.com/repos/open-power-host-os/infrastructure/tags","blobs_url":"https://api.github.com/repos/open-power-host-os/infrastructure/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/open-power-host-os/infrastructure/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/open-power-host-os/infrastructure/git/refs{/sha}","trees_url":"https://api.github.com/repos/open-power-host-os/infrastructure/git/trees{/sha}","statuses_url":"https://api.github.com/repos/open-power-host-os/infrastructure/statuses/{sha}","languages_url":"https://api.github.com/repos/open-power-host-os/infrastructure/languages","stargazers_url":"https://api.github.com/repos/open-power-host-os/infrastructure/stargazers","contributors_url":"https://api.github.com/repos/open-power-host-os/infrastructure/contributors","subscribers_url":"https://api.github.com/repos/open-power-host-os/infrastructure/subscribers","subscription_url":"https://api.github.com/repos/open-power-host-os/infrastructure/subscription","commits_url":"https://api.github.com/repos/open-power-host-os/infrastructure/commits{/sha}","git_commits_url":"https://api.github.com/repos/open-power-host-os/infrastructure/git/commits{/sha}","comments_url":"https://api.github.com/repos/open-power-host-os/infrastructure/comments{/number}","issue_comment_url":"https://api.github.com/repos/open-power-host-os/infrastructure/issues/comments{/number}","contents_url":"https://api.github.com/repos/open-power-host-os/infrastructure/contents/{+path}","compare_url":"https://api.github.com/repos/open-power-host-os/infrastructure/compare/{base}...{head}","merges_url":"https://api.github.com/repos/open-power-host-os/infrastructure/merges","archive_url":"https://api.github.com/repos/open-power-host-os/infrastructure/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/open-power-host-os/infrastructure/downloads","issues_url":"https://api.github.com/repos/open-power-host-os/infrastructure/issues{/number}","pulls_url":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls{/number}","milestones_url":"https://api.github.com/repos/open-power-host-os/infrastructure/milestones{/number}","notifications_url":"https://api.github.com/repos/open-power-host-os/infrastructure/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/open-power-host-os/infrastructure/labels{/name}","releases_url":"https://api.github.com/repos/open-power-host-os/infrastructure/releases{/id}","deployments_url":"https://api.github.com/repos/open-power-host-os/infrastructure/deployments","created_at":"2016-12-08T12:46:10Z","updated_at":"2017-01-04T12:11:10Z","pushed_at":"2017-02-09T19:15:41Z","git_url":"git://github.com/open-power-host-os/infrastructure.git","ssh_url":"git@github.com:open-power-host-os/infrastructure.git","clone_url":"https://github.com/open-power-host-os/infrastructure.git","svn_url":"https://github.com/open-power-host-os/infrastructure","homepage":"","size":76,"stargazers_count":0,"watchers_count":0,"language":"Groovy","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":4,"forks":6,"open_issues":4,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25"},"html":{"href":"https://github.com/open-power-host-os/infrastructure/pull/25"},"issue":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/issues/25"},"comments":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/issues/25/comments"},"review_comments":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25/comments"},"review_comment":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/pulls/25/commits"},"statuses":{"href":"https://api.github.com/repos/open-power-host-os/infrastructure/statuses/4a808623d484e4a315410c7a616c688afd210173"}}}}
{"actor":{"display_login":"olavphibm"}}