{"action":"created","issue":{"url":"https://api.github.com/repos/Origen-SDK/origen/issues/105","repository_url":"https://api.github.com/repos/Origen-SDK/origen","labels_url":"https://api.github.com/repos/Origen-SDK/origen/issues/105/labels{/name}","comments_url":"https://api.github.com/repos/Origen-SDK/origen/issues/105/comments","events_url":"https://api.github.com/repos/Origen-SDK/origen/issues/105/events","html_url":"https://github.com/Origen-SDK/origen/pull/105","id":244046759,"number":105,"title":"Global commands","user":{"login":"coreyeng","id":13453967,"avatar_url":"https://avatars2.githubusercontent.com/u/13453967?v=4","gravatar_id":"","url":"https://api.github.com/users/coreyeng","html_url":"https://github.com/coreyeng","followers_url":"https://api.github.com/users/coreyeng/followers","following_url":"https://api.github.com/users/coreyeng/following{/other_user}","gists_url":"https://api.github.com/users/coreyeng/gists{/gist_id}","starred_url":"https://api.github.com/users/coreyeng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/coreyeng/subscriptions","organizations_url":"https://api.github.com/users/coreyeng/orgs","repos_url":"https://api.github.com/users/coreyeng/repos","events_url":"https://api.github.com/users/coreyeng/events{/privacy}","received_events_url":"https://api.github.com/users/coreyeng/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2017-07-19T13:47:27Z","updated_at":"2017-07-20T12:34:07Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Origen-SDK/origen/pulls/105","html_url":"https://github.com/Origen-SDK/origen/pull/105","diff_url":"https://github.com/Origen-SDK/origen/pull/105.diff","patch_url":"https://github.com/Origen-SDK/origen/pull/105.patch"},"body":"We've been using Origen here to generate patterns with great success. So much success in fact, that we started building other, non-pattern-generation related utilities using Origen. These take advantage of several items that Origen gives us for free, such as; revision control, web documentation, email notification, and LDAP authentication services. This had inevitably led to us developing plugins in Origen for more generalized aspects (basically, just scripting) but that don't require any of the 'application' stuff Origen has. That is, these plugins aren't tied to any application and could care less if that application even exists. These plugins have shared commands that could be run with or without an application.\r\n\r\nCurrently, the only way to do this is to add a custom site config (possibly several, depending on how your environment is structured) that disables Bundler and allows one to manager their installation individually. However, we lose a huge asset (Bundler) when we do this. Additionally, this requires more user invention than should be necessary (making custom site configs, then debugging those configs, etc.). Alternatively, you could just create a dummy application and load your Gemfile with the tools required, but then you are required to invoke Origen from that application in order to do anything. This is just an inconvenience, but an inconvenience that I think could be fixed.\r\n\r\nThis pull request's purpose is allow user customization of their global installation AFTER first getting the general configuration from the site config. This allows them to customize their global Origen version and dependencies, including those that contain globally available commands.\r\n\r\nThe new flow in bin/origen is to first look if Origen is invoked in an application and run lbin/origen if so. If not, Origen will look at ~/.origen and see if a lbin/origen exists there. If so, that will be run, essentially allowing Origen to start up as if it were within an application, only at ~/.origen instead. If this doesn't exists, Origen will start up from the site config. However, the first startup of the site config with be akin to running Origen for the first time in an application, that is, the ~/.origen will be created, bundler will populate all the gems and the lbin, etc. That is, my home Origen directory now looks like this:\r\n\r\n![image](https://user-images.githubusercontent.com/13453967/28369927-976a67ec-6c5e-11e7-80e3-6d5e85fc65d4.png)\r\n\r\nPlugins can then add a 'global_commands' to the shared launcher in its config/application.rb file. This behaves exactly the same way as the shared_commands.rb. This only difference being global_commands will be available outside an application and shared_commands will still require an application. When run inside an application, global will be available as well though.\r\n\r\nIn pictures, this what I'm talking about:\r\n\r\n(Running within an application, with TE_Utils installed)\r\n![image](https://user-images.githubusercontent.com/13453967/28369463-4958b44c-6c5d-11e7-9cfd-3a97d95f6af0.png)\r\n\r\n![image](https://user-images.githubusercontent.com/13453967/28369445-3b397162-6c5d-11e7-905b-302ac47c8c1e.png)\r\n\r\n(Running externally to an application, with TE_Utils still installed)\r\n![image](https://user-images.githubusercontent.com/13453967/28369391-09bfde00-6c5d-11e7-829c-289e7ddc6c95.png)\r\n\r\nThis feature is optional though. Enabling it requires setting a site config variable to allow this. By default, Origen will behave as it currently does. So, users that are not familiar with this functionality do not need to use it.\r\n\r\nI did have some success with combining Gemfiles and programmatically adding dependencies. I opted to not do this because it becomes a distribution nightmare as an Origen application is now not dependent only on the Gemfile that's included. Additionally, we can't use Bundler for this, so we need to further build in utilities to handle getting a gem, checking its dependencies, etc., basically going back to the .imports that we abandoned once already in favor of something tried-and-tested. Otherwise, we would need to preprocess all Gemfiles, check the dependencies, etc. So, this note is really just saying: yes, we could combine multiple dependency sources, but I elected not to because it’s a lot of work to just end up violating our desire that a single application names all its dependencies.\r\n\r\nThis pull request is just to show my view of how Origen should start up to allow global dependencies and allow Origen to be used as a tool-dispersal platform in addition to its regular purpose. If this is accepted, I plan to take on adding the following as well:\r\n\r\n\t1) Document this new process on the Origen docs and include instructions for creating and editing a user site config, and the changes to LSF (see further down).\r\n\t2) Add a feature for origen to create a user site config upon start up (and have it be empty, or have commonly-used features commented out or set to false).\r\n\t3) Tie into either Origen 'new' or the app generator itself to allow for custom app generators.\r\n\t4) Add a way to add 'default' gems to the Gemfile upon start up. This would essentially allow a company's IT (or other toolset/origen expert) to setup a default site config and a default Gemfile, allowing for less experienced users to bypass this feature's setup altogether but still get the benefits. I plan this to be something along the lines of a site config path to a default Gemfile that will be compared at start up and copied and rerun if needed.\r\n\r\nI say this because I didn't want to add all of these but then have this feature declined. Not sure how the team feels about Origen being multi-purposed into a tool distribution platform as well, or if this should be the way to go about it. This was the most natural way I could think of to take advantage of Bundler and Ruby's built-in dependency features while maintaining an 'Origen' feel and ultimately give full authority to if and how it's used to the individual users.\r\n\r\nIn order to do this I had to slightly alter how Origen.root behaves. Some plugins (like testers) uses the plugin manager in application.rb, which could now be loaded even when not run in an app. Using the plugin manager uses Origen.root, which would fail with a lookup error when run outside of an application. In this PR, if Origen.root would normally fail, Origen.top will be returned instead. This has the bonus of making the origen_updater plugin happy, which was having some issues as well when being run globally. Origen.root! Stayed the same for now as I think that one should always work, since an application can't call it (there's no application), a plugin would have it point to itself, and Origen core doesn't use it from what I can see.\r\n\r\nAdded method running_globally? (aliased as invoked_globally?) that is just the opposite of in_app_workspace?\r\n\r\nAlso added a global LSF and LSF manager. This is the same as the application's LSF/LSF manager, but available when running globally. The log directory for this will be Origen.root, which is now ~/.origen, so ~/.origen/.logs. Since Origen.lsf method was already taken though, and changing that would break the API and a lot of plugins, I added Origen.lsf! to get the global @lsf object. Origen.lsf will now pick between the application's lsf_manager or the global lsf_manager, depending on where Origen is invoked.\r\n\r\nThe TL;DR of this pull request is I want to use Origen also as a means of general tool (i.e. scripts) distribution, but in order to really be able to do anything you must be running from an Origen application, or else have your site manager configure Origen for you and cease using Bundler. This pull request aims to allow users to customize their own global installs and allows for plugins to add global commands that can be run outside of an application."},"comment":{"url":"https://api.github.com/repos/Origen-SDK/origen/issues/comments/316689044","html_url":"https://github.com/Origen-SDK/origen/pull/105#issuecomment-316689044","issue_url":"https://api.github.com/repos/Origen-SDK/origen/issues/105","id":316689044,"user":{"login":"coreyeng","id":13453967,"avatar_url":"https://avatars2.githubusercontent.com/u/13453967?v=4","gravatar_id":"","url":"https://api.github.com/users/coreyeng","html_url":"https://github.com/coreyeng","followers_url":"https://api.github.com/users/coreyeng/followers","following_url":"https://api.github.com/users/coreyeng/following{/other_user}","gists_url":"https://api.github.com/users/coreyeng/gists{/gist_id}","starred_url":"https://api.github.com/users/coreyeng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/coreyeng/subscriptions","organizations_url":"https://api.github.com/users/coreyeng/orgs","repos_url":"https://api.github.com/users/coreyeng/repos","events_url":"https://api.github.com/users/coreyeng/events{/privacy}","received_events_url":"https://api.github.com/users/coreyeng/received_events","type":"User","site_admin":false},"created_at":"2017-07-20T12:34:07Z","updated_at":"2017-07-20T12:34:07Z","body":"Sure. When Origen is first sourced, bin/origen can now check if an lbin directory exists in the origen_home_dir location (~/.origen) by default. If it does it will use that. That is, ~/.origen/lbin/origen executable. I use Bundler to do that, so I also initialize a Gemfile. My home (or global) Origen install now somewhat resembles that of an application. That is, I have a lbin, .bundle, Gemfile, Gemfile.lock, and the new .session directories, in addition to the gems directory.\r\n\r\nThis Gemfile starts with just including Origen, nothing more. But, I can add additional gems to it, or point to a specific version of Origen or to a local path. For example, my current Gemfile (I was testing the startup of origen_testers as well) looks like this:\r\n\r\n![image](https://user-images.githubusercontent.com/13453967/28416881-6b430616-6d1b-11e7-91c4-718f9732b69b.png)\r\n\r\nThis file is in ~/.origen/Gemfile. Bundler will now use this. If I go to the ~/.origen/ and run Bundle I can see the new gems have been added.\r\n\r\n![image](https://user-images.githubusercontent.com/13453967/28417031-21ea75b6-6d1c-11e7-93fe-ad610bbf526b.png)\r\n\r\nIn this way I can add whatever gems I want to my global install. Or point to specific Origen version or a local copy. The bin/origen file still is in command though, it is still the starting point. The proper site config variables need to set and a Gemfile present then the /bin/origen will switch over to running that Gemfile for non-application Origen invocations.\r\n\r\nThe global_commands.rb file then acts in much the same way as the commands.rb file does with a plugin's shared_commands.rb. My config/application.rb file in TE_Utils now has this:\r\n\r\n![image](https://user-images.githubusercontent.com/13453967/28417156-9714d5ca-6d1c-11e7-8a24-1663675eb9e7.png)\r\n\r\nAnd I added a config/global_commands.rb file that looks very similar to the config/shared_commands.rb file:\r\n\r\n![image](https://user-images.githubusercontent.com/13453967/28417202-c28dba6e-6d1c-11e7-8e90-da87c10f32b1.png)\r\n\r\nI hope that gives a better explanation of what I did to add global gems. If not, I can try to provide more details, or do some kind of demo. I think the big thing is just that the ~/.origen before only had 'gems' now has its own Gemfile, .bundle, and lbin that can be used when no application is present."}}
{"actor":{"display_login":"coreyeng"}}