{"action":"created","issue":{"url":"https://api.github.com/repos/syndesisio/syndesis-rest/issues/636","repository_url":"https://api.github.com/repos/syndesisio/syndesis-rest","labels_url":"https://api.github.com/repos/syndesisio/syndesis-rest/issues/636/labels{/name}","comments_url":"https://api.github.com/repos/syndesisio/syndesis-rest/issues/636/comments","events_url":"https://api.github.com/repos/syndesisio/syndesis-rest/issues/636/events","html_url":"https://github.com/syndesisio/syndesis-rest/pull/636","id":260035894,"number":636,"title":"feat(runtime): Prefetch connector catalog [WIP]","user":{"login":"rhuss","id":99080,"avatar_url":"https://avatars3.githubusercontent.com/u/99080?v=4","gravatar_id":"","url":"https://api.github.com/users/rhuss","html_url":"https://github.com/rhuss","followers_url":"https://api.github.com/users/rhuss/followers","following_url":"https://api.github.com/users/rhuss/following{/other_user}","gists_url":"https://api.github.com/users/rhuss/gists{/gist_id}","starred_url":"https://api.github.com/users/rhuss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rhuss/subscriptions","organizations_url":"https://api.github.com/users/rhuss/orgs","repos_url":"https://api.github.com/users/rhuss/repos","events_url":"https://api.github.com/users/rhuss/events{/privacy}","received_events_url":"https://api.github.com/users/rhuss/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"jimmidyson","id":464659,"avatar_url":"https://avatars1.githubusercontent.com/u/464659?v=4","gravatar_id":"","url":"https://api.github.com/users/jimmidyson","html_url":"https://github.com/jimmidyson","followers_url":"https://api.github.com/users/jimmidyson/followers","following_url":"https://api.github.com/users/jimmidyson/following{/other_user}","gists_url":"https://api.github.com/users/jimmidyson/gists{/gist_id}","starred_url":"https://api.github.com/users/jimmidyson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jimmidyson/subscriptions","organizations_url":"https://api.github.com/users/jimmidyson/orgs","repos_url":"https://api.github.com/users/jimmidyson/repos","events_url":"https://api.github.com/users/jimmidyson/events{/privacy}","received_events_url":"https://api.github.com/users/jimmidyson/received_events","type":"User","site_admin":false},"assignees":[{"login":"chirino","id":103255,"avatar_url":"https://avatars0.githubusercontent.com/u/103255?v=4","gravatar_id":"","url":"https://api.github.com/users/chirino","html_url":"https://github.com/chirino","followers_url":"https://api.github.com/users/chirino/followers","following_url":"https://api.github.com/users/chirino/following{/other_user}","gists_url":"https://api.github.com/users/chirino/gists{/gist_id}","starred_url":"https://api.github.com/users/chirino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chirino/subscriptions","organizations_url":"https://api.github.com/users/chirino/orgs","repos_url":"https://api.github.com/users/chirino/repos","events_url":"https://api.github.com/users/chirino/events{/privacy}","received_events_url":"https://api.github.com/users/chirino/received_events","type":"User","site_admin":false},{"login":"jimmidyson","id":464659,"avatar_url":"https://avatars1.githubusercontent.com/u/464659?v=4","gravatar_id":"","url":"https://api.github.com/users/jimmidyson","html_url":"https://github.com/jimmidyson","followers_url":"https://api.github.com/users/jimmidyson/followers","following_url":"https://api.github.com/users/jimmidyson/following{/other_user}","gists_url":"https://api.github.com/users/jimmidyson/gists{/gist_id}","starred_url":"https://api.github.com/users/jimmidyson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jimmidyson/subscriptions","organizations_url":"https://api.github.com/users/jimmidyson/orgs","repos_url":"https://api.github.com/users/jimmidyson/repos","events_url":"https://api.github.com/users/jimmidyson/events{/privacy}","received_events_url":"https://api.github.com/users/jimmidyson/received_events","type":"User","site_admin":false},{"login":"zregvart","id":1306050,"avatar_url":"https://avatars0.githubusercontent.com/u/1306050?v=4","gravatar_id":"","url":"https://api.github.com/users/zregvart","html_url":"https://github.com/zregvart","followers_url":"https://api.github.com/users/zregvart/followers","following_url":"https://api.github.com/users/zregvart/following{/other_user}","gists_url":"https://api.github.com/users/zregvart/gists{/gist_id}","starred_url":"https://api.github.com/users/zregvart/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zregvart/subscriptions","organizations_url":"https://api.github.com/users/zregvart/orgs","repos_url":"https://api.github.com/users/zregvart/repos","events_url":"https://api.github.com/users/zregvart/events{/privacy}","received_events_url":"https://api.github.com/users/zregvart/received_events","type":"User","site_admin":false}],"milestone":null,"comments":5,"created_at":"2017-09-23T20:16:38Z","updated_at":"2017-09-24T07:49:48Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/syndesisio/syndesis-rest/pulls/636","html_url":"https://github.com/syndesisio/syndesis-rest/pull/636","diff_url":"https://github.com/syndesisio/syndesis-rest/pull/636.diff","patch_url":"https://github.com/syndesisio/syndesis-rest/pull/636.patch"},"body":"When creating project files, a `CamelCatalog` and `CamelConnectorCatalog`\nis required to calculate the proper endpoint URL\n\nUp to now, this is only done via Grape which does a live\nintrospection and download of connector jars.\n\nA new plugin goal `syndesis:extract-connector-descriptor` has been\nintroduced to fetch this information offline.\nBy adding this to `META-INF/camel/camel-meta.json`, the runtime\ncan query this file and build up a CamelCatalog from this information only.\nThis avoids to reach out to the internet, which can have unexpected\nside effects related to the performance.\n\n-------\n\nThis is still work in progress. Actually the generation works nicely and has been introduced to the runtime pom.xml\nDuring startup of the `ConnectorCatalog` this `/META-INF/camel/camel-meta.json` is parsed and the `CamelCatalog` as well as the `CamelConnectorCatalog` is build up.\n\nIt was tried to mimic the way how the `MavenArtifactProvider` resolves the required data, however it still fails.\n\nWhen using the catalog on a prefetched connector, the following error happens:\n\n```\n2017-09-23 18:17:50.608 ERROR [-,,,] 1 --- [pool-5-thread-1] i.s.c.i.online.ActivateHandler           : Integration -KujxQDBjnvVpgWd0MgV : Failure\n\njava.lang.IllegalArgumentException: Endpoint with scheme twitter-mention-connector has no syntax defined in the json schema\n\tat org.apache.camel.catalog.AbstractCamelCatalog.doAsEndpointUri(AbstractCamelCatalog.java:892) ~[camel-catalog-2.20.0.fuse-000101.jar!/:2.20.0.fuse-000101]\n\tat org.apache.camel.catalog.AbstractCamelCatalog.asEndpointUri(AbstractCamelCatalog.java:869) ~[camel-catalog-2.20.0.fuse-000101.jar!/:2.20.0.fuse-000101]\n\tat io.syndesis.connector.catalog.ConnectorCatalog.buildEndpointUri(ConnectorCatalog.java:164) ~[connector-catalog-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.project.converter.visitor.EndpointStepVisitor.createEndpoint(EndpointStepVisitor.java:129) ~[project-generator-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.project.converter.visitor.EndpointStepVisitor.createEndpoint(EndpointStepVisitor.java:125) ~[project-generator-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.project.converter.visitor.EndpointStepVisitor.visit(EndpointStepVisitor.java:64) ~[project-generator-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.project.converter.DefaultProjectGenerator.visitStep(DefaultProjectGenerator.java:219) ~[project-generator-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.project.converter.DefaultProjectGenerator.lambda$generateFlowYaml$4(DefaultProjectGenerator.java:207) ~[project-generator-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat java.util.Optional.ifPresent(Optional.java:159) ~[na:1.8.0_131]\n\tat io.syndesis.project.converter.DefaultProjectGenerator.generateFlowYaml(DefaultProjectGenerator.java:184) ~[project-generator-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.project.converter.DefaultProjectGenerator.generate(DefaultProjectGenerator.java:157) ~[project-generator-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.controllers.integration.online.ActivateHandler.createProjectFiles(ActivateHandler.java:251) ~[controllers-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.controllers.integration.online.ActivateHandler.execute(ActivateHandler.java:123) ~[controllers-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat io.syndesis.controllers.integration.IntegrationController.lambda$callStatusChangeHandler$10(IntegrationController.java:175) [controllers-0.1-SNAPSHOT.jar!/:0.1-SNAPSHOT]\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_131]\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_131]\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_131]\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_131]\n\tat java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_131]\n```\n\nThe following connectors have been preloaded from the plugin's dependencies:\n\n```\n2017-09-23 18:15:32.127  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for org.apache.camel:camel-salesforce:2.20.0.fuse-000101\n2017-09-23 18:15:32.128  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-on-update-connector:0.5.5\n2017-09-23 18:15:32.130  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-on-delete-connector:0.5.5\n2017-09-23 18:15:32.135  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-update-sobject-connector:0.5.5\n2017-09-23 18:15:32.143  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-upsert-sobject-connector:0.5.5\n2017-09-23 18:15:32.156  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-create-sobject-connector:0.5.5\n2017-09-23 18:15:32.160  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-get-sobject-connector:0.5.5\n2017-09-23 18:15:32.163  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-get-sobject-with-id-connector:0.5.5\n2017-09-23 18:15:32.166  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-delete-sobject-connector:0.5.5\n2017-09-23 18:15:32.167  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:salesforce-delete-sobject-with-id-connector:0.5.5\n2017-09-23 18:15:32.170  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for io.syndesis:sql-stored-connector:0.5.5\n2017-09-23 18:15:32.172  INFO [-,,,] 1 --- [           main] i.s.connector.catalog.ConnectorCatalog   : Prefetched meta for org.apache.camel:camel-sql:2.20.0.fuse-000101\n```\n\nDon't think its a major issue, but happy for any hint to fix this as I'm on the road tomorrow and this is still \nconsidered to be a blocker for TP1. 'will continue tomorrow, but who will fix this before me, will get (a) one pint of franconian beer and (b) a mini-jar of fresh Jolokia pepper powder from 2017 ;-)\n\n@lburgazzoli @jimmidyson @zregvart @davsclaus is this good enough of an offer ;-) ?"},"comment":{"url":"https://api.github.com/repos/syndesisio/syndesis-rest/issues/comments/331693579","html_url":"https://github.com/syndesisio/syndesis-rest/pull/636#issuecomment-331693579","issue_url":"https://api.github.com/repos/syndesisio/syndesis-rest/issues/636","id":331693579,"user":{"login":"davsclaus","id":477100,"avatar_url":"https://avatars1.githubusercontent.com/u/477100?v=4","gravatar_id":"","url":"https://api.github.com/users/davsclaus","html_url":"https://github.com/davsclaus","followers_url":"https://api.github.com/users/davsclaus/followers","following_url":"https://api.github.com/users/davsclaus/following{/other_user}","gists_url":"https://api.github.com/users/davsclaus/gists{/gist_id}","starred_url":"https://api.github.com/users/davsclaus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davsclaus/subscriptions","organizations_url":"https://api.github.com/users/davsclaus/orgs","repos_url":"https://api.github.com/users/davsclaus/repos","events_url":"https://api.github.com/users/davsclaus/events{/privacy}","received_events_url":"https://api.github.com/users/davsclaus/received_events","type":"User","site_admin":false},"created_at":"2017-09-24T07:49:48Z","updated_at":"2017-09-24T07:49:48Z","author_association":"NONE","body":"The CAMEL ticket is: https://issues.apache.org/jira/browse/CAMEL-10969 and https://issues.apache.org/jira/browse/CAMEL-11005"}}
{"actor":{"display_login":"davsclaus"}}