{"action":"created","issue":{"url":"https://api.github.com/repos/json-api/json-api/issues/1207","repository_url":"https://api.github.com/repos/json-api/json-api","labels_url":"https://api.github.com/repos/json-api/json-api/issues/1207/labels{/name}","comments_url":"https://api.github.com/repos/json-api/json-api/issues/1207/comments","events_url":"https://api.github.com/repos/json-api/json-api/issues/1207/events","html_url":"https://github.com/json-api/json-api/issues/1207","id":247788173,"number":1207,"title":"Overview: Extending and describing the spec","user":{"login":"dgeb","id":29122,"avatar_url":"https://avatars0.githubusercontent.com/u/29122?v=4","gravatar_id":"","url":"https://api.github.com/users/dgeb","html_url":"https://github.com/dgeb","followers_url":"https://api.github.com/users/dgeb/followers","following_url":"https://api.github.com/users/dgeb/following{/other_user}","gists_url":"https://api.github.com/users/dgeb/gists{/gist_id}","starred_url":"https://api.github.com/users/dgeb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dgeb/subscriptions","organizations_url":"https://api.github.com/users/dgeb/orgs","repos_url":"https://api.github.com/users/dgeb/repos","events_url":"https://api.github.com/users/dgeb/events{/privacy}","received_events_url":"https://api.github.com/users/dgeb/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2017-08-03T18:00:42Z","updated_at":"2017-08-04T21:06:53Z","closed_at":null,"body":"Apologies for the broadness of this issue, which overlaps with many other issues\r\n(see #1195 for perhaps the most up to date history and list of issues). However,\r\nI think it's vital that we agree on some fundamentals before moving forward to\r\naccept means to describe and extend usage of the base specification.\r\n\r\n---\r\n\r\nWhen viewed from the perspective of conformance, there are effectively\r\nthree ways to consider expressing additional semantics for the base\r\nspecification:\r\n\r\n* **conformant** - Semantics are expressed within the areas of the spec\r\n  explicitly reserved for implementations.\r\n\r\n* **additive** - Semantics are assigned to structural or procedural\r\n  elements that are reserved but not specified.\r\n\r\n* **breaking** - Structural or procedural elements are assigned semantics that\r\n  conflict with the spec.\r\n\r\nEach category is discussed below.\r\n\r\n### Breaking extensions\r\n\r\nAlthough the concept of breaking extensions was considered in 2014-2015, there\r\nhas been significant pushback (see #614) against this concept. I believe there's\r\nnow fairly unanimous agreement from the editors and the community that creating\r\na new derivative media type would be preferable to allowing breaking extensions in\r\nthe spec.\r\n\r\n### Additive extensions\r\n\r\nAdditive extensions would cover allowing additional member(s) in the reserved\r\nareas of the document structure. It's also possible for additive extensions to\r\ndictate how clients and servers should interpret those additional members.\r\n\r\nLet's look at a few examples of additive extensions that have been proposed:\r\n\r\n* **Bulk posting** - This extension would allow multiple resources to be created\r\n  in a single request. It would expect that top-level `data` be passed in as\r\n  an array, and that servers create all resources or none. Created resources\r\n  would be returned in an array.\r\n\r\n* **Side posting** - This extension is being explored in #1197, which\r\n  allows related resources to be created in the `included` array.\r\n\r\n* **Additive / subtractive fields** - This extension has been proposed in\r\n  #1176, which allows fields to be specified relative to an understanding\r\n  of a default set of fields per type.\r\n\r\n* **Operations** - I'll be proposing this extension in the next several days.\r\n  It will introduce a new `operations` array that can be used to process\r\n  multiple actions, including fetching and mutating resources, serially and\r\n  transactionally.\r\n\r\nWe need to be _extremely_ careful when discussing additive extensions. If\r\nadditive extensions are not curated by the editors of the spec itself, it is\r\ninevitable that changes to the spec will break additive extensions.\r\n\r\nA key requirement for additive extensions is that it would be technically\r\npossible for them to eventually become part of the base spec, even if we choose\r\nto keep them as extensions indefinitely. Therefore, additive extensions can\r\nnot express any demands on non-normative areas of the spec, such as `meta`\r\nvalues.\r\n\r\nTo avoid fragmentation in implementations of the spec, we should not introduce\r\nextensions with largely overlapping concerns. We should ensure that any additive\r\nextensions can work together.\r\n\r\nThe usage of additive extensions must be negotiated, probably through a media\r\ntype parameter. For an overview of how media type parameter negotiation might\r\noccur, see [this deprecated page on extensions](https://github.com/json-api/json-api/blob/9c7a03dbc37f80f6ca81b16d444c960e96dd7a57/extensions/index.md#-extension-negotiation).\r\n\r\n### Conformant mechanisms\r\n\r\nMechanisms to describe spec structure and usage that fully conform to the spec\r\nare tightly constrained. These constraints free us (the editors and the\r\ncommunity) from concerns about managing different implementations, ensuring the\r\nreasoning behind each is sound, and restricting overlap.\r\n\r\nEven within the constraints of the spec, there's still a lot of flexibility\r\navailable to implementations. There can be benefits to codifying and sharing\r\nthe choices made by implementations.\r\n\r\nAlthough we do not need to be concerned about how Organization A is\r\nusing the `page` parameter, it would be nice if Organization A could share\r\ntheir methodologies with the broader community so that Organization B might\r\nbenefit. And it would be even nicer if tooling were developed that shared these\r\nunderstandings so that Organization C could benefit without thinking deeply\r\nabout the implementation.\r\n\r\nThis aligns quite well with the concept of a \"profile\" as described in\r\n[RFC 6906](https://tools.ietf.org/html/rfc6906):\r\n\r\n> A media type defines both the semantics and the serialization of a\r\n  specific type of content.  In many cases, media types have some\r\n  built-in extensibility or openness, so that specific instances of the\r\n  media type can layer additional semantics on top of the media type's\r\n  foundation.  In this case, a profile is the appropriate mechanism to\r\n  signal that the original semantics and processing model of the media\r\n  type still apply, but that an additional processing model can be used\r\n  to extract additional semantics.\r\n\r\n@ethanresnick has explored these concepts these concepts in detail and has\r\nproposed that these be called \"profile extensions\", as outlined in\r\nhttps://github.com/json-api/json-api/pull/1195\r\n\r\nEthan's proposal allows profile extensions to be associated with structural\r\nelements in the document. However, I'm also concerned about allowing for\r\nspecification of _any_ unreserved areas of the spec, including structural\r\nelements like attributes, relationships, and meta, but also query params like\r\n`filter`, `page`, etc. It also seems in keeping with the spirit of a profile to\r\nallow the combination of additional semantics in a single profile - including\r\nboth usage and structural expectations.\r\n\r\nFor instance, let's take the `page` example above. Imagine developers want to\r\nshare a common usage of offset / limit pagination that uses the `page`\r\nparameter (e.g. `/articles?page[offset]=0&page[limit]=10`). And they want to\r\nallow servers to specify a `page` member in the top-level `meta` with the\r\n`total` member. It seems beneficial that these concepts could be wrapped\r\ntogether in a single \"profile\" or \"profile extension\", so that\r\nimplementations can agree on both structure and usage atomically.\r\n\r\nAre there other places better suited for the expression of capabilities? For\r\ninstance, we have discussed the \"home document\" proposal, as described in\r\nhttps://mnot.github.io/I-D/json-home/  Are there other options we have not yet\r\nconsidered?\r\n\r\n----\r\n\r\nI'm opening this issue to discuss a number of matters:\r\n\r\n* Do we agree on the basic classifications above?\r\n\r\n* Should negotiation mechanisms differ for additive vs. conformant\r\n  extensions (i.e. \"profiles\")?\r\n\r\n* What are the appropriate boundaries for a profile extension? Should they\r\n  include structure + usage?\r\n\r\n* Are there other standards, such as home documents and schemas, that should\r\n  be considered?\r\n\r\n* Is consensus strong enough re: additive extensions that we should consider\r\n  focusing on the extension mechanism before anything profile-related?\r\n\r\nI'd like to start as simply and conservatively as possible. But we also need to\r\nthink wholistically. We should establish a path by which every choice made by a\r\nparticular implementation of the spec can be described, standardized, and\r\nshared.\r\n\r\nThis is a great time to express your questions, concerns, and opinions about\r\nextending the specification."},"comment":{"url":"https://api.github.com/repos/json-api/json-api/issues/comments/320353362","html_url":"https://github.com/json-api/json-api/issues/1207#issuecomment-320353362","issue_url":"https://api.github.com/repos/json-api/json-api/issues/1207","id":320353362,"user":{"login":"dgeb","id":29122,"avatar_url":"https://avatars0.githubusercontent.com/u/29122?v=4","gravatar_id":"","url":"https://api.github.com/users/dgeb","html_url":"https://github.com/dgeb","followers_url":"https://api.github.com/users/dgeb/followers","following_url":"https://api.github.com/users/dgeb/following{/other_user}","gists_url":"https://api.github.com/users/dgeb/gists{/gist_id}","starred_url":"https://api.github.com/users/dgeb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dgeb/subscriptions","organizations_url":"https://api.github.com/users/dgeb/orgs","repos_url":"https://api.github.com/users/dgeb/repos","events_url":"https://api.github.com/users/dgeb/events{/privacy}","received_events_url":"https://api.github.com/users/dgeb/received_events","type":"User","site_admin":false},"created_at":"2017-08-04T21:06:53Z","updated_at":"2017-08-04T21:06:53Z","body":"@ethanresnick Hey Ethan - always appreciate your thoughts! Thanks for weighing in. I can't respond to everything at the moment, but will try to zoom in on a central theme right now.\r\n\r\n> However, I'm not sure that the way to acheive this is to let extensions dictate query parameter usage. Allowing extensions to specify how query parameters work (or anything about the URI) opens back up the possibility of extensions conflicting, both with one another and with other query parameters the server is already using.\r\n\r\nIf JSON:API did not already reserve and provide semantics for query parameters, such as `page` and `filter`, I would absolutely agree with your concerns. When I suggest that a conformant extension could dictate query parameter usage, I am restricting the discussion to query parameters already reserved by the spec. This space is just as much a part of the spec as the document structure. \r\n\r\n> Because of this risk of collision (and a couple other things), having a specification dictate anything about URLs violates an IETF best practice (RFC 7320), and the logic in that best practice all seems to apply to JSON:API extensions.\r\n\r\nThe base spec makes pragmatic concessions to IETF best practices recommendations by narrowly defining usage of specific query parameters. These are the only aspects of the URL that are reserved by the spec. I suppose we shouldn't try to re-litigate this post-1.0. I'm just pointing out that, for better or worse, any best practices violations have already been made by the base spec.\r\n\r\nI think the bottom line is that I am probably less concerned about _managing_ conflicts between conformant extensions than you are. Given that we can't constrain the creation and use of conformant extensions, any number of them _will_ be created to provide conflicting semantics for a particular aspect of document structure, such as a `meta` value, or usage, such as a `filter` parameter. However, I see it as the responsibility of implementations to choose to support non-conflicting extensions, just as it's the responsibility of implementations to design rational endpoints and to choose which aspects of the base spec to support.\r\n\r\n> The big approach that we haven't discussed much (in the context of extensions), but that is the conventional solution to this problem in the REST school of thought, is links/request templating.\r\n\r\nI'm definitely not opposed to expanding support for link templating. But I'm not sure it's a panacea for fully spec'ing out query param options. Imagine a complex filtering scheme described by an extension that specifies that the value of a filter be an encoded json object in the form `filter[field]={op: \"gt|lt|gte|lte|eq\", value: value}`. I'm not clear on the matrix of link templates that would need to be used to fully describe such a scheme across many possible filterable fields. It seems far more straightforward to just write out the usage expectations in an extension/profile spec and link to its URI, which seems inline with the approach from [RFC 6906](https://tools.ietf.org/html/rfc6906).\r\n\r\nAnyway, I think that we agree more than we disagree, so sorry to focus on what I see as the main disagreement. Maybe we need to bring more use cases and perspectives into this conversation to help us work through this. @wycats ? @steveklabnik ?\r\n\r\nAnd I'll try to respond to your other points ASAP.\r\n"}}
{"actor":{"display_login":"dgeb"}}