{"action":"created","issue":{"url":"https://api.github.com/repos/KhronosGroup/glTF/issues/835","repository_url":"https://api.github.com/repos/KhronosGroup/glTF","labels_url":"https://api.github.com/repos/KhronosGroup/glTF/issues/835/labels{/name}","comments_url":"https://api.github.com/repos/KhronosGroup/glTF/issues/835/comments","events_url":"https://api.github.com/repos/KhronosGroup/glTF/issues/835/events","html_url":"https://github.com/KhronosGroup/glTF/issues/835","id":206994089,"number":835,"title":"glTF 2.0 Texture Formats","user":{"login":"lexaknyazev","id":3479527,"avatar_url":"https://avatars.githubusercontent.com/u/3479527?v=3","gravatar_id":"","url":"https://api.github.com/users/lexaknyazev","html_url":"https://github.com/lexaknyazev","followers_url":"https://api.github.com/users/lexaknyazev/followers","following_url":"https://api.github.com/users/lexaknyazev/following{/other_user}","gists_url":"https://api.github.com/users/lexaknyazev/gists{/gist_id}","starred_url":"https://api.github.com/users/lexaknyazev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lexaknyazev/subscriptions","organizations_url":"https://api.github.com/users/lexaknyazev/orgs","repos_url":"https://api.github.com/users/lexaknyazev/repos","events_url":"https://api.github.com/users/lexaknyazev/events{/privacy}","received_events_url":"https://api.github.com/users/lexaknyazev/received_events","type":"User","site_admin":false},"labels":[{"id":358193360,"url":"https://api.github.com/repos/KhronosGroup/glTF/labels/2.0","name":"2.0","color":"c5def5","default":false},{"id":32401424,"url":"https://api.github.com/repos/KhronosGroup/glTF/labels/specification","name":"specification","color":"d7e102","default":false},{"id":448917267,"url":"https://api.github.com/repos/KhronosGroup/glTF/labels/Vulkan","name":"Vulkan","color":"d4c5f9","default":false},{"id":434403498,"url":"https://api.github.com/repos/KhronosGroup/glTF/labels/WebGL%202","name":"WebGL 2","color":"fef2c0","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2017-02-11T17:18:55Z","updated_at":"2017-02-13T14:41:16Z","closed_at":null,"body":"glTF 2.0 being runtime API-agnostic format must have a robust and extendable texture handling framework.\r\n\r\nRight now, glTF has a very simple `image` object for referencing images of Web-browser compatible codecs (such as JPG or PNG) and a `texture` object for specifying desired GPU representation. glTF 1.0 doesn't provide any standard way to use native GPU formats (both compressed and uncompressed). Also it lacks support of more advanced texture usage patterns, such as cubemaps or MIP-levels.\r\n\r\nThere was an attempt (#739) to refactor `image` and `texture` object in a way that would enable aforementioned use cases, however that proposal came way before glTF 2.0 API-neutrality strategy change.\r\n\r\nRemember, we don't want to introduce any breaking changes after glTF 2.0 release, so such core functionality must be as future-proof as possible. While we don't need to enable every possible format and feature now, we must be sure that future changes could be done in a non-destructive way.\r\n\r\n---\r\n\r\n# State of cross-API formats support\r\n\r\n## GPU formats\r\n\r\nHere's an overview of GPU texture formats that could be supported across different APIs (OpenGL ES 2.0/3.0, OpenGL 4.5, Vulkan, Metal, D3D11/12). Keep in mind, that in some cases API support doesn't guarantee hardware support. OpenGL ES 2.0 supports very few of them, so it's present there only for reference.\r\n\r\n### Uncompressed (byte-aligned) formats\r\n\r\nThe following table contains a matched list of byte-aligned GPU formats. Formats, exclusive to only one API are not mentioned.\r\n\r\n- __Green:__ All modern APIs support.\r\n- __Yellow:__ Most modern APIs support natively (OpenGL ES 3.0 extensions could be required for some).\r\n- __Red:__ Support is non-uniform.\r\n\r\n![image](https://cloud.githubusercontent.com/assets/3479527/22855489/b20372f0-f09a-11e6-9e14-058413e397c9.png)\r\n\r\n### Packed formats\r\n\r\nThe following table contains a matched list of packed GPU formats.\r\n\r\n- __Green:__ All modern APIs support (there's exception with D3D for formats available in WebGL 1.0).\r\n- __Yellow:__ Most modern APIs support (no OpenGL ES 3.0).\r\n- __Red:__ Native support is non-uniform.\r\n\r\n![image](https://cloud.githubusercontent.com/assets/3479527/22855520/e869c47a-f09a-11e6-8722-e13a86d45d90.png)\r\n\r\n\r\n### Compressed formats\r\n\r\nThe following table contains a matched list of compressed GPU formats. Actual support varies both by API and hardware. Note, that ETC1 format isn't mentioned there, however ETC2-enabled systems support it.\r\n\r\n- __Yellow:__ BC1 RGB\r\n- __Green:__ BCn family\r\n- __Gray:__ PVRTC\r\n- __Blue:__ ETC2/EAC\r\n- __Red:__ ASTC\r\n\r\n![image](https://cloud.githubusercontent.com/assets/3479527/22855535/46a3e642-f09b-11e6-98be-2c71c24c649a.png)\r\n\r\n\r\n## Web formats\r\n\r\nThese formats are universally supported in web browsers, but supporting them in mobile or embedded environment could be inefficient. They require client-side decompression, hence, client RAM and CPU cycles. GPU will got uncompressed data. Client can do recompression to GPU-friendly format, hence even more processing.\r\n\r\n### BMP\r\nMany versions of format exist. Could have anything from 1 bit-per-pixel BW to 32-bit RGBA. Browser support for rare combinations varies (e.g., look at [comments in the Chromium source](https://chromium.googlesource.com/chromium/blink.git/+/master/Source/platform/image-decoders/bmp/BMPImageReader.cpp#440)).\r\nWhy was it allowed in the glTF 1.0?\r\n\r\n### GIF\r\nCould have 24-bit RGB colors, alpha is limited to 1-bit. LZW compression.\r\n\r\n### PNG\r\nCould have L, LA, RGB, or RGBA data with up to 16-bit per channel. Deflate compression.\r\n\r\n### JPEG\r\nCould have 24-bit RGB or 8-bit Luminance. No alpha. \r\n\r\n\r\n# State of glTF texture support\r\n\r\n## Issues\r\n\r\n- `texture.format`, `texture.internalformat`, and `texture.type` make little sense with Web formats. Other formats aren't supported, though.\r\n\r\n- Extending `texture.target` beyond `GL_TEXTURE_2D` (e.g. cubemaps) requires a way to associate multiple images with one texture or use some sort of image container format. \r\nWe need to support some use cases, but mustn't introduce breaking changes within 2.x lifecycle.\r\n\r\n- glTF 1.0 assumes WebGL unpacking rule (upper left corner first). This is an issue for [export workflow](https://github.com/sketchfab/Unity-glTF-Exporter#flipy-texture-flag).\r\n\r\n- GPU-friendly formats require full data specification (width, height, pixel type). There's no standard way in glTF 1.0 to address that.\r\n\r\n## KTX proposal\r\n\r\nTo address some of issues above, KTX format support was proposed in #739. Main changes include (keep in mind that they are breaking from 1.0, so such decision could only be done with major version upgrade):\r\n\r\n- Allow `image/ktx` as a valid image MIME-type and container format. KTX container will handle stuff like cubemaps, MIP levels, etc. Loading KTX in GL (ES) is trivial, other API require mapping values from KTX header to their enums (see tables above).\r\n\r\n- Make `image` contain an array of its representations in different formats. These could include both Web and GPU formats. Different GPU-compressed formats could be provided for asset compatibility.\r\n\r\n- For web formats to achieve feature-parity with KTX, use JSON as metadata container and refer actual data via array of URIs or bufferViews.\r\n\r\n- Possibly, move Web formats support out of the core or not require them outside of web environment (so mobile clients won't have to implement them).\r\n\r\nSuch layout will allow adding more container formats (like Crunch or Basis) in post-2.0 minor updates. \r\n\r\n### Example (adapted from #739)\r\n\r\nThis is just an example, comments are welcome. `undefined` used for illustrative purposes.\r\n\r\n```javascript\r\n{\r\n    \"textures\": [\r\n        {\r\n            \"image\": 0,\r\n            \"sampler\": 0\r\n        }\r\n    ],\r\n    \"images\": [\r\n        {\r\n            \"formats\": [\r\n                {\r\n                    \"format\": 33779, // GL_COMPRESSED_RGBA_S3TC_DXT5_EXT\r\n                    \"mimeType\": \"image/ktx\",\r\n                    \"uri\": undefined,\r\n                    \"bufferView\": 0\r\n                },\r\n                {\r\n                    \"format\": 32856, // GL_RGBA8\r\n                    \"mimeType\": \"image/png\",  // valid only when \"KHR_image_web\" enabled\r\n                    \"uri\": undefined,\r\n                    \"bufferView\": undefined,\r\n                    \"extensions\": {\r\n                        \"KHR_image_web\": {\r\n                            \"flipY\": true,\r\n                            \"width\": 256,\r\n                            \"height\": 256,  // specify 0 for a 1D texture\r\n                            \"depth\": 1, // optional; depth of mip level 0 of a 3D texture; must be one for 2D and cube textures\r\n                            \"layers\": 1, // optional; used for 2D texture arrays and cube map arrays\r\n                            \"faces\": 1, // optional; 1 or 6 (cube map)\r\n                            \"levels\": 2, // optional; 0 means run-time should call generateMipmap()\r\n                            \"uri\": undefined,\r\n                            \"bufferViews\": [  // list images in the order: depth-layers-faces-levels\r\n                                1,\r\n                                2\r\n                            ]\r\n                        }\r\n                    }\r\n                }\r\n            ]\r\n        }\r\n    ]\r\n}\r\n```\r\n\r\n# Final remarks\r\n\r\nFor mass assets distribution it's vital to use compressed image data. At some point, we should expect libraries like Basis or Crunch to be universally integrated in exporting workflows.\r\n\r\nKTX usage of more than one parameter (`glFormat`, `glInternalFormat`, `glBaseInternalFormat`, `glType`) to describe image format is sub-optimal. It also locks valid formats to those supported by GL. \r\n\r\nKTX data layout isn't streaming-friendly: it's hard to fetch low-res image first.\r\n\r\nMore modern container (KTX2?) could be proposed later.\r\n\r\n\r\n# References\r\n\r\n- OpenGL ES 2.0/3.0, OpenGL 4.5 specs\r\n- [VkFormat](https://www.khronos.org/registry/vulkan/specs/1.0/man/html/VkFormat.html)\r\n- [MTLPixelFormat](https://developer.apple.com/reference/metal/mtlpixelformat)\r\n- [DXGI_FORMAT](https://msdn.microsoft.com/en-us/library/windows/desktop/bb173059(v=vs.85).aspx)\r\n- [KTX File Format Specification](https://www.khronos.org/opengles/sdk/tools/KTX/file_format_spec/)\r\n---\r\n\r\nCC @pjcozzi @javagl @sbtron @bghgary @AurL @cedricpinson @mlimper @lasalvavida"},"comment":{"url":"https://api.github.com/repos/KhronosGroup/glTF/issues/comments/279410405","html_url":"https://github.com/KhronosGroup/glTF/issues/835#issuecomment-279410405","issue_url":"https://api.github.com/repos/KhronosGroup/glTF/issues/835","id":279410405,"user":{"login":"pjcozzi","id":782098,"avatar_url":"https://avatars.githubusercontent.com/u/782098?v=3","gravatar_id":"","url":"https://api.github.com/users/pjcozzi","html_url":"https://github.com/pjcozzi","followers_url":"https://api.github.com/users/pjcozzi/followers","following_url":"https://api.github.com/users/pjcozzi/following{/other_user}","gists_url":"https://api.github.com/users/pjcozzi/gists{/gist_id}","starred_url":"https://api.github.com/users/pjcozzi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pjcozzi/subscriptions","organizations_url":"https://api.github.com/users/pjcozzi/orgs","repos_url":"https://api.github.com/users/pjcozzi/repos","events_url":"https://api.github.com/users/pjcozzi/events{/privacy}","received_events_url":"https://api.github.com/users/pjcozzi/received_events","type":"User","site_admin":false},"created_at":"2017-02-13T14:41:16Z","updated_at":"2017-02-13T14:41:16Z","body":"Ah, I see.  Yes, I think always using arrays (with length === 1 for KTX) is reasonable."}}
{"actor":{"display_login":"pjcozzi"}}