{"action":"created","issue":{"url":"https://api.github.com/repos/dotnet/corefx/issues/24607","repository_url":"https://api.github.com/repos/dotnet/corefx","labels_url":"https://api.github.com/repos/dotnet/corefx/issues/24607/labels{/name}","comments_url":"https://api.github.com/repos/dotnet/corefx/issues/24607/comments","events_url":"https://api.github.com/repos/dotnet/corefx/issues/24607/events","html_url":"https://github.com/dotnet/corefx/issues/24607","id":264998743,"number":24607,"title":"UTF8 Parsing and Formatting","user":{"login":"AtsushiKan","id":13223578,"avatar_url":"https://avatars0.githubusercontent.com/u/13223578?v=4","gravatar_id":"","url":"https://api.github.com/users/AtsushiKan","html_url":"https://github.com/AtsushiKan","followers_url":"https://api.github.com/users/AtsushiKan/followers","following_url":"https://api.github.com/users/AtsushiKan/following{/other_user}","gists_url":"https://api.github.com/users/AtsushiKan/gists{/gist_id}","starred_url":"https://api.github.com/users/AtsushiKan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AtsushiKan/subscriptions","organizations_url":"https://api.github.com/users/AtsushiKan/orgs","repos_url":"https://api.github.com/users/AtsushiKan/repos","events_url":"https://api.github.com/users/AtsushiKan/events{/privacy}","received_events_url":"https://api.github.com/users/AtsushiKan/received_events","type":"User","site_admin":false},"labels":[{"id":163655927,"url":"https://api.github.com/repos/dotnet/corefx/labels/api-approved","name":"api-approved","color":"159818","default":false},{"id":306867402,"url":"https://api.github.com/repos/dotnet/corefx/labels/area-System.Buffers","name":"area-System.Buffers","color":"d4c5f9","default":false}],"state":"open","locked":false,"assignee":{"login":"KrzysztofCwalina","id":9724236,"avatar_url":"https://avatars0.githubusercontent.com/u/9724236?v=4","gravatar_id":"","url":"https://api.github.com/users/KrzysztofCwalina","html_url":"https://github.com/KrzysztofCwalina","followers_url":"https://api.github.com/users/KrzysztofCwalina/followers","following_url":"https://api.github.com/users/KrzysztofCwalina/following{/other_user}","gists_url":"https://api.github.com/users/KrzysztofCwalina/gists{/gist_id}","starred_url":"https://api.github.com/users/KrzysztofCwalina/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KrzysztofCwalina/subscriptions","organizations_url":"https://api.github.com/users/KrzysztofCwalina/orgs","repos_url":"https://api.github.com/users/KrzysztofCwalina/repos","events_url":"https://api.github.com/users/KrzysztofCwalina/events{/privacy}","received_events_url":"https://api.github.com/users/KrzysztofCwalina/received_events","type":"User","site_admin":false},"assignees":[{"login":"KrzysztofCwalina","id":9724236,"avatar_url":"https://avatars0.githubusercontent.com/u/9724236?v=4","gravatar_id":"","url":"https://api.github.com/users/KrzysztofCwalina","html_url":"https://github.com/KrzysztofCwalina","followers_url":"https://api.github.com/users/KrzysztofCwalina/followers","following_url":"https://api.github.com/users/KrzysztofCwalina/following{/other_user}","gists_url":"https://api.github.com/users/KrzysztofCwalina/gists{/gist_id}","starred_url":"https://api.github.com/users/KrzysztofCwalina/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KrzysztofCwalina/subscriptions","organizations_url":"https://api.github.com/users/KrzysztofCwalina/orgs","repos_url":"https://api.github.com/users/KrzysztofCwalina/repos","events_url":"https://api.github.com/users/KrzysztofCwalina/events{/privacy}","received_events_url":"https://api.github.com/users/KrzysztofCwalina/received_events","type":"User","site_admin":false},{"login":"AtsushiKan","id":13223578,"avatar_url":"https://avatars0.githubusercontent.com/u/13223578?v=4","gravatar_id":"","url":"https://api.github.com/users/AtsushiKan","html_url":"https://github.com/AtsushiKan","followers_url":"https://api.github.com/users/AtsushiKan/followers","following_url":"https://api.github.com/users/AtsushiKan/following{/other_user}","gists_url":"https://api.github.com/users/AtsushiKan/gists{/gist_id}","starred_url":"https://api.github.com/users/AtsushiKan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AtsushiKan/subscriptions","organizations_url":"https://api.github.com/users/AtsushiKan/orgs","repos_url":"https://api.github.com/users/AtsushiKan/repos","events_url":"https://api.github.com/users/AtsushiKan/events{/privacy}","received_events_url":"https://api.github.com/users/AtsushiKan/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/dotnet/corefx/milestones/12","html_url":"https://github.com/dotnet/corefx/milestone/12","labels_url":"https://api.github.com/repos/dotnet/corefx/milestones/12/labels","id":2323293,"number":12,"title":"2.1.0","description":".NET Core 2.1, next major release. See [official release dates](https://github.com/dotnet/core/blob/master/roadmap.md) for details.","creator":{"login":"karelz","id":5776068,"avatar_url":"https://avatars2.githubusercontent.com/u/5776068?v=4","gravatar_id":"","url":"https://api.github.com/users/karelz","html_url":"https://github.com/karelz","followers_url":"https://api.github.com/users/karelz/followers","following_url":"https://api.github.com/users/karelz/following{/other_user}","gists_url":"https://api.github.com/users/karelz/gists{/gist_id}","starred_url":"https://api.github.com/users/karelz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/karelz/subscriptions","organizations_url":"https://api.github.com/users/karelz/orgs","repos_url":"https://api.github.com/users/karelz/repos","events_url":"https://api.github.com/users/karelz/events{/privacy}","received_events_url":"https://api.github.com/users/karelz/received_events","type":"User","site_admin":false},"open_issues":356,"closed_issues":2625,"state":"open","created_at":"2017-02-15T02:14:11Z","updated_at":"2017-10-24T19:13:45Z","due_on":null,"closed_at":null},"comments":14,"created_at":"2017-10-12T16:05:04Z","updated_at":"2017-10-24T19:30:02Z","closed_at":null,"author_association":"MEMBER","body":"# UTF8 Parsing\r\n\r\nThe current .NET Framework parsing APIs (e.g. int.TryParse) can parse text represented by System.String (UTF16).\r\n```c#\r\nstring text = ...\r\nif(int.TryParse(text, out int value)) { ... }\r\n```\r\n\r\nThese APIs work great in many scenarios, e.g. parsing text contained in GUI application's text boxes. They are not suitable for processing modern network protocols, which are often text (e.g. JSON, HTTP headers), but encoded with UTF8/ASCII, not UTF16, and contained in byte buffers, not strings. Because of this, all modern web servers written for the .NET platform either don't use the current BCL parsing APIs or take a performance hit to transcode between UTF8 and UTF16, and to copy from buffers to strings.\r\n\r\nTo address modern networking scenarios better, we will provide parsing APIs that:\r\n\r\n- Can parse byte buffers without the need to have a `String` to parse\r\n- Can parse text encoded as UTF8 (and possibly other encodings)\r\n- Can parse without any GC heap allocations\r\n\r\n# Proposed Api\r\n\r\n```c#\r\nnamespace System.Buffers.Text\r\n{\r\n   public struct ParsedFormat \r\n   {\r\n        public ParsedFormat(char symbol, byte precision=NoPrecision);\r\n\r\n        public char Symbol { get; }\r\n\r\n        public bool HasPrecision { get; }\r\n        public byte Precision { get; }\r\n\r\n        public bool IsDefault { get; }\r\n\r\n        public static ParsedFormat Parse(ReadOnlySpan<char> format);\r\n        public static ParsedFormat Parse(string format);\r\n\r\n        public static implicit operator ParsedFormat (char symbol);\r\n\r\n        public const byte MaxPrecision = (byte)99;\r\n        public const byte NoPrecision = (byte)255;\r\n    }\r\n\r\n    public static class Utf8Formatter\r\n    {\r\n        public static bool TryFormat(bool value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(byte value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(DateTime value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(DateTimeOffset value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(decimal value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(double value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(Guid value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(short value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(int value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(long value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(sbyte value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(float value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(TimeSpan value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(ushort value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(uint value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n        public static bool TryFormat(ulong value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\n     }  \r\n\r\n     public static class Utf8Parser \r\n     {\r\n        public static bool TryParseBoolean(ReadOnlySpan<byte> text, out bool value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseByte(ReadOnlySpan<byte> text, out byte value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseDateTime(ReadOnlySpan<byte> text, out DateTimevalue, out int bytesConsumed, char format = null);\r\n        public static bool TryParseDateTimeOffset(ReadOnlySpan<byte> text, out DateTimeOffset value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseDouble(ReadOnlySpan<byte> text, out double value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseDecimal(ReadOnlySpan<byte> text, out decimal value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseGuid(ReadOnlySpan<byte> text, out Guid value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseInt16(ReadOnlySpan<byte> text, out short value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseInt32(ReadOnlySpan<byte> text, out int value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseInt64(ReadOnlySpan<byte> text, out long value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseSByte(ReadOnlySpan<byte> text, out sbyte value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseSingle(ReadOnlySpan<byte> text, out float value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseTimeSpan(ReadOnlySpan<byte> text, out TimeSpan value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseUInt16(ReadOnlySpan<byte> text, out ushort value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseUInt32(ReadOnlySpan<byte> text, out uint value, out int bytesConsumed, char format = null);\r\n        public static bool TryParseUInt64(ReadOnlySpan<byte> text, out ulong value, out int bytesConsumed, char format = null);\r\n    }\r\n}\r\n```\r\n\r\nThe APIs are policy free (not specific to scenarios, higher level frameworks, etc.) and optimized for speed, and will look like the following (per data type being parsed; the sample shows Int32 APIs):\r\n\r\n```\r\npublic static bool TryFormat(int value, Span<byte> buffer, out int bytesWritten, ParsedFormat format=null);\r\npublic static bool TryParseInt32(ReadOnlySpan<byte> text, out int value, out int bytesConsumed, char format = null);\r\n```\r\n\r\n# Cultures\r\n\r\nAPIs listed in this proposal support only the invariant culture. In corfxlab, we have APIs that support culture-aware [formatting ](https://github.com/dotnet/corefxlab/blob/master/src/System.Text.Primitives/System/Text/Formatters/Custom.Integer.cs) and [parsing](https://github.com/dotnet/corefxlab/blob/master/src/System.Text.Primitives/System/Text/Parsers/Signed.cs#L68) of basic primitive types, but these APIs are not part of this proposal, as they are not strictly needed by SignalR. \r\n\r\n# ParsedFormat\r\n\r\n`ParsedFormat` is an efficient (non allocating, preparsed) representation of the standard format strings. It's used in the formatting APIs. If `default(ParsedFormat)` is passed, a type-specific default (typically 'G') is used.\r\n\r\nFor the parsing API's, we pass only a `char` as the \"precision\" component is meaningless when parsing. Furthermore, format specifiers that differ only in upper/lowercasing out of the output are treated as equivalent by the Parse methods. Thus, passing 'X' or 'x' to `TryParseInt32` is equivalent (the parser will accept either upper or lower case or mixed hex characters.) If `default(char)` is passed, a type-specific default (typically 'G') is used.\r\n\r\nApis that attempt to parse all possible formats were considered but this approach would add overhead and make the future addition of formats an event that changes prior API behavior.\r\n\r\nMore information about the parsing APIs is avaliable [here](https://github.com/dotnet/corefxlab/blob/master/docs/specs/parsing.md)"},"comment":{"url":"https://api.github.com/repos/dotnet/corefx/issues/comments/339104707","html_url":"https://github.com/dotnet/corefx/issues/24607#issuecomment-339104707","issue_url":"https://api.github.com/repos/dotnet/corefx/issues/24607","id":339104707,"user":{"login":"nil4","id":139133,"avatar_url":"https://avatars1.githubusercontent.com/u/139133?v=4","gravatar_id":"","url":"https://api.github.com/users/nil4","html_url":"https://github.com/nil4","followers_url":"https://api.github.com/users/nil4/followers","following_url":"https://api.github.com/users/nil4/following{/other_user}","gists_url":"https://api.github.com/users/nil4/gists{/gist_id}","starred_url":"https://api.github.com/users/nil4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nil4/subscriptions","organizations_url":"https://api.github.com/users/nil4/orgs","repos_url":"https://api.github.com/users/nil4/repos","events_url":"https://api.github.com/users/nil4/events{/privacy}","received_events_url":"https://api.github.com/users/nil4/received_events","type":"User","site_admin":false},"created_at":"2017-10-24T19:30:02Z","updated_at":"2017-10-24T19:30:02Z","author_association":"NONE","body":"The `format` parameter is a struct in both `Utf8Formatter` and `Utf8Parser`: *A value of type '`<null>`' cannot be used as a default parameter because there are no standard conversions to type 'char'*. \r\n\r\nShould it be nullable or declared `= default` instead?"}}
{"actor":{"display_login":"nil4"}}