{"action":"created","issue":{"url":"https://api.github.com/repos/Microsoft/visualfsharp/issues/2250","repository_url":"https://api.github.com/repos/Microsoft/visualfsharp","labels_url":"https://api.github.com/repos/Microsoft/visualfsharp/issues/2250/labels{/name}","comments_url":"https://api.github.com/repos/Microsoft/visualfsharp/issues/2250/comments","events_url":"https://api.github.com/repos/Microsoft/visualfsharp/issues/2250/events","html_url":"https://github.com/Microsoft/visualfsharp/pull/2250","id":200691840,"number":2250,"title":"Improve and dogfood the dotnet core sdk support","user":{"login":"enricosada","id":147243,"avatar_url":"https://avatars.githubusercontent.com/u/147243?v=3","gravatar_id":"","url":"https://api.github.com/users/enricosada","html_url":"https://github.com/enricosada","followers_url":"https://api.github.com/users/enricosada/followers","following_url":"https://api.github.com/users/enricosada/following{/other_user}","gists_url":"https://api.github.com/users/enricosada/gists{/gist_id}","starred_url":"https://api.github.com/users/enricosada/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/enricosada/subscriptions","organizations_url":"https://api.github.com/users/enricosada/orgs","repos_url":"https://api.github.com/users/enricosada/repos","events_url":"https://api.github.com/users/enricosada/events{/privacy}","received_events_url":"https://api.github.com/users/enricosada/received_events","type":"User","site_admin":false},"labels":[{"id":197650126,"url":"https://api.github.com/repos/Microsoft/visualfsharp/labels/cla-already-signed","name":"cla-already-signed","color":"009800","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2017-01-13T18:02:40Z","updated_at":"2017-02-01T17:12:42Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Microsoft/visualfsharp/pulls/2250","html_url":"https://github.com/Microsoft/visualfsharp/pull/2250","diff_url":"https://github.com/Microsoft/visualfsharp/pull/2250.diff","patch_url":"https://github.com/Microsoft/visualfsharp/pull/2250.patch"},"body":"This PR\r\n\r\n1. Uses latest released .NET Core Sdk (the latest dotnet cli) preview4 to build and publish fsc\r\n2. Creates a nuget package ``Microsoft.FSharp.Compiler.Sdk.netcore`` for use in .net core sdk fsproj.\r\n\r\nAdditional info, and some context about how the new msbuild/fsproj/sdk work  https://gist.github.com/enricosada/539da221ef3d87f0151d395f7889f94c\r\n\r\nThis direction will eventually lead to the removal of project.json for the Core CLR build, leading to the use of the new `fsproj` msbuild based, with a normal msbuild-based templates for .net core sdk projects** (so exactly like `dotnet new -l fsharp -t console`).\r\n\r\nThis gives us the basics we need to polish and finalize support for the new msbuild+fsproj format, and dogfood .the net core sdk fsproj in visualfsharp repo.  This will be crucial to ensure the extensibility points in the SDK are adequate for our needs.\r\n\r\nSteps in the PR are:\r\n* install correct sdk \r\n* restore \r\n* build fsc\r\n* publish\r\n* package\r\n* test end to end of the built packages in isolation.\r\n\r\nThis PR builds and packages just fsc (not fsharp.core or fsharp.compiler) - the other already built assemblies are used as normal `<Reference />`. So this can be run in parallel to the existing build script for .NET Core to use the advanced functionality of .net core sdk (like publish) and the other projects \r\ncan be converted in future steps.\r\n\r\nThe script is added as additional step in `build.cmd` when passing the `netcoresdk` argument.  Generated packages are put in the usual `$(Configuration)\\artifacts` directory.\r\n\r\nThis PR makes no changes to the current build script/code for .NET Core (the `coreclr`) argument, that's why all code is addition, but duplicated code is minimal.\r\n\r\n### Documentation\r\n\r\n* To use the .net core sdk script: see `./scripts/use_dotnet_core_sdk.README.md`\r\n\r\n* Build script documentation: see comment at end of `./build-using-netcore-sdk.proj`\r\n\r\n* Examples (to easy locally dev VF#): see `examples/dotnetcoresdk/simple/README.md`\r\n\r\n### The `Microsoft.FSharp.Compiler.Sdk.netcore` package \r\n\r\nThe `Package` target creates a new package `Microsoft.FSharp.Compiler.Sdk.netcore` with\r\n\r\n- the correct layout to be consumed by .fsproj in .net core sdk using props files who specify fsc/fsharp.build path. \r\n  these property are used by FscTask inside `FSharp.NET.Sdk` package.\r\n- the fsc inside `build/netcoreapp1.0` is a normal console app published as [framework dependent deployment](https://docs.microsoft.com/en-us/dotnet/articles/core/deploying/).\r\n  That mean can be run with `dotnet build/netcoreapp1.0/fsc.dll` like all normal .net core app published with `dotnet publish`.\r\n- this package is tested End to End in isolation in target `TestE2E`\r\n\r\n## How `fsc` is built\r\n\r\nThe `src/fsharp/Fsc/Fsc.netcore.fsproj` is added.  It's a normal console app fsproj for .NET Core Sdk 1.0.0-preview4.\r\n\r\n- `build coreclr` (because use the fsharp.core/fsharp.compiler assemblies already built)\r\n- `cd src/fsharp/Fsc`\r\n- `dotnet restore Fsc.netcore.fsproj --configfile Fsc.netcore.NuGet.Config` (needed until the first package is on nuget.org)\r\n- `dotnet publish Fsc.netcore.fsproj -c Release`\r\n\r\nNOTE: At the moment the generated fsc.dll doesnt work because is not signed correctly.  To workaround this issue for now, the already built `fsc.exe` is copied as `fsc.dll` (so the `dotnet build` output is overwritten) Running `dotnet msbuild build-using-netcore-sdk.proj /p:Configuration=Release /t:PublishFsc` will generate the fixed published fsc. This will be fixed in a later PR, see TODO\r\n\r\n## Example .NET Core sdk preview4 fsproj\r\n\r\nThis PR adds an example of a .NET Core Sdk fsproj, see `examples/dotnetcoresdk/simple/README.md` This can compile using both the `dotnet path/to/fsc.ll` (netcore) or `fsc-proto.exe` (.net full), \r\njust set the `FscToolPath`/`Exe` as usual.\r\n\r\nThe `Microsoft.FSharp.Compiler.Sdk.netcore` can be overriden by adding as `PackageReference`, like in E2E tests.\r\n\r\n## Install script\r\n\r\nAll the steps are scripted using msbuild (run `dotnet msbuild build-using-netcore-sdk.proj`, see same file for help).    Why?\r\n\r\n- See README `./scripts/use_dotnet_core_sdk.README.md` for doc\r\n- This is easy to use. Just run `.\\scripts\\use_dotnet_core_sdk.bat` (or `.ps1`). After that the right sdk is in `PATH`, so just `dotnet --version`\r\n- This is fast. The .net core sdk is downloaded/install **only if needed**. If is already in `PATH` (like in machine wide install) that installation is used\r\n- This gives reproducible builds. the version is **always** checked, if doesnt match the one in `DotnetCoreSdkVersion.txt` it's downlaoded and installed \r\n  in `.dotnetsdk` directory and added to PATH (like current install script)\r\n- The sdk used by scripts/test is always the one in PATH (because is not in a fixed path) and  easy to check with `dotnet --version`\r\n- This is maintenable. Download and use the official install script from http://github.com/dotnet/cli repo, when needed\r\n\r\n## Build scripts\r\n\r\nThe `./build-using-netcore-sdk.proj` and in `./build/*`.\r\n\r\n- Based on a simplified structure of the build script used by http://github.com/dotnet/cli\r\n- Can be run with msbuild 15, so `dotnet msbuild`\r\n- All phases are distinct, and new packages can be easy added.\r\n- Docs is at end of `./build-using-netcore-sdk.proj` and all targets are commented about usage\r\n- Should be ok for xplat\r\n\r\n## How is tested\r\n\r\nE2E test in isolation, based on fsharp suite.\r\n\r\n- The tests (`TestE2E` target) are like the one in fsharp suite (reused lot of code of fsharp suite, same dsl).\r\n- All tests methods can be run indivisually (like fsharp suite, so run fsi is an option), no xunit/nunit magic (/cc @dsyme).\r\n- Atm suite runs with `dotnet run` so is just a normal console app.\r\n- Use xunit (nunit doesnt work yet with `dotnet vstest`) but not any features of xunit, it's only based on the \r\n  official xunit test template (`dotnet new -t xunittest -l fsharp`)\r\n- the test use the built package to restore/build/run/pack in isolation (doesnt mess with cache) a console and a lib app.\r\n\r\n\r\n## TODO not in this pr (next one)\r\n\r\n- Fix signing issue for fsc, so the fsc.netcore.fsproj will works ok with just `dotnet publish`\r\n- add converted `.\\scripts\\use_dotnet_core_sdk.ps1` as shell script `.\\scripts\\use_dotnet_core_sdk.bat.sh`\r\n- run the ./build-using-netcore-sdk.proj xplat\r\n- run tests with `dotnet test` (instead of `dotnet run`), need configuration of logging (trx) for cI, localdev\r\n- the `examples/dotnetcoresdk/simple/Simple.sln` doesnt reference the fsproj\r\n- minimize duplication of code (msbuild .props define), this PR prioritize parallel code instead of replace existing in one shot (safer)\r\n- fix generation of assemblyfile.fs who contains assemblyinfoattributes\r\n\r\n/cc people for review \r\n\r\n* @dsyme the added test suite is like fsharp suite, no nunit magic, and can be fixed easy to be run by fsi\r\n\r\n* @KevinRansom that's using only released .net core sdk and packages. Add ihmo a good base to use fully the .net core sdk for build. i didnt changed/messed with internal build tools/script, so no regression. The msbuild is inside .net core sdk (dotnet msbuild), like nuget (dotnet nuget), so is using normal tools/commands provided by sdk (and used by normal devs already). workflow is the same basic of developing a .net core app with sdk, i excluded advanced like crossgen (multi framework), pack (packages are generaeted from a prepared  directory, easy to check). I think we can use some part of sdk (like publish/pack) to make it easier to create normal netcore apps (fsc) or lib (fsharp.core), without changing a lot the build script/projects.\r\n\r\n* @cartermp @brettfo this add dogfooding and enable lots of smaller steps, see TODO\r\n\r\n* @vasily-kirichenko @cloudRoutine @dungpa i hope `./examples` and the easy install script will help to add support for .net core proj to vf# extension (i added you all because you develop the extension). i think lots can be done using https://github.com/dotnet/roslyn-project-system as base (easier maybe than chaning existing) and defining a fsharp capability for languageservice (maybe this require discussion in a new issue)\r\n\r\n* @forki i hope the installs script, build script are ok for contributors (feedback wanted about that), i tried to maximize (in the limits of msbuild) the readability and developer workflow (phases, simple targets, and normal project can be run directly from cli using normal `dotnet commands`, the same run by script)\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/Microsoft/visualfsharp/issues/comments/276718417","html_url":"https://github.com/Microsoft/visualfsharp/pull/2250#issuecomment-276718417","issue_url":"https://api.github.com/repos/Microsoft/visualfsharp/issues/2250","id":276718417,"user":{"login":"enricosada","id":147243,"avatar_url":"https://avatars.githubusercontent.com/u/147243?v=3","gravatar_id":"","url":"https://api.github.com/users/enricosada","html_url":"https://github.com/enricosada","followers_url":"https://api.github.com/users/enricosada/followers","following_url":"https://api.github.com/users/enricosada/following{/other_user}","gists_url":"https://api.github.com/users/enricosada/gists{/gist_id}","starred_url":"https://api.github.com/users/enricosada/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/enricosada/subscriptions","organizations_url":"https://api.github.com/users/enricosada/orgs","repos_url":"https://api.github.com/users/enricosada/repos","events_url":"https://api.github.com/users/enricosada/events{/privacy}","received_events_url":"https://api.github.com/users/enricosada/received_events","type":"User","site_admin":false},"created_at":"2017-02-01T17:12:42Z","updated_at":"2017-02-01T17:12:42Z","body":"> @enricosada I think replacing the \"build coreclr\" path (rather than adding a new one) would be better - why would we keep the old one active in master?\r\n\r\nNo reason at all, is just another pr (i can do that) because i fear big one shot changes.\r\nI can replace `build coreclr` completly adding new fsproj in parallel, and use .net core to build it. That remove all complexity for coreclr from fsproj/build script (see below).\r\nBuild artifacts (fsc.dll, fsharp.core.dll, etc) can be placed in expected directories, so rest of build.cmd doesnt change.\r\nthat was my idea, just splitted in two PR (this is the first one)\r\n\r\n> @enricosada I suspect very few of the people have actually use Preview 4 and the new .csproj/msbuild-based .NET Core SDK. I had zero real context on all of this until I read this PR.\r\n\r\nThe csproj VS tooling is out of preview in VS2017 RC3. the sdk itself is really near rtm.\r\nI'll probably use that at work too when VS2017 goes RTM, to replace our current aspnetcore with project.json and VS2015 btw (and we use it with .NET FW/iis, not .net core).\r\n\r\nFor c# guys is better, and they are gonna use that, because msbuild format is easier than project.json (less alien). Also VS2017 already support it, so is not difficult to sell at all.\r\n\r\n> @enricosada Could you publish a prelim version of Microsoft.FSharp.Compiler.Sdk.netcore yourself? Or is the signing issue to problematic?\r\n\r\nI can do that, that's what the build script does btw. I just need to change the name, but is not a problem. can be put inside FSharp.Compiler.Tools too (so it support netcore and net40).\r\nI can generate the package with script, and overwrite the assemblies with the signed one generated by `build coreclr` . result is [here in dev feed](https://www.myget.org/F/netcorecli-fsc-preview4-fsctask/api/v2/package/Microsoft.FSharp.Compiler.Sdk.netcore/1.0.0-rc-170113), i already use that to boostrap things (because easier). \r\n\r\nWorking example using dev feed are inside this PR\r\n\r\n```\r\ncd examples\\dotnetcoresdk\\simple\\console\r\ndotnet restore\r\ndotnet run\r\n```\r\n\r\n> @enricosada The duplication of build/install scripts is pretty disturbing.\r\n> Perhaps listing all the stuff that will eventually be deleted would be helpful.\r\n\r\nThere are two parts, build script orchestration and project files. Fsc.fsproj is a proj, but build require nuget restore/packaging/signing/runtests. That can be offloaded to .net core sdk. Some deletion will\r\ngreatly simplify files, but not remove it completly\r\n\r\nI cannot migrate project files until VS support new fsproj, but we can simplify greatly current script. It's ortogonal to my pr (needed to improve sdk), can be done anyway (just give me a go).\r\n\r\n*I assume i will build the .net core version of components with new fsproj and sdk*, so we remove current complexities in old fsproj.\r\n*I assume build orchestration goes in a msbuild .proj, using the sdk featuers* (like my pr)\r\n*I assume all stuff like restore/packaging/runtests will be executed by .net core sdk*\r\n\r\nTo delete:\r\n\r\n- msbuild location code -> dotnet msbuild\r\n- .nuget directory => dotnet nuget\r\n- Tools directory (minus specific stuff like localitazion target) because is pratically a custom packaged version of the sdk. And not used because new fsproj build netcore. Signing is already support by sdk.\r\n- init-tools and install script, useless, because are a custom version of the official .net core install script (same features)\r\n- all project.json\r\n- all netcore related stuff\r\n- tests runner from build script complexity => dotnet run or dotnet test\r\n\r\nit's possibile to `dotnet build` the old fsproj (will use normal targets), just use the new msbuild.exe. That remove lot of complexity in build script. and make easier to run xplat\r\n\r\nOld fsproj in parallel to new fsproj. used for .net core. Common properties like defines can be shared using specific .props file (like my pr)\r\nHaving both side by side, the new fsproj can also include build of net40 too, so when vs support it, we can just remove the old one.\r\n\r\nNote, the .NET Core SDK can build ALL framework, so .NET Framework, .NET Core, Portalble profiles, etc in same fsproj. **is not only for netcore**. \r\nAnd `dotnet build` -> `dotnet msbuild /t:Build`, no magic\r\n\r\n\r\n- where is msbuild? `dotnet msbuild`\r\n- where is nuget? `dotnet nuget`\r\n- restore all needed packages? `dotnet restore --packages /path/to/save/packages` (aftet that the fsproj know that path, no need to manually fix reference)\r\n- build fsc as net40? `dotnet build src\\fsharp\\fsc\\fsc.fsproj -f net40 -c Release`\r\n- generate the fsharp core package? `dotnet pack src\\fsharp\\fsharp.core\\fsharp.core.fsprj -c Release /p:Version=4.1.1.0` (the package metadata are property inside fsproj, or extenal props file, or from cmdargs)\r\n\r\nlike before, it's msbuild so following works:\r\n\r\n- build fsc as using proto? `dotnet build src\\fsharp\\fsc\\fsc.fsproj -f netcoreapp1.0 -c Release /p:FscPath=protodir/fsc.exe`\r\n- run cambride test? `dotnet run tests\\fsharp\\FSharp.Tests.FSharpSuite.fsproj` (or `dotnet test` for nunit like experience, with logs/filter/etc). Will continue to work from fsi obv.\r\n\r\nThe same build workflow can be used by developers too\r\n\r\n- `cd src/fsharp/Fsc`\r\n- `dotnet restore`\r\n- `dotnet run -f net40` //build all deps, run fsc for net40\r\n- `dotnet publish -f netcoreapp1.0` //build all deps, save in a dir an xcopy deployable fsc for .netcore\r\n- `dotnet publish -f net40` //build all deps, save in a dir an xcopy deployable fsc for .net 40\r\n\r\n\r\n"}}
{"actor":{"display_login":"enricosada"}}