Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation #559

Perksey · 2021-08-05T17:33:44Z

Very very wip

In-scope for this PR

Barebones joint Roslyn/CLI infrastructure is working
- Documented with for-contributors markdown
XML generation is working
- Documented with for-contributors markdown

Out-of-scope for this PR

XML parsing is working
- Documented with for-contributors markdown
XML mods are working
- Documented with for-contributors markdown
XML transformation to C# code is working
- Documented with for-contributors markdown
User documentation?

src/Directory.Build.props

src/bindings/Directory.Build.targets

HurricanKai · 2021-08-05T17:45:54Z

src/bindings/Directory.Build.targets

@@ -0,0 +1,6 @@
+<Project>
+    <ItemGroup>
+        <AdditionalFiles Include="$(MSBuildThisFileDirectory)silktouch.json" />


That seems weird, why do all bindings share this file?

Changing, dunno why.

Changed to use one file per project with an optional "global file" for common config i.e. license header.

src/bindings/Silk.NET.Vulkan/Vk.cs

HurricanKai · 2021-08-05T17:46:47Z

src/bindings/silktouch.json

+    "projects": {
+        "Silk.NET.Vulkan": {
+            "scraper": {
+
+            }
+        }
+    }


have you considered, instead of doing this, doing the configuration in the actual project? like... this seems pointless.

Changing to be per-project

Changed to use one file per project with an optional "global file" for common config i.e. license header.

src/generators/Silk.NET.SilkTouch.ClangSharp.Xml/Silk.NET.SilkTouch.ClangSharp.Xml.csproj

HurricanKai · 2021-08-05T17:49:14Z

src/generators/Silk.NET.SilkTouch.Common/Configuration/Config.cs

+
+namespace Silk.NET.SilkTouch.Configuration
+{
+    public static class Config


please no
Microsoft.Extensions.Configuration > static class Config

I don't think that package is applicable here. This is just a simple JSON configuration, we don't have massive infrastructure, extensibility concerns, or anything else that could possibly warrant a dedicated configuration package. I'm all for using smart libraries, but only when we actually need to do smart things. I'm not going to force packages into the codebase where they're not needed.

If you read the code for this particular class, these are just helpers and wrappers for loading configuration to reduce verbosity elsewhere in the codebase. It holds no state whatsoever.

The Configuration package is tiny and would allow us to use different configuration sources, use configuration sections, etc.
It would allow us to separate the configuration of the different stages entirely, which I think is very desirable.

The Configuration package in particular doesn't really make much sense outside of the hosting infrastructure or any other scenario where we might abstractions or injection. Moreover, it makes it far less obvious what's actually going on during configuration loading.

We really don't need this at all.

No change deemed necessary. If this is a requirement please close this PR.

HurricanKai · 2021-08-05T17:50:36Z

src/generators/Silk.NET.SilkTouch.Common/Configuration/Json/ExcludesJsonConverter.cs

+                // the performance is horrible but it's not too much a cause for concern as it is a "fixed cost".
+                // definitely improve it in the future though as we do have access to new string(char*), meaning we can
+                // come up with a Span-based solution it'll just be very very very verbose.
+                if (kvp[0].ToLower().Trim() == "hint" && Enum.TryParse


Equals + IgnoreCase?

Changed to Equals + IgnoreCase.

HurricanKai · 2021-08-05T17:56:03Z

src/generators/Silk.NET.SilkTouch.Scraper/ScraperGenerationExtensions.cs

+{
+    public static class ScraperGenerationExtensions
+    {
+        public static async ValueTask<bool> RunScraperAsync<T>(this SilkTouchGenerator generator)


This doesn't feel very ValueTask to me?

This has been changed to Task.

HurricanKai · 2021-08-05T18:00:43Z

src/generators/Silk.NET.SilkTouch.Common/Generation/SilkTouchContext.cs

+    public sealed record SilkTouchContext
+    (
+        string AssemblyName,
+        IEnumerable<CSharpSyntaxTree> SyntaxTrees,


I need a compilation. Without a Compilation Overloads & Emitter probably won't work

What specifically do you need from the compilation though?

like the semantic model and stuff?

Ok fair enough.

Now passing the Compilation to SilkTouchContext.

Perksey · 2021-08-10T18:43:15Z

This is as far as this PR will progress. It is 100% XML documented and has contributor documentation describing the general implementation design. This is ready for review.

Perksey

Apparently I had review comments pending so forgive me if these are outdated and invalid, I can't be bothered to read through them lol.

src/Directory.Build.props

Perksey · 2021-08-07T16:27:51Z

src/bindings/Directory.Build.targets

@@ -0,0 +1,6 @@
+<Project>
+    <ItemGroup>
+        <AdditionalFiles Include="$(MSBuildThisFileDirectory)silktouch.json" />


Changed to use one file per project with an optional "global file" for common config i.e. license header.

src/bindings/Directory.Build.targets

src/bindings/Silk.NET.Vulkan/Vk.cs

Perksey · 2021-08-07T16:28:45Z

src/bindings/silktouch.json

+    "projects": {
+        "Silk.NET.Vulkan": {
+            "scraper": {
+
+            }
+        }
+    }


Changed to use one file per project with an optional "global file" for common config i.e. license header.

src/generators/Silk.NET.SilkTouch.ClangSharp.Xml/Silk.NET.SilkTouch.ClangSharp.Xml.csproj

Perksey · 2021-08-07T16:29:25Z

src/generators/Silk.NET.SilkTouch.Common/Configuration/Config.cs

+
+namespace Silk.NET.SilkTouch.Configuration
+{
+    public static class Config


No change deemed necessary. If this is a requirement please close this PR.

Perksey · 2021-08-07T16:30:28Z

src/generators/Silk.NET.SilkTouch.Common/Configuration/Json/ExcludesJsonConverter.cs

+                // the performance is horrible but it's not too much a cause for concern as it is a "fixed cost".
+                // definitely improve it in the future though as we do have access to new string(char*), meaning we can
+                // come up with a Span-based solution it'll just be very very very verbose.
+                if (kvp[0].ToLower().Trim() == "hint" && Enum.TryParse


Changed to Equals + IgnoreCase.

Perksey · 2021-08-07T16:30:54Z

src/generators/Silk.NET.SilkTouch.Common/Generation/SilkTouchContext.cs

+    public sealed record SilkTouchContext
+    (
+        string AssemblyName,
+        IEnumerable<CSharpSyntaxTree> SyntaxTrees,


Now passing the Compilation to SilkTouchContext.

Perksey · 2021-08-07T16:31:08Z

src/generators/Silk.NET.SilkTouch.Scraper/ScraperGenerationExtensions.cs

+{
+    public static class ScraperGenerationExtensions
+    {
+        public static async ValueTask<bool> RunScraperAsync<T>(this SilkTouchGenerator generator)


This has been changed to Task.

Perksey · 2021-08-27T14:52:06Z

This is ready for another review.

Perksey · 2021-11-05T11:42:46Z

TODO: the Visual Studio resolution logic was backported to 2.X in Build improvements & dynamic Windows SDK resolution (backport from 3.0) #661 but further modified for stability. These changes should be brought back forward to 3.0!

HurricanKai

Merging to continue editing

…#559)

Perksey added 5 commits August 1, 2021 00:16

Starting work on SilkTouch, no Scraper-specific work yet

7edcc35

Start work on the subagent

117d4a2

Spent far too long working on a Windows SDK/MSVS resolver and not tested

4e26e3d

Scraper (up to XML gen) nearly ready&joint CLI/roslyn iface nearly ready

8e07317

Fine Kai

cac8089

Perksey added area-SilkTouch tenet-Maintainability labels Aug 5, 2021

Perksey added this to the 3.0 milestone Aug 5, 2021

Perksey self-assigned this Aug 5, 2021

Perksey changed the title ~~SilkTouch Scraper~~ Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation Aug 5, 2021

HurricanKai reviewed Aug 5, 2021

View reviewed changes

Perksey added 5 commits August 7, 2021 17:34

Lots of changes, some review fixes, very almost generating XML now.

f40802d

More fixes

b0e80ab

XML is being generated, ClangSharp bugs to fix uncovered.

8b20fbf

Fully XML documented, markdown pending

4dbdff5

Contributor documentation

7b22e99

Perksey mentioned this pull request Aug 10, 2021

Document how to use 3.0's SilkTouch #570

Open

13 tasks

Perksey marked this pull request as ready for review August 10, 2021 18:43

Perksey commented Aug 10, 2021

View reviewed changes

Perksey requested a review from HurricanKai August 10, 2021 18:54

Perksey assigned HurricanKai and unassigned Perksey Aug 23, 2021

Perksey mentioned this pull request Oct 24, 2021

Build improvements & dynamic Windows SDK resolution (backport from 3.0) #661

Merged

HurricanKai approved these changes Jan 28, 2022

View reviewed changes

HurricanKai merged commit 3268e1b into develop/3.0 Jan 28, 2022

HurricanKai deleted the feature/3.0/silktouch-scraper branch January 28, 2022 17:46

Perksey added a commit that referenced this pull request Feb 20, 2022

Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation (…

255ede5

…#559)

HurricanKai mentioned this pull request Feb 12, 2023

Getting back on track #1288

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation #559

Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation #559

Perksey commented Aug 5, 2021 •

edited

Loading

HurricanKai Aug 5, 2021

Perksey Aug 6, 2021

Perksey Aug 7, 2021

HurricanKai Aug 5, 2021

Perksey Aug 6, 2021

Perksey Aug 7, 2021

HurricanKai Aug 5, 2021

Perksey Aug 6, 2021 •

edited

Loading

HurricanKai Aug 6, 2021

Perksey Aug 6, 2021

Perksey Aug 7, 2021

HurricanKai Aug 5, 2021

Perksey Aug 7, 2021

HurricanKai Aug 5, 2021

Perksey Aug 7, 2021

HurricanKai Aug 5, 2021

Perksey Aug 6, 2021

HurricanKai Aug 6, 2021

Perksey Aug 6, 2021

Perksey Aug 7, 2021

Perksey commented Aug 10, 2021

Perksey left a comment •

edited

Loading

Perksey Aug 7, 2021

Perksey Aug 7, 2021

Perksey Aug 7, 2021

Perksey Aug 7, 2021

Perksey Aug 7, 2021

Perksey Aug 7, 2021

Perksey commented Aug 27, 2021

Perksey commented Nov 5, 2021 •

edited

Loading

HurricanKai left a comment

Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation #559

Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation #559

Conversation

Perksey commented Aug 5, 2021 • edited Loading

In-scope for this PR

Out-of-scope for this PR

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Perksey Aug 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Perksey commented Aug 10, 2021

Perksey left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Perksey commented Aug 27, 2021

Perksey commented Nov 5, 2021 • edited Loading

HurricanKai left a comment

Choose a reason for hiding this comment

Perksey commented Aug 5, 2021 •

edited

Loading

Perksey Aug 6, 2021 •

edited

Loading

Perksey left a comment •

edited

Loading

Perksey commented Nov 5, 2021 •

edited

Loading