Skip to content

Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation #559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jan 28, 2022
Merged

Pass 1 of SilkTouch Scraper - Joint Infrastructure and XML Generation #559

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
7edcc35
Starting work on SilkTouch, no Scraper-specific work yet
Perksey Jul 31, 2021
117d4a2
Start work on the subagent
Perksey Aug 2, 2021
4e26e3d
Spent far too long working on a Windows SDK/MSVS resolver and not tested
Perksey Aug 3, 2021
8e07317
Scraper (up to XML gen) nearly ready&joint CLI/roslyn iface nearly ready
Perksey Aug 4, 2021
cac8089
Fine Kai
Perksey Aug 5, 2021
f40802d
Lots of changes, some review fixes, very almost generating XML now.
Perksey Aug 7, 2021
b0e80ab
More fixes
Perksey Aug 7, 2021
8b20fbf
XML is being generated, ClangSharp bugs to fix uncovered.
Perksey Aug 7, 2021
4dbdff5
Fully XML documented, markdown pending
Perksey Aug 9, 2021
7b22e99
Contributor documentation
Perksey Aug 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "src/submodules/Vulkan-Headers"]
path = src/submodules/Vulkan-Headers
url = https://github.com/KhronosGroup/Vulkan-Headers.git
142 changes: 142 additions & 0 deletions Silk.NET.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,15 @@ ProjectSection(SolutionItems) = preProject
EndProjectSection
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "bindings", "bindings", "{9020C7C6-C366-4BD3-8C8A-F81394EC7174}"
ProjectSection(SolutionItems) = preProject
src\bindings\silktouch.json = src\bindings\silktouch.json
src\bindings\Directory.Build.targets = src\bindings\Directory.Build.targets
EndProjectSection
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "generators", "generators", "{8238D9F3-E158-4633-8017-B29AA3AD61F7}"
ProjectSection(SolutionItems) = preProject
src\generators\README.md = src\generators\README.md
EndProjectSection
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.Assimp.Native", "src\infrastructure\Silk.NET.Assimp.Native\Silk.NET.Assimp.Native.csproj", "{1FBF178D-3F9B-46C5-9D50-21D4D547113E}"
EndProject
Expand All @@ -46,6 +53,24 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.OpenAL.Soft.Native
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.Maths.Benchmarks", "src\benchmarks\Silk.NET.Maths.Benchmarks\Silk.NET.Maths.Benchmarks.csproj", "{CB8B28DE-456A-4B8E-85A6-2C50CEE08CA2}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.SilkTouch.Scraper", "src\generators\Silk.NET.SilkTouch.Scraper\Silk.NET.SilkTouch.Scraper.csproj", "{EA623F04-DADA-4714-B2C5-44C82E211492}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.SilkTouch.Common", "src\generators\Silk.NET.SilkTouch.Common\Silk.NET.SilkTouch.Common.csproj", "{C6833D95-14A8-499A-85E4-C09FA094DCE8}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SilkTouch", "src\generators\SilkTouch\SilkTouch.csproj", "{7DF67686-3F46-4569-935F-4A9E9903B575}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.SilkTouch.Roslyn", "src\generators\Silk.NET.SilkTouch.Roslyn\Silk.NET.SilkTouch.Roslyn.csproj", "{11841F56-7603-40D9-BC06-10EBBE17D905}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.SilkTouch.ClangSharp.Xml", "src\generators\Silk.NET.SilkTouch.ClangSharp.Xml\Silk.NET.SilkTouch.ClangSharp.Xml.csproj", "{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.SilkTouch.Emitter", "src\generators\Silk.NET.SilkTouch.Emitter\Silk.NET.SilkTouch.Emitter.csproj", "{50F26B27-32B6-4D66-ADD5-CC9C38373B19}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.SilkTouch.Overloader", "src\generators\Silk.NET.SilkTouch.Overloader\Silk.NET.SilkTouch.Overloader.csproj", "{46A89F89-77B3-4679-A82F-04A552545B16}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.Core", "src\libraries\Silk.NET.Core\Silk.NET.Core.csproj", "{69CF4437-59F7-4304-9AE1-4B58E1A93367}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Silk.NET.Vulkan", "src\bindings\Silk.NET.Vulkan\Silk.NET.Vulkan.csproj", "{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -128,6 +153,114 @@ Global
{CB8B28DE-456A-4B8E-85A6-2C50CEE08CA2}.Release|x64.Build.0 = Release|Any CPU
{CB8B28DE-456A-4B8E-85A6-2C50CEE08CA2}.Release|x86.ActiveCfg = Release|Any CPU
{CB8B28DE-456A-4B8E-85A6-2C50CEE08CA2}.Release|x86.Build.0 = Release|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Debug|Any CPU.Build.0 = Debug|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Debug|x64.ActiveCfg = Debug|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Debug|x64.Build.0 = Debug|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Debug|x86.ActiveCfg = Debug|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Debug|x86.Build.0 = Debug|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Release|Any CPU.ActiveCfg = Release|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Release|Any CPU.Build.0 = Release|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Release|x64.ActiveCfg = Release|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Release|x64.Build.0 = Release|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Release|x86.ActiveCfg = Release|Any CPU
{EA623F04-DADA-4714-B2C5-44C82E211492}.Release|x86.Build.0 = Release|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Debug|x64.ActiveCfg = Debug|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Debug|x64.Build.0 = Debug|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Debug|x86.ActiveCfg = Debug|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Debug|x86.Build.0 = Debug|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Release|Any CPU.Build.0 = Release|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Release|x64.ActiveCfg = Release|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Release|x64.Build.0 = Release|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Release|x86.ActiveCfg = Release|Any CPU
{C6833D95-14A8-499A-85E4-C09FA094DCE8}.Release|x86.Build.0 = Release|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Debug|Any CPU.Build.0 = Debug|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Debug|x64.ActiveCfg = Debug|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Debug|x64.Build.0 = Debug|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Debug|x86.ActiveCfg = Debug|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Debug|x86.Build.0 = Debug|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Release|Any CPU.ActiveCfg = Release|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Release|Any CPU.Build.0 = Release|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Release|x64.ActiveCfg = Release|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Release|x64.Build.0 = Release|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Release|x86.ActiveCfg = Release|Any CPU
{7DF67686-3F46-4569-935F-4A9E9903B575}.Release|x86.Build.0 = Release|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Debug|Any CPU.Build.0 = Debug|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Debug|x64.ActiveCfg = Debug|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Debug|x64.Build.0 = Debug|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Debug|x86.ActiveCfg = Debug|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Debug|x86.Build.0 = Debug|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Release|Any CPU.ActiveCfg = Release|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Release|Any CPU.Build.0 = Release|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Release|x64.ActiveCfg = Release|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Release|x64.Build.0 = Release|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Release|x86.ActiveCfg = Release|Any CPU
{11841F56-7603-40D9-BC06-10EBBE17D905}.Release|x86.Build.0 = Release|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Debug|Any CPU.Build.0 = Debug|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Debug|x64.ActiveCfg = Debug|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Debug|x64.Build.0 = Debug|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Debug|x86.ActiveCfg = Debug|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Debug|x86.Build.0 = Debug|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Release|Any CPU.ActiveCfg = Release|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Release|Any CPU.Build.0 = Release|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Release|x64.ActiveCfg = Release|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Release|x64.Build.0 = Release|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Release|x86.ActiveCfg = Release|Any CPU
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793}.Release|x86.Build.0 = Release|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Debug|Any CPU.Build.0 = Debug|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Debug|x64.ActiveCfg = Debug|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Debug|x64.Build.0 = Debug|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Debug|x86.ActiveCfg = Debug|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Debug|x86.Build.0 = Debug|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Release|Any CPU.ActiveCfg = Release|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Release|Any CPU.Build.0 = Release|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Release|x64.ActiveCfg = Release|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Release|x64.Build.0 = Release|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Release|x86.ActiveCfg = Release|Any CPU
{50F26B27-32B6-4D66-ADD5-CC9C38373B19}.Release|x86.Build.0 = Release|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Debug|Any CPU.Build.0 = Debug|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Debug|x64.ActiveCfg = Debug|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Debug|x64.Build.0 = Debug|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Debug|x86.ActiveCfg = Debug|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Debug|x86.Build.0 = Debug|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Release|Any CPU.ActiveCfg = Release|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Release|Any CPU.Build.0 = Release|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Release|x64.ActiveCfg = Release|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Release|x64.Build.0 = Release|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Release|x86.ActiveCfg = Release|Any CPU
{46A89F89-77B3-4679-A82F-04A552545B16}.Release|x86.Build.0 = Release|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Debug|Any CPU.Build.0 = Debug|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Debug|x64.ActiveCfg = Debug|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Debug|x64.Build.0 = Debug|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Debug|x86.ActiveCfg = Debug|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Debug|x86.Build.0 = Debug|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Release|Any CPU.ActiveCfg = Release|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Release|Any CPU.Build.0 = Release|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Release|x64.ActiveCfg = Release|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Release|x64.Build.0 = Release|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Release|x86.ActiveCfg = Release|Any CPU
{69CF4437-59F7-4304-9AE1-4B58E1A93367}.Release|x86.Build.0 = Release|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Debug|Any CPU.Build.0 = Debug|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Debug|x64.ActiveCfg = Debug|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Debug|x64.Build.0 = Debug|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Debug|x86.ActiveCfg = Debug|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Debug|x86.Build.0 = Debug|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Release|Any CPU.ActiveCfg = Release|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Release|Any CPU.Build.0 = Release|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Release|x64.ActiveCfg = Release|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Release|x64.Build.0 = Release|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Release|x86.ActiveCfg = Release|Any CPU
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand All @@ -146,6 +279,15 @@ Global
{44CA62B1-EE9A-41BC-A23E-D0A2F3791D76} = {F07CABFC-DC6A-4B5B-BC56-B10EEC2C0BFA}
{DB186ABF-C51D-432F-B9F3-A13B53B1BD07} = {F07CABFC-DC6A-4B5B-BC56-B10EEC2C0BFA}
{CB8B28DE-456A-4B8E-85A6-2C50CEE08CA2} = {FD15E196-1C63-47D6-8AD5-64F015120B4B}
{EA623F04-DADA-4714-B2C5-44C82E211492} = {8238D9F3-E158-4633-8017-B29AA3AD61F7}
{C6833D95-14A8-499A-85E4-C09FA094DCE8} = {8238D9F3-E158-4633-8017-B29AA3AD61F7}
{7DF67686-3F46-4569-935F-4A9E9903B575} = {8238D9F3-E158-4633-8017-B29AA3AD61F7}
{11841F56-7603-40D9-BC06-10EBBE17D905} = {8238D9F3-E158-4633-8017-B29AA3AD61F7}
{CD47A2B9-D3D6-40CF-AE0A-B571EF9F7793} = {8238D9F3-E158-4633-8017-B29AA3AD61F7}
{50F26B27-32B6-4D66-ADD5-CC9C38373B19} = {8238D9F3-E158-4633-8017-B29AA3AD61F7}
{46A89F89-77B3-4679-A82F-04A552545B16} = {8238D9F3-E158-4633-8017-B29AA3AD61F7}
{69CF4437-59F7-4304-9AE1-4B58E1A93367} = {C9718C94-2F21-4E8D-B55D-8F0B1A131346}
{AF6E05E0-9C51-4D52-AC7E-056714CAC5F4} = {9020C7C6-C366-4BD3-8C8A-F81394EC7174}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {F5273D7F-3334-48DF-94E3-41AE6816CD4D}
Expand Down
13 changes: 13 additions & 0 deletions documentation/for-contributors/generators/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# SilkTouch

SilkTouch is the Silk.NET project's C# native interoperability code generation stack. It contains useful generators which together generate the majority of the library's codebase.

It does this by splitting the generation process into multiple phases, all anchored around the concept of a "shared infrastructure" - abstractions for SilkTouch generators which are agnostic of where they're running. Currently the only places SilkTouch can run is in a Roslyn Source Generator and using a `dotnet tool` Command Line Interface, but the idea is so long as no incompatibilities are introduced by each individual phase (such as being locked to .NET 6 instead of .NET Standard 2.0) the generator should be able to run in whatever form factor is desirable - the Shared Infrastructure will provide everything a generator could need for consuming a JSON configuration with C# input code and producing additional generated C# code as a result.

Learn more about each individual cornerstone of the SilkTouch Stack:
- [Shared Infrastructure](shared-infrastructure.md)
- [Emitter](emitter.md)
- [Overloader](overloader.md)
- [Scraper](scraper.md)

TODO: Add a link to the original proposal once the proposals folder has been reorganised according to the Software Development Plan.
3 changes: 3 additions & 0 deletions documentation/for-contributors/generators/emitter.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Emitter

The Emitter.
3 changes: 3 additions & 0 deletions documentation/for-contributors/generators/overloader.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Overloader

The Overloader.
39 changes: 39 additions & 0 deletions documentation/for-contributors/generators/scraper.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Scraper

The SilkTouch Scraper is responsible for walking through the abstract syntax tree (AST) of a C/C++ header and all the referenced headers therein to generate C# bindings to as much of this as possible where the configuration indicates this is desirable. Only certain headers will have bindings generated for them, which can be specified by providing the paths to the headers to "traverse" in the JSON configuration.

## ClangSharp

To walk through the C/C++ AST, ClangSharp's P/Invoke Generator is used, which is a proven tool for generating C# bindings to C/C++ APIs published by Microsoft. It does this by using the clang compiler to walk through the AST.

ClangSharp's P/Invoke Generator is capable of producing C# bindings on its own, however these are raw bindings that use DllImport and don't aim to lift the bindings to be more C#-friendly. However, as part of Scraper development work was done to add an "XML output mode" to ClangSharp which instead of producing pure C# code, it outputs XML which roughly represents what the C# code would've looked like.

## The Subagent

ClangSharp uses libclang, which has funny rules over multithreading. To maintain a degree of parallelism during generation, the invocation of ClangSharp was moved to be done in a separate process. The original intention was to invoke the `ClangSharpPInvokeGenerator` tool as shipped publicly, but unfortunately `dotnet tool`s cannot be referenced by other libraries or `dotnet tool`s. As such, we created our own shim which mimics the behaviour of and duplicates a lot of code from the `ClangSharpPInvokeGenerator` tool, but within the same CLI executable.

### The `ISubagent` abstraction

`ISubagent` is responsible for launching the ClangSharp tool with certain parameters, inputs, and configuration. This is an abstraction such that the main Scraper assembly remains easily portable to another form factor should we enable the Scraper to run in another form factor in the future, as well as to prevent the library assembly knowing too much about the assembly in which its executing.

The only implementation of this interface is the `ClangSharpSubagent` class in the `SilkTouch` assembly (CLI implementation of the Shared Infrastructure) which gathers information on the currently executing assembly, and relaunches it with `clangsharp` as the first argument and a JSON-encoded record as the second.

### The `silktouch clangsharp` command

This is the command that invokes ClangSharp as described in the previous section. This deserializes the JSON-encoded record and passes the data held therein to the `PInvokeGenerator` class within ClangSharp. This will generate the XML dump and then quit, possibly with some ClangSharp-generated diagnostics and a non-zero error code if those diagnostics are severe. The implementation of this is in the `ClangSharpHandoff` class.

## XML Mods

Once an initial XML dump has been generated by The Subagent, the XML dump will be parsed by the `Silk.NET.SilkTouch.ClangSharp.Xml` library and produce a tree of records. This is done in the `ScraperGenerator` ahead of the XML mods stage.

The purpose of the XML mods stage is to walk this tree of records, and modify it depending on the project configuration (for example changing all names to be PascalCase, or adding custom attributes to cause the Overloader to generate certain overloads). Each mod must be specifically enabled, and has free range over the XML. As such, they execute in the order in which they were specified in the project configuration.

Each XML mod has access to the `ScraperJobConfiguration.ModOptions` and the full XML tree. It does not have any access to resources beyond that, such as the SilkTouch Context.

TODO: Write words here about each XML mod once XML mods have been implemented

## Transformation

After the XML mods have been applied, the transformation stage is responsible for converting the bindings represented by the XML tree back into C# code that can be output via the SilkTouch Context (and added to the project compilation). Unlike ClangSharp, this stage will output bindings which are compatible with/invoke the Emitter and Overloader instead of using DllImport directly.

TODO: Write more words here once this stage is implemented.
Loading