Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI ONLY] [NativeAOT] ObjWriter in C# #92705

Closed
wants to merge 120 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
120 commits
Select commit Hold shift + click to select a range
ab2e067
Move ObjectWriter and ObjectWritingOptions into its own namespace
filipnavara Oct 18, 2022
ebef200
Add external sources for Melanzana.MachO and LibObjectFile
filipnavara Oct 18, 2022
f1bccfd
Disable versioning NuGets in external projects (MinVer, Nerdbank.GitV…
filipnavara Oct 18, 2022
eae83de
Add first draft of MachO and ELF object writers
filipnavara Oct 10, 2022
04a4c34
Generate relocatable DWARF info in ELF, fix X64 register mapping
filipnavara Oct 19, 2022
a889ee7
Fix writing base address with relocations
filipnavara Oct 19, 2022
e1733e7
Emit information about static variables, minor DWARF fixes
filipnavara Oct 19, 2022
ce591f7
Fix generated section layout in symbol table for Mach-O after EmitSym…
filipnavara Oct 19, 2022
2f52a06
Add support for ARM64 ELF
filipnavara Oct 19, 2022
bd94741
Add ObjectWriter class and derive UnixObjectWriter from it
filipnavara Oct 20, 2022
e8e3ef3
Add initial take on CoffObjectWriter
filipnavara Oct 21, 2022
212d4c2
Emit control flow guard information
filipnavara Oct 21, 2022
8fec3ff
Add win-arm64 support
filipnavara Oct 23, 2022
0b1a508
Misc cleanups of the COFF object writer
filipnavara Oct 24, 2022
5e96cd2
Remove GetSection
filipnavara Oct 24, 2022
4fde393
Add COFF COMDAT support
filipnavara Oct 25, 2022
1f71dc5
Add support for writing extended COFF object files (> 65279 sections)
filipnavara Oct 25, 2022
88ce825
Wrap some logic in SectionWriter struct; clean up code
filipnavara Oct 25, 2022
c27cb58
Emit GNU-stack note in ELF
filipnavara Oct 25, 2022
dbee5e1
Propagate DWARF5 option
filipnavara Oct 25, 2022
101e8c0
Initial take on CodeView type section builder
filipnavara Oct 26, 2022
d1f7644
Move SectionWriter out of ObjectWriter so it can be used by other bui…
filipnavara Oct 27, 2022
51f143c
Switch DwarfEhFrame to use UnixObjectWriter
filipnavara Oct 27, 2022
8f7384b
WIP: CodeView debugging information
filipnavara Oct 27, 2022
5f034a1
WIP: CodeView line mapping
filipnavara Oct 28, 2022
1958033
WIP: CodeView restructuring
filipnavara Oct 29, 2022
718a936
CodeView cleanup
filipnavara Oct 31, 2022
711c987
CodeView: Handle continuation records in lists
filipnavara Nov 1, 2022
3a78e7a
Add comments and asserts
filipnavara Nov 1, 2022
6540fd5
Emit CodeView symbols correctly when COMDAT is used
filipnavara Nov 1, 2022
b5390a0
Minor cleanup in CodeView, add ARM64 register mapping
filipnavara Nov 2, 2022
43416ce
Optimize memory usage when writing section content
filipnavara Nov 2, 2022
cbe9608
CodeView: Fix emitting size of big structures
filipnavara Nov 3, 2022
a028b32
Fix too aggressive zeroing of buffer
filipnavara Nov 3, 2022
868b8e2
Make sure not to dereference null returned from _debugInfo.GetLocalVa…
filipnavara Nov 3, 2022
6a6bda8
HACK: Do not emit debug information where LLVM ObjWriter would not.
filipnavara Nov 3, 2022
c06c030
Avoid some unnecessary name mangling calls
filipnavara Nov 3, 2022
5fdb1c1
Cache mangled names
filipnavara Nov 3, 2022
822d513
Fix ArgumentOutOfRangeException when writing DWARF debug info
filipnavara Nov 4, 2022
b6b8c93
Fix build after rebase
filipnavara Sep 10, 2023
90e652c
Add support for SectionType.Uninitialized
filipnavara Sep 10, 2023
eb14514
Add support for iOS-like platforms
filipnavara Sep 10, 2023
d96c8e1
Invoke object dumper from shared ObjectWriter
filipnavara Sep 10, 2023
12ed8d0
macOS/iOS: Emit simple compact unwinding information
filipnavara Sep 10, 2023
039c87c
Add support for new TLS relocations in ELF
filipnavara Sep 11, 2023
78fd4ef
Generate debug info even if managed debug info not present
filipnavara Sep 11, 2023
716c282
Fix counting the DwarfLineProgramTable layout (backport of https://gi…
filipnavara Sep 11, 2023
fd65a27
Remove incorrect assert in MachObjectWriter (text section starts at v…
filipnavara Sep 11, 2023
afafc76
Temporarily disable optimization that results in incorrect LSDA point…
filipnavara Sep 11, 2023
9428098
Fix emitting unwind tables in Mach-O
filipnavara Sep 11, 2023
ab5f878
Emit the MH_SUBSECTIONS_VIA_SYMBOLS header flag for Mach-O
filipnavara Sep 21, 2023
44a9057
Fix seeking in ObjectWriterStream
filipnavara Sep 21, 2023
9048667
Avoid excessive seeks by implementing SliceStream.CopyTo
filipnavara Sep 21, 2023
cc1975e
Use ObjectWriterStream in ElfObjectWriter
filipnavara Sep 21, 2023
52d5584
Use ObjectWriterStream for Mach-O
filipnavara Sep 21, 2023
789cda4
Fix setting the UNWIND_HAS_LSDA flag in compact unwind table
filipnavara Sep 21, 2023
0ddefe2
COFF:
filipnavara Sep 26, 2023
afcd1e4
COFF: Fix off-by-one error when writing more than 0xffff relocations …
filipnavara Sep 26, 2023
e5cf2e3
Make ObjectWriterStream the only supported stream and fix padding emi…
filipnavara Sep 26, 2023
ee21394
Fix scope overlapping in NAOT DWARF info (backport of https://github.…
filipnavara Sep 27, 2023
efd5b92
ELF: Use the address of __managedcode in DWARF info (backport of http…
filipnavara Sep 27, 2023
1fd9f55
ELF: Fix emitting global symbols
filipnavara Sep 27, 2023
ab2be43
ELF: Fix TLS support on linux-x64 Release builds
filipnavara Sep 27, 2023
6de5920
Return sorted set from GetUndefinedSymbols to avoid extra sorting
filipnavara Sep 26, 2023
f5017bc
COFF: Fix generating COMDAT section references in big-obj files
filipnavara Sep 27, 2023
2f5c20c
Generate Mach-O for iOS-like platforms, not ELF
filipnavara Sep 27, 2023
1c0fc8d
Fix EH frames on linux-arm64
filipnavara Sep 28, 2023
1176432
COFF: Support addend for IMAGE_REL_BASED_ARM64_PAGEOFFSET_12A relocation
filipnavara Sep 30, 2023
dcaea22
COFF: Emit alignment for .pdata section
filipnavara Sep 30, 2023
553cce4
Fix padding emitted in ObjectWriterStream.AppendPadding
filipnavara Sep 30, 2023
b00650b
Fix ObjectWriterStream.Position setter (again)
filipnavara Sep 30, 2023
f713a2b
COFF: Emit .pdata/.xdata as associative COMDAT sections/symbols
filipnavara Sep 30, 2023
97435c0
ELF/LibObjectFile experiment: Add support for building optimized stri…
filipnavara Oct 2, 2023
c0ede64
LibObjectFile: Add support for >= 0xff00 sections
filipnavara Oct 4, 2023
b81765f
ELF: Add basic COMDAT support, don't emit empty relocation sections
filipnavara Oct 4, 2023
dfa9a82
ELF: Emit correct addresses and relocations in debug info, references…
filipnavara Oct 5, 2023
1dff572
Add missing StringComparer.Ordinal to dictionaries
filipnavara Oct 5, 2023
7821383
Remove incorrect assert
filipnavara Oct 5, 2023
632b638
WIP: Rewritten DWARF debug info emitter
filipnavara Oct 18, 2023
9124e97
Fix state change in ObjectWriterStream triggering an Assert
filipnavara Oct 20, 2023
a7aa171
WIP: Mach-O/DWARF
filipnavara Oct 20, 2023
e025342
DWARF Mach-O relocations done right
filipnavara Oct 20, 2023
5365b73
Do not output debug information for static variables that were trimmed
filipnavara Oct 22, 2023
bf697f7
WIP: Cleanup
filipnavara Oct 22, 2023
6cd80a2
Rework how symbol tables are generated, and how DWARF relocations are…
filipnavara Oct 23, 2023
1d921ca
Revert accidental change
filipnavara Oct 23, 2023
4d3f9d6
Update LibObjectFile to version 0.5.0.
filipnavara Oct 23, 2023
3c06121
Fix ELF section symbols and DW_AT_low_pc in DWARF debug info
filipnavara Oct 23, 2023
5b80655
Disable MinVer package in LibObjectFile
filipnavara Oct 23, 2023
6366906
ELF: Generate symbol table section index if there's more than 0xff00 …
filipnavara Oct 23, 2023
eb8c2af
Revert "Do not output debug information for static variables that wer…
filipnavara Oct 23, 2023
d8265ab
Fix typo
filipnavara Oct 23, 2023
9057747
Fix null dereference in CoffObjectWriter.CreateSection
filipnavara Oct 23, 2023
a303401
COFF: Add X86 support (tested on SmolSharp.HelloWorld)
filipnavara Oct 24, 2023
97a494f
Fix COMDAT symbol values
filipnavara Nov 12, 2023
f0306c8
Do not generate broken debug info for non-methods
filipnavara Nov 17, 2023
ad50dfb
Add generic StringTableBuilder and use it in COFF
filipnavara Nov 25, 2023
8bbdce3
Fix couple of bugs in last commit:
filipnavara Nov 26, 2023
9c883a4
COFF: Add support for '//BBBBBB' section names.
filipnavara Nov 26, 2023
93fb572
Add simple ELF writer
filipnavara Dec 6, 2023
1b08ff9
Port from main: Fix generating debug info for static fields
filipnavara Dec 6, 2023
819f2d0
Add (untested) x86 support to ElfObjectWriter
filipnavara Dec 7, 2023
6ef6e79
Cleanup
filipnavara Dec 7, 2023
094cdf0
Move Dwarf/CodeView files into their respective directories
filipnavara Dec 8, 2023
95a1fca
Make bulk of the ObjectWriter classes internal, improve logging
filipnavara Dec 8, 2023
10bb132
Use IBinaryInteger for WriteLittleEndian helper method
filipnavara Dec 8, 2023
28ee6ca
Add simple Mach-O object file writer that doesn't use Melanzana.MachO
filipnavara Dec 9, 2023
c219e63
Rename LlvmObjectWriter to LegacyObjectWriter to prevent conflict wit…
filipnavara Dec 9, 2023
3d53df5
Mach-O: Swap the order of symbol table and string table to make strip…
filipnavara Dec 9, 2023
fd274ed
Fix the CPU type/subtype
filipnavara Dec 10, 2023
9f69232
Fix Release build
filipnavara Dec 10, 2023
7213ce7
Eliminate relocations for known symbols within the same section
filipnavara Dec 11, 2023
0e9c899
Fix IMAGE_REL_BASED_ARM64_BRANCH26
filipnavara Dec 11, 2023
618e8c5
Attempt to fix CI errors
filipnavara Dec 11, 2023
558c2d6
Fix incorrect addends in COFF
filipnavara Dec 11, 2023
56ae35f
Fix ELF rela section flags; omit frame symbols for ELF
filipnavara Dec 11, 2023
ec4d5fa
Split the writing and reading part of section data
filipnavara Dec 11, 2023
b8be8b6
Cleanup
filipnavara Dec 11, 2023
e3499e3
Pattern matching consistency
filipnavara Dec 11, 2023
c7b4984
Comments, more cleanup
filipnavara Dec 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ public enum SectionType
Writeable,
Executable,
Uninitialized,
Debug,
}

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ public enum RelocType
// This is a special NGEN-specific relocation type
// for relative pointer (used to make NGen relocation
// section smaller)
IMAGE_REL_SECTION = 0x79, // 16 bit section index containing target

IMAGE_REL_BASED_ARM64_PAGEBASE_REL21 = 0x81, // ADRP
IMAGE_REL_BASED_ARM64_PAGEOFFSET_12A = 0x82, // ADD/ADDS (immediate) with zero shift, for page offset
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Buffers;
using System.Buffers.Binary;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Text;

using static ILCompiler.ObjectWriter.CodeViewNative;

namespace ILCompiler.ObjectWriter
{
internal sealed class CodeViewFileTableBuilder
{
private MemoryStream _stringTableWriter = new();
private MemoryStream _fileTableWriter = new();
private Dictionary<string, uint> _fileNameToIndex = new();

public CodeViewFileTableBuilder()
{
// Insert empty entry at the beginning of string table
_stringTableWriter.Write(stackalloc byte[2]);
}

public uint GetFileIndex(string fileName)
{
if (fileName == "")
{
fileName = "<stdin>";
}

if (_fileNameToIndex.TryGetValue(fileName, out uint fileIndex))
{
return fileIndex;
}
else
{
uint stringTableIndex = (uint)_stringTableWriter.Position;
_stringTableWriter.Write(Encoding.UTF8.GetBytes(fileName));
_stringTableWriter.WriteByte(0);

uint fileTableIndex = (uint)_fileTableWriter.Position;
Span<byte> fileTableEntry = stackalloc byte[sizeof(uint) + sizeof(uint)];
BinaryPrimitives.WriteUInt32LittleEndian(fileTableEntry, stringTableIndex);
BinaryPrimitives.WriteUInt32LittleEndian(fileTableEntry.Slice(4), 0);
_fileTableWriter.Write(fileTableEntry);

_fileNameToIndex.Add(fileName, fileTableIndex);

return fileTableIndex;
}
}

public void Write(SectionWriter sectionWriter)
{
sectionWriter.WriteLittleEndian<uint>((uint)DebugSymbolsSubsectionType.FileChecksums);
sectionWriter.WriteLittleEndian<uint>((uint)_fileTableWriter.Length);
sectionWriter.EmitData(_fileTableWriter.GetBuffer().AsMemory(0, (int)_fileTableWriter.Length));
sectionWriter.WriteLittleEndian<uint>((uint)DebugSymbolsSubsectionType.StringTable);
sectionWriter.WriteLittleEndian<uint>((uint)_stringTableWriter.Length);
sectionWriter.EmitData(_stringTableWriter.GetBuffer().AsMemory(0, (int)_stringTableWriter.Length));
}
}
}

Large diffs are not rendered by default.

Loading
Loading