Tar: Fix buffer is too small error when writing Prefix on large filenames #75023

jozkee · 2022-09-02T20:24:09Z

Found while addressing #74316 (comment).

System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(testCaseName: "file_longsymlink") [FAIL]
  System.ArgumentException : The output byte buffer is too small to contain the encoded data, encoding codepage '20127' and fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')
  Stack Trace:
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.cs(1090,0): at System.Text.Encoding.ThrowBytesOverflow()
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.cs(1101,0): at System.Text.Encoding.ThrowBytesOverflow(EncoderNLS encoder, Boolean nothingEncoded)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.Internal.cs(682,0): at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/ASCIIEncoding.cs(411,0): at System.Text.ASCIIEncoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.Internal.cs(497,0): at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/ASCIIEncoding.cs(320,0): at System.Text.ASCIIEncoding.GetBytes(ReadOnlySpan`1 chars, Span`1 bytes)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(382,0): at System.Formats.Tar.TarHeader.WritePosixName(Span`1 buffer)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(353,0): at System.Formats.Tar.TarHeader.WriteAsPaxSharedInternal(Span`1 buffer, Int64& actualLength)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(322,0): at System.Formats.Tar.TarHeader.WriteAsPaxInternal(Stream archiveStream, Span`1 buffer)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(153,0): at System.Formats.Tar.TarHeader.WriteAsPax(Stream archiveStream, Span`1 buffer)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(291,0): at System.Formats.Tar.TarWriter.WriteEntryInternal(TarEntry entry)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(219,0): at System.Formats.Tar.TarWriter.WriteEntry(TarEntry entry)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(164,0): at System.Formats.Tar.TarWriter.ReadFileFromDiskAndWriteToArchiveStreamAsEntry(String fullPath, String entryName)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(135,0): at System.Formats.Tar.TarWriter.WriteEntry(String fileName, String entryName)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs(297,0): at System.Formats.Tar.TarFile.CreateFromDirectoryInternal(String sourceDirectoryName, Stream destination, Boolean includeBaseDirectory, Boolean leaveOpen)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs(102,0): at System.Formats.Tar.TarFile.CreateFromDirectory(String sourceDirectoryName, String destinationFileName, Boolean includeBaseDirectory)
    /home/david/runtime/src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs(280,0): at System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(String testCaseName)
       at InvokeStub_TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(Object, Object, IntPtr*)
       at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

TODO:
Add similar test with TarWriter, async, and Stream.

Since this may be backport-worthy, consider merging without the aforementioned.

ghost · 2022-09-02T20:24:23Z

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

Found while addressing #74316 (comment).

System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(testCaseName: "file_longsymlink") [FAIL]
  System.ArgumentException : The output byte buffer is too small to contain the encoded data, encoding codepage '20127' and fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')
  Stack Trace:
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.cs(1090,0): at System.Text.Encoding.ThrowBytesOverflow()
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.cs(1101,0): at System.Text.Encoding.ThrowBytesOverflow(EncoderNLS encoder, Boolean nothingEncoded)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.Internal.cs(682,0): at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/ASCIIEncoding.cs(411,0): at System.Text.ASCIIEncoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/Encoding.Internal.cs(497,0): at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar)
    /home/david/runtime/src/libraries/System.Private.CoreLib/src/System/Text/ASCIIEncoding.cs(320,0): at System.Text.ASCIIEncoding.GetBytes(ReadOnlySpan`1 chars, Span`1 bytes)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(382,0): at System.Formats.Tar.TarHeader.WritePosixName(Span`1 buffer)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(353,0): at System.Formats.Tar.TarHeader.WriteAsPaxSharedInternal(Span`1 buffer, Int64& actualLength)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(322,0): at System.Formats.Tar.TarHeader.WriteAsPaxInternal(Stream archiveStream, Span`1 buffer)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs(153,0): at System.Formats.Tar.TarHeader.WriteAsPax(Stream archiveStream, Span`1 buffer)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(291,0): at System.Formats.Tar.TarWriter.WriteEntryInternal(TarEntry entry)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(219,0): at System.Formats.Tar.TarWriter.WriteEntry(TarEntry entry)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(164,0): at System.Formats.Tar.TarWriter.ReadFileFromDiskAndWriteToArchiveStreamAsEntry(String fullPath, String entryName)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarWriter.cs(135,0): at System.Formats.Tar.TarWriter.WriteEntry(String fileName, String entryName)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs(297,0): at System.Formats.Tar.TarFile.CreateFromDirectoryInternal(String sourceDirectoryName, Stream destination, Boolean includeBaseDirectory, Boolean leaveOpen)
    /home/david/runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs(102,0): at System.Formats.Tar.TarFile.CreateFromDirectory(String sourceDirectoryName, String destinationFileName, Boolean includeBaseDirectory)
    /home/david/runtime/src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs(280,0): at System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(String testCaseName)
       at InvokeStub_TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(Object, Object, IntPtr*)
       at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

TODO:
Add similar test with TarWriter, async, and Stream.

Since this may be backport-worthy, consider merging without the aforementioned.

Author:	Jozkee
Assignees:	Jozkee
Labels:	`area-System.IO`
Milestone:	-

src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs

src/libraries/System.Formats.Tar/tests/TarTestsBase.cs

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

carlossanlop · 2022-09-02T20:44:14Z

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

+
+        [ConditionalTheory]
+        [PlatformSpecific(TestPlatforms.AnyUnix)]
+        [MemberData(nameof(GetPaxAndGnuTestCaseNames))]


We need another condition here confirming that the /usr/bin/tar binary exists. We don't know if some Unix flavors put the binary somewhere else.

We assume /bin/ln always exists, I thought we could do the same here.

runtime/src/libraries/Common/tests/System/IO/ReparsePointUtilities.cs

Line 94 in 9ea5d2d

symLinkProcess.StartInfo.FileName = "/bin/ln";

I guess we will know when the CI is done.

Another option is to assume /usr/bin is on the $PATH in every distro we test on. Just as you're assuming here that C:\Windows\System32 is on the Windows PATH

Hmm, on Alpine, it's implemented by Busybox, with a link. So /usr/bin/tar does not exist, but tar should work:

danmoseL:~# ls -alF /bin/tar lrwxrwxrwx 1 root root 12 Mar 31 2021 /bin/tar -> /bin/busybox*

And incidentally, the busybox tar is probably significantly different to the GNU and BSD tar's so it would be good to try it. (You can also do it locally in WSL2 of course if you want)

src/libraries/System.Formats.Tar/tests/TarTestsBase.cs

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

carlossanlop · 2022-09-02T20:56:13Z

src/libraries/System.Formats.Tar/tests/TarTestsBase.cs

+
+            using Process tarToolProcess = new Process();
+            tarToolProcess.StartInfo.FileName = "/usr/bin/tar";
+            tarToolProcess.StartInfo.Arguments = $"-xf {source} -C {destination}";


We could add an additional boolean parameter isCompressed to this ExtractWithTarTool method. If true, then the arguments could include the z letter, which is used to extract *.tar.gz files.

We would just have to adjust line 271 of the ResultingArchive_CanBeExtractedByOtherTools test method, so that it takes a variable CompressionMethod.

Yes, that should be part of the TODOs about adding more variants of this test.

carlossanlop

LGTM so far. We can add the rest of the tests in a separate PR. Let's wait for the CI to finish.

carlossanlop · 2022-09-02T21:35:26Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-09-02T21:35:42Z

Azure Pipelines successfully started running 1 pipeline(s).

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

jozkee · 2022-09-06T17:30:54Z

One CI issue is just that some unix images doesn't have /usr/bin/tar.

Second CI issue seems to be an encoding error on macOS, see below diff comparing what filenames bsd tar produces vs what we produce:

- path: /földër bytes: 2F66C3B66C64C3AB72
+ path: /földër bytes: 2F666FCC886C6465CC8872
- path: /földër/áöñ.txt bytes: 2F66C3B66C64C3AB722FC3A1C3B6C3B12E747874
+ path: /földër/áöñ.txt bytes: 2F666FCC886C6465CC88722F61CC816FCC886ECC832E747874

danmoseley · 2022-09-06T19:37:52Z

@jozkee you mention this is addressing the question "do we write archives other tools/libraries can successfully open" .. can you say what testing you're doing / what your strategy is? eg.., how are you constructing the range of archives, and what tools are you testing accept them. How did you find this bug?

jozkee · 2022-09-06T19:57:30Z

@danmoseley I just wrote a test (in this PR, named ResultingArchive_CanBeExtractedByOtherTools) that does the following:

takes *.tar files from runtime-assets and extracts them with /usr/bin/tar into a folder which is going to be used as the baseline.
The baseline folder is passed to our dotnet API TarFile.CreateFromDirectory
The archive from step 2 is then also extracted with /usr/bin/tar and the result is compared with the baseline folder i.e: the file system entries should match.

We could do the same for other tools, such a 7zip, but I thought that testing against the tar utility was a good starting point.

danmoseley · 2022-09-06T20:04:11Z

That seems like a pretty good strategy.

jozkee · 2022-09-06T20:04:37Z

How did you find this bug?

We didn't have tests that tried to write files like the ones in https://github.com/dotnet/runtime-assets/tree/main/src/System.Formats.Tar.TestData/tar/gnu. So, some write edge-cases went untested, in the case of the bug I found, it was attempting to write a file with a large name (~256 characters).

danmoseley · 2022-09-06T21:00:54Z

BTW, you might try both GNU tar (typical in Linux) and BSD tar (the one in Windows) if it's easy to do both. The latter is based on libarchive, and I think (?) GNU tar is not, so there's different coverage.

src/libraries/System.Formats.Tar/tests/TarTestsBase.cs

danmoseley · 2022-09-07T21:17:54Z

src/libraries/System.Formats.Tar/tests/TarTestsBase.cs

+            tarToolProcess.StartInfo.Arguments = "--help";
+
+            tarToolProcess.StartInfo.RedirectStandardOutput = true;
+            tarToolProcess.Start();


this will fail if the test is run on a version of Windows prior to when they added 'tar'. do we have a "find on path" method or existing code we can use? or, we could catch whatever the exception is when the file does not exist.

According to this: https://docs.microsoft.com/en-us/virtualization/community/team-blog/2017/20171219-tar-and-curl-come-to-windows

Beginning in Insider Build 17063, we’re introducing two command-line tools to the Windows toolchain: curl and bsdtar

I assume they're talking about Windows 11. Interestingly, my Windows 10 machine also has tar available, so I think the document is incomplete.

On Windows 10, executing tar -h on PowerShell gets me this:

bsdtar 3.5.2 - libarchive 3.5.2 zlib/1.2.5.f-ipp

And on Windows 11:

bsdtar 3.5.2 - libarchive 3.5.2 zlib/1.2.5.f-ipp bz2lib/1.0.6

Decoder .. this is Windows 10 RS4, in 2018.

Ah good to know. The article's date is 04/25/2022, that's why I was assuming Win11.

Also note we support Windows 10 from RS1 (1607)

@danmoseley if it fails, it will just skip the test, something like this:

Discovering: System.Formats.Tar.Tests (method display = ClassAndMethod, method display options = None) Discovered: System.Formats.Tar.Tests (found 1 of 578 test case) Starting: System.Formats.Tar.Tests (parallel test collections = on, max threads = 12) System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools [SKIP] Condition(s) not met: "CanExtractWithTarTool (TargetInvocationException)" Finished: System.Formats.Tar.Tests === TEST EXECUTION SUMMARY === System.Formats.Tar.Tests Total: 1, Errors: 0, Failed: 0, Skipped: 1, Time: 0.172s

I will wrap the code in a try-catch, just in case.

Now I wonder, if that's how xUnit handles exceptions on expressions used for ConditionalFact/TheoryAttributes, does it mean that we can end up with tests no longer running and we will never notice?
cc @dotnet/area-infrastructure-libraries

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

…ursive to base

carlossanlop · 2022-09-08T01:05:00Z

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

@@ -253,5 +254,50 @@ public void SkipRecursionIntoBaseDirectorySymlink()

            Assert.Null(reader.GetNextEntry());
        }
+
+        [ConditionalTheory(nameof(CanExtractWithTarTool))]
+        [MemberData(nameof(GetPaxAndGnuTestCaseNames))]


As discussed via chat - Some link tests are failing, possibly because the OS does not support the creation of symlinks, so they should get separated into their own tests and decorated with the CanCreateSymbolicLinks condition.

Addressed slightly different on latest commit:

runtime/src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

Lines 267 to 271 in 1b4bf33

string[] symlinkTestCases = { "file_symlink", "foldersymlink_folder_subfolder_file", "file_longsymlink" };

if (symlinkTestCases.Contains(testCaseName) && !MountHelper.CanCreateSymbolicLinks)

{

throw new SkipTestException("Symlinks are not supported on this platform.");

}

Nvm, this wasn't the reason of the failure, my guess is that Windows Nano Server has a bsdtar version that does not support extraction of symlinks.

Maybe, but won't this fail on any Windows version if MountHelper.CanCreateSymbolicLinks is false? I don't see it being checked in the current commit.

As you know symlinks on Windows require elevation unless you have developer mode. You almost certainly have developer mode enabled on your own machine, but it may not be enabled on others' machines. This won't show up in Helix because I believe they run all tests elevated, but any test we have that creates symlinks mst test MountHelper.CanCreateSymbolicLinks (even testing you're elevated may not be enough, eg ?Nano)

I don't see it being checked in the current commit.

Yeah, I removed it because nano was still failing (even with the CanCreateSymbolicLinks check).
You can see the run for that commit (1b4bf33):
https://github.com/dotnet/runtime/pull/75023/checks?check_run_id=8242739135

Got this error log:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-75023-merge-fbe5d4bb79764adabe/System.Formats.Tar.Tests/1/console.da17f2b2.log?helixlogtype=result

I believe you, just saying that if you fix whatever that problem is, you'd still need that check

How are we using CanCreateSymbolicLinks in our System.IO symlink tests?

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

Make CanExtractWithTarTool return false for old Windows versions.

jozkee · 2022-09-08T04:21:25Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-09-08T04:21:38Z

Azure Pipelines successfully started running 1 pipeline(s).

jozkee · 2022-09-08T15:18:13Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-09-08T15:19:09Z

Azure Pipelines successfully started running 1 pipeline(s).

jozkee · 2022-09-08T15:25:12Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-09-08T15:25:40Z

Azure Pipelines successfully started running 1 pipeline(s).

danmoseley · 2022-09-08T15:50:42Z

Just curious @jozkee what configuration are you needing to rerun here? Extra platforms uses 100's of machine hours each time so we've talked about making it more fine grained so it's less often used.

jozkee · 2022-09-08T15:57:16Z

@danmoseley right now I need android and windows nano server, the last one is part of the normal CI.

jozkee · 2022-09-08T16:58:45Z

The new test is now failing on Linux_bionic
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-75023-merge-2e8a6f0f2ed64304a2/System.Formats.Tar.Tests/1/console.1c3d2205.log?helixlogtype=result

[16:21:20] dbug: Working with API Android Debug Bridge version 1.0.41
Version 29.0.6-6198805
Installed as /datadisks/disk1/work/AA8B08DE/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22451.1/runtimes/any/native/adb/linux/adb

How is this platform not considered android i.e: Not caught by PlatformDetection.IsAndroid?

jozkee · 2022-09-08T17:00:03Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-09-08T17:00:37Z

Azure Pipelines successfully started running 1 pipeline(s).

jozkee · 2022-09-08T22:02:27Z

And again, now is failing on Win10 Arm64 and I don't know how to pull that image.
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-75023-merge-49b55036c3ed46348e/System.Formats.Tar.Tests/1/console.16a561df.log?helixlogtype=result

C:\helix\work\workitem\e>"C:\helix\work\correlation\dotnet.exe" exec --runtimeconfig System.Formats.Tar.Tests.runtimeconfig.json --depsfile System.Formats.Tar.Tests.deps.json xunit.console.dll System.Formats.Tar.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
  Discovering: System.Formats.Tar.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Formats.Tar.Tests (found 592 test cases)
  Starting:    System.Formats.Tar.Tests (parallel test collections = on, max threads = 2)
    System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(testCaseName: "specialfiles") [SKIP]
      specialfiles is only supported on Unix and it needs sudo permissions for extraction.
link.txt: Can't create '\\\\?\\C:\\Windows\\TEMP\\1qnpiub5.d4h\\tarExtracted\\link.txt'
tar: Error exit delayed from previous errors.
    System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(testCaseName: "file_symlink") [FAIL]
      Assert.Equal() Failure
      Expected: 0
      Actual:   1
      Stack Trace:
        /_/src/libraries/System.Formats.Tar/tests/TarTestsBase.cs(673,0): at System.Formats.Tar.Tests.TarTestsBase.ExtractWithTarTool(String source, String destination)
        /_/src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs(280,0): at System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(String testCaseName)
           at InvokeStub_TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(Object, Object, IntPtr*)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
childlink: Can't create '\\\\?\\C:\\Windows\\TEMP\\lres20zl.paz\\tarExtracted\\childlink'
tar: Error exit delayed from previous errors.
    System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(testCaseName: "foldersymlink_folder_subfolder_file") [FAIL]
      Assert.Equal() Failure
      Expected: 0
      Actual:   1
      Stack Trace:
        /_/src/libraries/System.Formats.Tar/tests/TarTestsBase.cs(673,0): at System.Formats.Tar.Tests.TarTestsBase.ExtractWithTarTool(String source, String destination)
        /_/src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs(280,0): at System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(String testCaseName)
           at InvokeStub_TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(Object, Object, IntPtr*)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
link.txt: Can't create '\\\\?\\C:\\Windows\\TEMP\\335s1okh.1j4\\tarExtracted\\link.txt'
tar: Error exit delayed from previous errors.
    System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(testCaseName: "file_longsymlink") [FAIL]
      Assert.Equal() Failure
      Expected: 0
      Actual:   1
      Stack Trace:
        /_/src/libraries/System.Formats.Tar/tests/TarTestsBase.cs(673,0): at System.Formats.Tar.Tests.TarTestsBase.ExtractWithTarTool(String source, String destination)
        /_/src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs(280,0): at System.Formats.Tar.Tests.TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(String testCaseName)
           at InvokeStub_TarFile_CreateFromDirectory_File_Tests.ResultingArchive_CanBeExtractedByOtherTools(Object, Object, IntPtr*)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
  Finished:    System.Formats.Tar.Tests
=== TEST EXECUTION SUMMARY ===
   System.Formats.Tar.Tests  Total: 1507, Errors: 0, Failed: 3, Skipped: 1, Time: 3.791s

jozkee · 2022-09-08T22:04:33Z

I think this fix is valuable but the test is clearly unreliable for merging (even undesirable for porting to 7.0). I think the best course of action right now is to submit the fix with a minimal test and defer tar tool validation for another PR for 8.0.
Thoughts?

carlossanlop · 2022-09-08T23:40:16Z

But the ARM64 failure is that links cannot be created in the filesystem:

childlink: Can't create '\\\\?\\C:\\Windows\\TEMP\\lres20zl.paz\\tarExtracted\\childlink'

We could just skip the test in that platform.

jozkee · 2022-09-09T16:00:38Z

@carlossanlop I feel there will be another config that will then start failing either in the future of main branch or in release/7.0.
Closing in favor of #75350.

Fix WritePosixName to use a correct size buffer

e32acb8

jozkee requested a review from carlossanlop September 2, 2022 20:24

dotnet-issue-labeler bot added the area-System.IO label Sep 2, 2022

ghost assigned jozkee Sep 2, 2022

jozkee requested a review from stephentoub September 2, 2022 20:31