Strawberry Perl POC and Status of 5.34.1

[rafulbrig]

I am preparing to upgrade to the latest release of Strawberry Perl (which shows as 5.32.1.1). I then noticed that Perl itself is at release 5.34.1. I tried to find the name of the POC for Strawberry Perl, but was unsuccessful. What is the contact info for the current manager of Strawberry Perl releases?

I am trying to find out if the upgrade to Strawberry Perl to release 5.34.1 is imminent. If it is, I am happy to wait before upgrading, so I do not have to upgrade twice. Any info you can provide will be most appreciated. Thanks.

[genio]

We are trying to get things working. Life has hindered some of us from being able to work on it some. It's in the works, but I can't give an honest estimate of the time it will take at the moment. Apologies for that.

[rafulbrig]

Thank you Genio. I appreciate all the effort involved in a new release.

[aero]

Perl 5.36.0 released.
I would like to see the new version of strawberry perl.

[ludovicroland]

Hello :)

Same here, I'd love to test Perl 5.34 in order to use, at least, the native try catch blocks :)

Hope to see an update soon 🤞

[sisyphus]

I would recommend building perl yourself.
It's really easy - and the more people that are building perl on windows, then the better our chances that it will stay that way .... and the better our chances of getting the "messy bits" (see below) fixed.

Start by grabbing the perl source:
https://cpan.metacpan.org/authors/id/X/XS/XSAWYERX/perl-5.34.0.tar.gz
https://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl-5.36.0.tar.gz

(It wasn't until I finished writing this that I realized you had specified 5.34.0, not 5.36.0.)

Untar that perl source into some directory - say C:\straw-comp.
Then cd to the C:\straw-comp\perl-5.36.0\win32 folder.

Given that I want to install a 64 bit build of perl-5.36.0 into C:\straw\perl-5.36.0, I'll need to include the following command line argument:

INST_TOP=C:\straw\perl-5.36.0

(Modify accordingly if you want to install it elsewhere.)

I'll also want to specify CCHOME, which is the directory that contains
bin/gcc.exe. For me, that directory is C:_64\strawberry-5.32.0\c.
Therefore, I will need to include the command line argument:

CCHOME=C:\_64\strawberry-5.32.0\c 

(Modify accordingly if your CCHOME is somewhere else.)

##############
The messy bits
##############

Annoyingly, there's some cpan/IO-Compress/t/ tests that hang for me.
To avoid this issue during the running of the perl test suite, I first set the following environment variable:

set IO_COMPRESS_SKIP_STDIN_TESTS=1

Then, after I've built perl with 'gmake .... test', I clear that environment variable, cd to 'cpan/IO-Compress', run 'gmake test' and sit back while all test pass.

There might also be a hang with IPC-Cmd/t/01_IPC-Cmd.t. (Let me know if you see that one.)
I actually patch that test file to skip all tests if $ENV{IPC_CMD_SKIP_TESTS} is set to a true value.
It's a simple patch - just insert the following at the beginning of the file, before building perl:

if($ENV{IPC_CMD_SKIP_TESTS}) {
  ok('1' eq '1', 'test 1');
  exit 0;
}

And, of course, I also:

set IPC_CMD_SKIP_TESTS=1

Then, after I've built perl with 'gmake ... test', and still in the 'win32' folder, I clear that environment variable and run:

..\perl -I..\lib ../cpan/IPC-Cmd/t/01_IPC-Cmd.t >out.txt

I can then examine out.txt, whereupon I clearly see that all tests have passed.

If a test hangs, you can just kill it using Process Explorer (if you have that).
If you kill it with Ctrl-C, then the running of the test suite aborts immediately.

#####
Build it
#####

Run 'gmake test':
C:\straw-comp\perl-5.36.0\win32> gmake INST_TOP=C:\straw\perl-5.36.0 CCHOME=C:_64\strawberry-5.32.0\c test

Then run 'gmake install', repeating the same options as were used for 'gmake test'.
C:\straw-comp\perl-5.36.0\win32> gmake INST_TOP=C:\straw\perl-5.36.0 CCHOME=C:_64\strawberry-5.32.0\c install

That's it. Then just run:
set PATH=C:\straw\perl-5.36.0\bin;%PATH% perl -V
and all should be good.
I've actually just gone through the above procedure, using my 64-bit Strawberry Perl-5.32.0 installation.
Apart from the afore-mentioned hangs, it was perfect.

#############
SOME OTHER OPTIONS:
#############

Not actually necessary, but I recommend adding

USE_MINGW_ANSI_STDIO=define BUILDOPTEXTRA=-fdiagnostics-color=never

to the above command line args.

If your Strawberry Perl (and therefore gcc also) is 32-bit then you'll need to add

WIN64=undef

to the above command line args.

If your Strawberry Perl (and therefore gcc also) is 32-bit and you want 64-bit integers then you'll need to add

USE_64_BIT_INT=define

to the above command line args.

Alternatively, if you want your nvtype to be 'long double', then you'll need to add:

USE_LONG_DOUBLE=define

to the above command line args.

Alternatively, if you want your nvtype to be '__float128', then you'll need to add:

USE_QUADMATH=define I_QUADMATH=define

to the above command line args.

If you want to re-use the same source to build a perl with a different configuration, or using a different compiler, just run 'make distclean' first.

Cheers,
Rob

[aero]

Besides it could be built own windows perl.

The new language version announcement with binaries for major OSes is considered common and basic.

See the new kids on the block & hipster language zig's download page( https://ziglang.org/download/ ).
Sister languge Raku is also doing so. https://rakudo.org/downloads

Why not for perl itself?

I think this is a matter that TPF(the perl foundation) should be concerned about.

[genio]

@aero It's an extremely complex process being completed by volunteers. The TPF doesn't have anything to do with the Strawberry build of Windows to my knowledge. We're making progress on it every day, but we're trying to do so in a way that will make future builds less opaque and allow more people to work on it. Please don't come in here being rude and saying because others do it, we should be able to with no problem.

Understand the entirety of the problem set starting with some of the information here: https://github.com/StrawberryPerl/spbuild and then start working through the steps. None of it's simple on Windows.

[aero]

It's an extremely complex process being completed by volunteers. The TPF doesn't have anything to do with the Strawberry build of Windows to my knowledge. We're making progress on it every day, but we're trying to do so in a way that will make future builds less opaque and allow more people to work on it. Please don't come in here being rude and saying because others do it, we should be able to with no problem.

I'm sorry if it felt rude.

I hope perl has multiple options like python.
Official release:

• https://www.python.org/downloads/windows/
  Alternative python distributions:
• https://wiki.python.org/moin/PythonDistributions

I mentioned TPF because I think TPF needs to support or be responsible for the official windows perl package like other major languages by donation or grant apart from Strawberry perl.

Understand the entirety of the problem set starting with some of the information here: https://github.com/StrawberryPerl/spbuild and then start working through the steps. None of it's simple on Windows.

Thank you for your unseen efforts.

[sisyphus]

@genio, I don't think @aero was being rude.
My take is that he was simply wishing that the perl developers would release perl binaries for windows (and other systems) - just as the raku developers do with raku, and the python developers do with python.
He took it a bit further by also expressing his view that it would be in the best interests of perl itself if the perl developers were to do precisely that.

Of course, they never have, and they probably never will - but @aero's POV on this makes reasonable sense to me.

I don't think he was suggesting that it was time for the perl developers to take responsibility for the release of Strawberry Perl.
If they can't be bothered releasing CORE perl binaries, then they're certainly not going to get involved in providing a gcc toolchain, 3rd party C libraries and associated perl extensions as well.

Cheers,
Rob

[blues1875]

Thanks for that great HOWTO, Rob (sisyphus). I decided to have a go myself and managed to build 5.36 starting with my SP 5.32 installation, as you did. Interestingly, I had no problem with IO::Compress, but my build failed on HTTP::Tiny. My solution was to move that folder out of the cpan folder and running gmake again. I'll do some more experimenting when I get time (you asked us to let you know about test failures, but I'm not sure where; I don't think this is the correct forum).

As others have pointed out, Strawberry Perl is more than just the latest perl, so I kept the rest of the distribution, swapping the original strawberry/perl folder with the new one. I bootstrapped cpanm by copying the fat-packed script over as cpanm.pl and running

perl cpanm.pl App::cpanminus

After that, I could build and install other modules in the usual way, e.g.

cpanm Moo

I did hit a snag with namespace::clean where it complained about the missing libstdc++-6.dll. I guess this is where SP is more than just perl... Any advice on that? I just copied that and the other additional DLLs over from SP 5.32 and re-ran cpanm. That appeared to work and some simple scripts are running fine, but I just worry that I've left a ticking time bomb in there!

[sisyphus]

On Sun, Jul 17, 2022 at 10:30 PM blues1875 ***@***.***> wrote: Thanks for that great HOWTO, Rob (sisyphus). I decided to have a go myself and managed to build 5.36 starting with my SP 5.32 installation, as you did.

Excellent !! I encourage you to make a habit of repeating the exercise from time to time. You'll find basic documentation on those (and other) options that I mentioned in win32/GNUmakefile in the perl source. You can even implement those options by editing that file by hand - but I think it's simpler to just specify them on the command line, especially if you're doing scripted builds. (Editing the GNUmakefile by hand is tedious.)

Interestingly, I had no problem with IO::Compress, but my build failed on HTTP::Tiny. My solution was to move that folder out of the cpan folder and running gmake again. I'll do some more experimenting when I get time (you asked us to let you know about test failures, but I'm not sure where; I don't think this is the correct forum).

That's a failure I've not struck. It should be reported to https://github.com/chansen/p5-http-tiny/issues

As others have pointed out, Strawberry Perl is more than just the latest perl, so I kept the rest of the distribution, swapping the original strawberry/perl folder with the new one. I bootstrapped cpanm by copying the fat-packed script over as cpanm.pl and running perl cpanm.pl App::cpanminus After that, I could build and install other modules in the usual way, e.g. cpanm Moo I did hit a snag with namespace::clean where it complained about the missing libstdc++-6.dll.

libstdc++-6.dll is in your strawberry/c/bin folder - alongside other crucial files like gcc.exe and gmake.exe. You'll want to keep that folder in your path.

I guess this is where SP is more than just perl... Any advice on that? I just copied that and the other additional DLLs over from SP 5.32 and re-ran cpanm. That appeared to work and some simple scripts are running fine, but I just worry that I've left a ticking time bomb in there!

My StrawberryPerl-5.32.0 installation is from the "portable" distro, and is located in C:\_64\strawberry-5.32.0. So, I double-clicked C:\_64\strawberry-5.32.0\portableshell.bat, which opened up a new shell with an appropriately set PATH Then I built perl-5.36.0 as previously described, installing it into C:/ctraw/perl-5.36.0. Then, all I did, was to prepend my perl-5.36.0 to the already existing PATH - by running: set PATH=C:\straw\perl-5.36.0\bin;%PATH% When I run 'cpan -i namespace::clean' I get the very same error wrt libstdc++-6.dll as you got - yet I know that libstdc++-6.dll is in my PATH. Apparently the namespace::clean test suite does abusive stuff to the PATH. One thing that SP does is to place copies of libgcc_s_seh-1.dll, libstdc++-6.dll and libwinpthread-1.dll (all of which are found in SP's c/bin folder) into SP's perl/bin folder. I suspect they do that to ensure that if something does remove c/bin from the PATH, then those 3 files will still be found - by virtue of them being in the same directory as the perl dll. Anyway ... copying those 3 files across from SP's c/bin to my perl-5.36.0/bin fixed that issue for me. I don't think there should be any bombs in there ... of course, I haven't really played about with this much, so I could be wrong. I was surprised to find that my Math::GMPz, Math::GMPq, Math::GMPf and Math::MPFR modules all built straight out of the box on this perl-5.36.0, using the static gmp and mpfr libraries that shipped with SP-5.32.0. I had thought that this perl-5.36.0 might not know where to find those libraries by default ... but it found them just fine. I was even able to build and install Net-SSLeay-1.92 after setting the OPENSSL_PREFIX environment variable to C:\_64\strawberry-5.32.0\c. Of course, it's quite possible that the next SP releases will ship with more recent versions of those libraries, and hopefully a more recent version of the gcc toolchain, too. Cheers, Rob

[blues1875]

I just hit another problem with my new 5.36 build: Perl-Critic wouldn't build because of a problem with Module-Pluggable, provoked by an update to File-Find. See simonwistow/Module-Pluggable#24

[sisyphus]

Interesting. There's a number of Linux smokers that would have detected this problem when File-Find-1.39 was first introduced to bleadperl ...., if it was something that affected Linux.
But, of course, there are no such Windows smokers, so it goes undetected until a user actually hits the issue some time later.

I gather that the problem is not that File::Find is ding something it shouldn't ... instead it's a case of Module::Pluggable making an assumption that it shouldn't ?

It took me a bit longer than it should have to work out what the patch at simonwistow/Module-Pluggable#24 actually was.
For others who are also a bit slow, it's:

--- Object.pm_orig	2022-07-25 11:42:24 +1000
+++ Object.pm	2022-07-25 12:12:02 +1000
@@ -66,6 +66,7 @@
     $self->{'on_instantiate_error'} ||= sub { my ($plugin, $err) = @_; carp "Couldn't instantiate $plugin: $err"; return 0 };
 
     # default whether to follow symlinks
+    $self->{'follow_symlinks'} = 0 if ($^O eq 'MSWin32' && !exists $self->{'follow_symlinks'});
     $self->{'follow_symlinks'} = 1 unless exists $self->{'follow_symlinks'};
 
     # check to see if we're running under test

The author of Module::Pluggable hasn't released anything on CPAN for nearly 7 years, so I'm not sure how long we might be waiting for the patch to be implemented and a fixed version of Module::Pluggable released.

Cheers,
Rob

[blues1875]

This has been quite an interesting exercise! I think you are right to encourage others to build their own perl... the devil is in the details, as always! I had problems with Module::Find, too. That module also uses File::Find and is related to symlinks, but that one is a bit harder to figure out. Once SP 5.36 (or even 5.34) is released, I'm sure more people will find more of these problems, but I understand that there is a lot of work involved with putting together a new release of SP--many thanks to you guys. I hope my experience helps, if just a bit. Building our own perl and finding these glitches is a good way for the rest of us to contribute.

Glenn

[sisyphus]

I suspect that the "real spanner in the works" (and the the thing that needs fixing) is File::Find.
However, I don't really know.
The telling part of the diff between File-Find-1.37 and File-Find-1.39 is:

....
-our $VERSION = '1.37';
+our $VERSION = '1.39';
 require Exporter;
 require Cwd;
 
@@ -161,9 +161,8 @@
     $pre_process       = $wanted->{preprocess};
     $post_process      = $wanted->{postprocess};
     $no_chdir          = $wanted->{no_chdir};
-    $full_check        = $Is_Win32 ? 0 : $wanted->{follow};
-    $follow            = $Is_Win32 ? 0 :
-                             $full_check || $wanted->{follow_fast};
+    $full_check        = $wanted->{follow};
+    $follow            = $full_check || $wanted->{follow_fast};
     $follow_skip       = $wanted->{follow_skip};
     $untaint           = $wanted->{untaint};
     $untaint_pat       = $wanted->{untaint_pattern};
@@ -840,6 +839,9 @@
....

It doesn't feel right to me that such changes have been made, given that they're breaking both Module::Pluggable and Module::Find. (And I think it's quite possible that the developers who made those changes are not even aware of those breakages.)

Should we question the validity of those changes to Find.pm by raising an Issue at https://github.com/Perl/perl5/issues ??

Cheers,
Rob

[genio]

@sisyphus You might want to float this in #p5p on magnet and get some conversation going there before putting in a bug report. They'll help make sure it has all of the info int it that it needs

[bekuno]

The change was intentional to support Win32 symlinks, see
Perl/perl5@0d00729

[jkeenan]

Should we question the validity of those changes to Find.pm by raising an Issue at https://github.com/Perl/perl5/issues ??

Well, you're getting pretty far away from the original complaint raised in this issue, so you should probably start a fresh ticket somewhere. If you believe the problem is in the Perl core distribution -- as distinct from Strawberry Perl or some (non-core) CPAN module, then you should probably check your results both before and at this commit:

$ gitshowf 0d00729c03a
commit 0d00729c03a1f68e1b51e986d1ce9000b0e3d301
Author:     Tony Cook <tony@develop-help.com>
AuthorDate: Wed Oct 7 12:07:31 2020 +1100
Commit:     Tony Cook <tony@develop-help.com>
CommitDate: Tue Dec 1 15:29:33 2020 +1100

    File::Find: support Win32 symlinks
    
    find.t, taint.t: check that symlink() works under the current
    permissions/filesystem rather than assuming it will work
    
    find.t: since symlinks are now available, an earlier test block
    set $FileFileTests_OK, and the tests in this Win32 block don't use
    either of the follow options, which is required for fast file tests.
    
    taint.t: ensure we get "/" separated names to match File::Find's output

[blues1875]

Here is another issue with perl 5.36, @sisyphus. Path-Class fails build tests. See kenahoo/Path-Class#55.

Path-Class is a dependency for DBIx-Class-Schema-Loader.

[sisyphus]

Path-Class fails build tests.

Yeah ... there's an issue with t/01-basic.t on Windows whereby tests like:
ok $file3, 'dir/foo/bar.txt';
are failing because $file3 is actually 'dir\foo\bar.txt'.

To overcome this, t/01-basic.t begins with:

BEGIN {
  $^O = 'Unix'; # Test in Unix mode
}

and that takes care of that particular issue nicely. (There's at least one other Windows-vs-Unix incompatibility that it also attends to.)
Unfortunately, on perl-5.34.0 and later, it also leads to the fatal error "Can't call method "relative" on an undefined value ...".

I think the first thing that needs to be established is whether this hack of overriding $^O is deemed a valid technique.
(I think this means raising another Perl5 Issue.)
If it's not valid, then an alternative method of correctly performing the tests is required.
Otherwise, the functionality that allowed the $^O override to work in perl-5.32 and earlier needs to be restored.

My reading of the "$^O" documentation in perlvar leads me to think that this technique is invalid - but there's no specific forbidding of overriding it.

Have a play around with removing the $^O override in t/01-basic.t. And check it on both perl-5.32 and perl-5.36.
What do you think ?

Cheers,
Rob

[blues1875]

I removed the $^O override. It doesn't bale out now on the fatal error, but it does fail 30 if the 78 tests with lots of complaints about backslashes like this:

# Test 6 got: "dir\\bar.txt" (t/01-basic.t at line 20)
#   Expected: "dir/bar.txt"

I guess I should report this over on Path-Class issues.

[sisyphus]

I don't know where to report it. Also, yesterday I failed to point out that (using un-edited Path-Class-0.37 source with perl-5.34.0 and later) the way in which t/01-basic.t fails depends upon whether it's run inside or outside the Test::Harness. 'gmake test' will trigger the "Can't call method...." error, but 'perl -Mblib t/01-basic.t' or 'perl t/01-basic.t' will trigger the failures that the $^O override is supposed to fix. I've just created https://www.perlmonks.org/?node_id=11145885 in an attempt to find out a bit more about what should be done. Cheers, Rob

…

On Tue, Aug 2, 2022 at 9:58 AM blues1875 ***@***.***> wrote: I removed the $^O override. It doesn't bale out now on the fatal error, but it does fail 30 if the 78 tests with lots of complaints about backslashes like this: # Test 6 got: "dir\\bar.txt" (t/01-basic.t at line 20) # Expected: "dir/bar.txt" I guess I should report this over on Path-Class issues. — Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAR3PBUHZ53ZWKXSN7HU23VXBQC7ANCNFSM5VAT63HQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kenneth-olwing]

Hi all,

I also have an interest in seeing new Strawberry dists...OR, in fact I may consider building my own Perl dist given the insights gained here. I already have a complete build process to do that on Linux, while it currently piggybacks on Strawberry, and adds a huge number of modules for our inhouse deployable Perl in order for me to ensure I have 'the same' stuff on our platforms.

Regardless of that, I'm interested in helping in various areas to the extent I'm able, with the goal of getting new Strawberry dists out. If that helps me in 'building 'my own dist', so much the better ;-). Since there's such a multitude of things to be aware of, the more of us the merrier I think when ferreting all the itty-bitty issues out in the open...

So in regard to that I seem to have a few other issues than already discussed above which I would like to compare notes on with others.

======

I currently see two issues with building perl-5.36.0 ('raw', figure this is best before trying to tackle the Perl::Dist::Strawberry build system):

For reference, I use the 'CCHOME' in a portable Strawberry 5.32.1 to build, a freshly unpacked perl-5.36.0.tar.gz and

gmake CCHOME=c:\ken1\slask\straw\SP-5.32.1.1\c INST_TOP=c:\ken1\slask\straw\ken1-perl-5.36.0 USE_MINGW_ANSI_STDIO=define

(note that using 5.34, I was forced to add "USE_64_BIT_INT=define" despite it shouldn't be needed...)

1. Build error in ext/Errno

NOTE: this problem will be reported on the https://github.com/Perl/perl5, but if you want to build 5.36.0, it can be patched to work around it. I haven't been able to test yet, but it seems that blead as of now should have the same problem.

During build of ext/Errno, the Errno_PL.PM is invoked, which basically as I understand it, collects all the possible include files in
the CCHOME, creates a tmp .c file which #include's them all, runs gcc to have it spit out all #defines, and then interprets that to find all possible error codes to create Errno.pm with. I think.

The problem (which apparently happens only with 5.36.0 [?]) is that the gcc run commonly errors out with something like this:

gmake[1]: Entering directory 'C:/ken1/slask/straw/perl-5.36.0/ext/Errno'
"C:\ken1\slask\straw\perl-5.36.0\miniperl.exe" "-I..\..\lib" Errno_pm.PL Errno.pm 
In file included from includes.c:2:
c:/Users/ken1/slask/straw/SP-5.32.1.1/c/lib/gcc/x86_64-w64-mingw32/8.3.0/include/avx5124fmapsintrin.h:25:3: error: #error "Never use <avx5124fmapsintrin.h> directly; include <x86intrin.h> instead."
 # error "Never use <avx5124fmapsintrin.h> directly; include <x86intrin.h> instead."
   ^~~~~

Unfortunately, this isn't trapped by the code (nor will the make file trap it even if the code would exit with an error exitcode).
It should also be noted that sometimes it doesn't happen, or you may get other variations of this
error: there's a number of include files that is coded in this manner, and since they are detected initially using a hash, the order in which they get listed as includes is not defined. It should be noted that there is an attempt at fixing this in the Errno_PL.PM code, it's just that it's only good for one or two particular include files (see Perl/perl5#18026 for this).
I'm unsure if this error actually makes a big difference, but still, it shouldn't happen.

If someone is working on a perl-5.36.0.pp file in P::D::S, this problem can be patched by making the file list be treated in
sorted order (no, I'm aware it's not a proper fix, it just happens to make the build happy as it makes the include files
come in acceptable and dependable order...):

diff --git a/ext/Errno/Errno_pm.PL b/ext/Errno/Errno_pm.PL
index ee2f4a3..2ad6cce 100644
--- a/ext/Errno/Errno_pm.PL
+++ b/ext/Errno/Errno_pm.PL
@@ -21,7 +21,7 @@ if ($Config{gccversion} ne '' && $^O eq 'MSWin32') {
     my %seen;
     open INCS, '>', 'includes.c' or
        die "Cannot open includes.c";
-    foreach $file (@files) {
+    foreach $file (sort @files) {
        next if $file eq 'errno.c';
        next unless -f $file;
        if ( $file eq 'avx512vpopcntdqvlintrin.h' || $file eq 'avx512bwintrin.h' ) {

2. Test errors

Test Summary Report
-------------------
re/reg_mesg.t                                                      (Wstat: 65280 (exited 255) Tests: 339 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
re/regexp_unicode_prop_thr.t                                       (Wstat: 65280 (exited 255) Tests: 4 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 1110 tests but ran 4.
porting/authors.t                                                  (Wstat: 0 Tests: 1 Failed: 1)
  Failed test:  1
porting/pending-author.t                                           (Wstat: 0 Tests: 1 Failed: 1)
  Failed test:  1
porting/regen.t                                                    (Wstat: 512 (exited 2) Tests: 15 Failed: 1)
  Failed test:  15
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 42 tests but ran 15.
win32/popen.t                                                      (Wstat: 2304 (exited 9) Tests: 0 Failed: 0)
  Non-zero exit status: 9
  Parse errors: No plan found in TAP output
../dist/Storable/t/utf8.t                                          (Wstat: 65280 (exited 255) Tests: 5 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 6 tests but ran 5.
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2766, Tests=1147620, 4469 wallclock secs (80.92 usr + 155.01 sys = 235.94 CPU)
Result: FAIL
gmake: *** [GNUmakefile:1810: test] Error 3

Some are inconsequential, but some individual errors, as well as some test runs exiting with 255 indicating some kind of core (seems to be unicode/utf8 related stuff though...some C libs that are broken on win32?).

Haven't tried to follow up on them, but anyone seeing the same and/or have some ideas on why?

===========

Lastly...this issue seems to be slightly overcrowded in discussing issues a bit beyond tracking the original subject, clouding it a bit. And I do the same...:-)

Is there a more convenient forum to discuss selfbuilding perl on win32 considering this? In particular to jointly help with the wish to see new Strawberry dists.

ken1

[sisyphus]

(note that using 5.34, I was forced to add "USE_64_BIT_INT=define" despite it shouldn't be needed...)

Oooo ... alarm bells ... this should only be the case if the gcc compiler is 32 bit.
And if you are using a 32-bit gcc compiler on a 64-bit Windows system then you'll need to add 'WIN64=undef'.

I have struck nothing remotely like the errors you are reporting.

It's probably a good idea to create a perl5 issue about this - rather than further pollute this thread. (I will follow that issue with interest.)
When you do create that Issue, please provide the output of running gcc --version, as I feel that's something we'll need to see.
If the one line of output produced by that command matches x86_64 then it's a 64 bit compiler ... else it's a 32-bit compiler,
Also, make sure you identify which version of Windows you're running - as that could also be relevant.

Cheers,
Rob

[kenneth-olwing]

Rob, thanks for your comment.

Re: polluting this thread (in general) - I agree; any suggestions where these type of postings could/should be made 'best'? Having said that I'll give some short replies here:

Re: my first problem, I've just filed Perl/perl5#20033; for me it's quite real and very reproducible and I'm guessing you actually can recreate it too; just run the Errno_pm.PL script a number of times per my example if you're curious. Then again, no biggie.

Re: 5.34 and USE_64_BIT_INT - I'm 64 bit, Strawberry 5.32.1, Win11 as build toolchain and certainly see that gcc, just as with 5.36 so it's confusing - as you say it shouldn't be necessary. I'll retest and ensure reproducibility and file an issue,,,

Finally: apart from the tests you mention giving problems (which they don't for me!), are you otherwise getting a clean test run? Only major difference (?) I can think of is that I'm using Win11...I'll soon be firing up my work computer with Win10 and try things there but...I'm certainly curious what I might be doing different/wrong...:-/

Thanks again,
ken1

[genio]

While I don't mind having this issue open for you to keep the information somewhere, it might be helpful for you to join the #p5p IRC channel as they'd have the most expertise in building Perl in general.

[kenneth-olwing]

Thank you!

[sisyphus]

I can reproduce the behaviour with perl Errno_pm.pl, but in building a couple of hundred different configurations of current perls on this windows box over the last couple of years it's not something that has ever cropped up.
(Once perl has been built, I'm not in the habit of ever running perl Errno_pm.pl :-)

I'll keep an eye on Perl/perl5#20033 and chime in if I've got anything to add - though I might not get a chance until early next week.

I've double-checked on the perl-5.36.0 that I built with Strawberry's gcc-8.3.0 toolchain. I definitely didn't explicitly define USE_64_BIT_INT, though $Config{use64bitint} reports 'define', $Config{ivtype} is 'long long' and $Config{ivsize} is 8.
The GNUmakefile contains:

ifeq ($(WIN64),define)
USE_64_BIT_INT	= define
endif

Unless that condition is evaluated as false, I don't see how USE_64_BIT_INT could be anything other than 'define'.

Cheers,
Rob

[shawnlaffan]

New dev release for 5.36.1, compiled using gcc-13.1, is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5361_20230506_gcc13

[shawnlaffan]

A second dev 5.36.1 dev release is now available from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5361_20230524_gcc13

The main change is that PDL now builds.

The list of open issues is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/labels/perl5.36

The ones I think we need to resolve for a proper release are BerkeleyDB and DBD::Oracle.

[PhilterPaper]

Regarding BerkeleyDB, I don't use it (it came preinstalled with Strawberry), but have successfully upgraded it in 5.24, 5.26, and 5.32.

Regarding DBD::Oracle, I don't use it (it came preinstalled with Strawberry), and have never been able to upgrade it. It appears to be missing some sort of Oracle client. I think consideration should be given to omitting it entirely, as most systems are not going to have Oracle installed by default. Ditto for DBD::Pg, as most systems are not going to have PostgreSQL installed by default. At worst, perhaps you could have dummy stubs for DBD::Oracle and DBD::Pg just to tell the user that Perl support is available, but not attempt an upgrade (e.g., an impossibly high version number, and instructions on how to install the real thing)?

It would be interesting to see if IPC::Run will install and upgrade, as I have not been able to upgrade it since 0.96. Math::MPFR chronically fails to upgrade earlier than 5.32, but as of 5.32 it appears to be OK. Finally 'B' only upgrades on the current (5.36?) Perl -- I have asked CPAN to consider putting in some guards so cpan doesn't try to upgrade it on older Perls. I don't know if you can do anything on your end. I don't use any of these packages, so it's merely a nuisance that the cpan tool tries (and fails) to upgrade them (I get around that by having a wrapper to exclude problem packages).

[shawnlaffan]

Thanks @PhilterPaper

The issue with BerkeleyDB is that it does not build against libdb built with gcc-13. Currently the SP5.36 dev releases are re-using the libdb from SP5.32. This works but some builds show warnings about not being able to find gcc-8 files, warnings I am assuming come from libdb files.

WRT DBD::Oracle I'm starting to think the same way as you. If I understand correctly, the license means we cannot redistribute its binaries. They are not provided with SP5.32, and probably earlier. If DBD::Oracle builds cleanly when the binaries are available (and the user sets the right env vars) then we perhaps don't need to provide it.

WRT DBD::Pg, Strawberry Perl provides the Postgres binaries. The possible issue is when the user tries to use its PL/Perl interface. FWIW, it seems not to be exercised in the DBD::Pg tests.

SP5.36 come with the latest IPC::Run.

There is also the spectre of build failures due to changes in gcc-13 (probably anything since gcc-8.3). We already patch in a number of perl 5.37 changes to get things to build (e.g. PDL), and it is possible some other modules will also show similar failures (maybe those for Tk are a manifestation of this). This can perhaps be considered just a normal part of the build cycle and thus not a blocker to any SP releases as those modules will need to update eventually anyway.

[sisyphus]

Math::MPFR chronically fails to upgrade earlier than 5.32,

Now fixed in the github repo (https://github.com/sisyphus/math-mpfr), and therefore will be fixed in Math-MPFR-4.27 when it is released.
At least, that (github) version of Math::MPFR now builds and installs fine for me on Strawberry Perl-5.24.4 - but if there are yet other problems to be dealt with, please open an Issue at https://github.com/sisyphus/math-mpfr/issues.

Cheers,
Rob

[PhilterPaper]

There is also the spectre of build failures due to changes in gcc-13 (probably anything since gcc-8.3). We already patch in a number of perl 5.37 changes to get things to build (e.g. PDL), and it is possible some other modules will also show similar failures (maybe those for Tk are a manifestation of this). This can perhaps be considered just a normal part of the build cycle and thus not a blocker to any SP releases as those modules will need to update eventually anyway.

That worries me. Ideally, you should not custom patch any package to get it to build on Windows/Strawberry, as opposed to a patch (to that package's basic source) that is universally distributed and works on all platforms, or at least, can conditionally build on Strawberry. There's no point in shipping Strawberry with a custom build of some package, when the package's next update on CPAN will then fail. At least, that's how I'm reading (misreading?) the quoted post.

Patches to Perl itself, to get it to play nice on Windows, are another matter. Presumably no one with Strawberry Perl installed is going to upgrade their Perl with a non-Strawberry replacement, so as long as Strawberry Perl language and behavior meets specs for Perl programs, that should be OK. There may still be the issue of a program or package developed on Strawberry, failing to run properly on non-Strawberry platforms, but hopefully that can be avoided by either tweaking Strawberry Perl, or doing something with the OS detected in the program.

TL;DR: if a package on CPAN can be built (without customization) for shipping with Strawberry, it should be possible to install/update directly from CPAN, and work.

[shawnlaffan]

TL;DR: if a package on CPAN can be built (without customization) for shipping with Strawberry, it should be possible to install/update directly from CPAN, and work.

@PhilterPaper - We're in agreement.

Exceptions for patching apply when a module has a reasonable number of dependencies (particularly in the SP builds), the issues have been reported upstream, and where a new release seems unlikely or not imminent. The four I am building patched versions of are File::Find::Rule, Socket6, Pod::Parser and String::Escape.

There are four others that have been patched for previous builds and which I have not looked into in any detail, except to note most do not have recent releases on CPAN. To see them, search for perl-modules-patched in the build config.

One point to note from the build config is that tests are skipped, in whole or in part, for many of the modules. Some of these are failures only on docker. Others I have not looked into but I do think it is reasonable to trust the decisions of the previous maintainers.

[PhilterPaper]

Good point about the dependencies chain -- I hadn't really considered that. If package X won't build, knocking dependent packages out of the build, and so on and so on, any idea how much of the package list would have to be removed in all?

I presume that you're keeping an eye out for problems in the tools themselves, rather than a fault in the end package itself? Compiler and linker bugs, library bugs, etc. Of course, getting such a bug fixed may set off a chain of failures in packages that were "fixed" to get around a tool bug in the first place!

[shawnlaffan]

I presume that you're keeping an eye out for problems in the tools themselves, rather than a fault in the end package itself?

I'm watching the winlibs repo if that's what you mean. I also have the occasional look at the gcc development but am happy leaving that to the experts. Edit - and winlibs has a new gcc-13.1 release which we need to try StrawberryPerl/spbuild#17

WRT the perl dependency chains, we build more than 230 packages from CPAN, plus their dependencies. The Task::Kensho checks in #103 have also been useful. This gives a pretty good sample.

The main breakages for SP 5.36.1 are now related to symlinks and directory separators. The latter is easily worked around by path normalisation. The latter might need more investigation.

There are other issues that might be related to memory alignment but which seem to be Strawberry specific. #102, #98, #26 (comment) (Edit - I meant don't seem to be Strawberry specific).

[shawnlaffan]

I've just uploaded a new dev release. https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5361_20230609_gcc13

The main change is that we now use release 5 of the winlibs gcc-13 stack. Compilation using its gcc is substantially faster than previous versions.

Unless some substantive new issues are identified I am thinking the next will be a proper release.

[shawnlaffan]

Perl 5.38.0 RC1 builds with the same test failures as 5.36.1 (#60), which is good news.

A portable version is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5380RC1_20230618_gcc13 if people want to test their favourite modules.

PDL and Inline::C work. I have not tested the Task::Kensho list yet. #103 (comment)

[shawnlaffan]

A build of Perl 5.38.0 RC2 is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5.38.0_RC2_20230626_gcc13

[shawnlaffan]

A version of Strawberry Perl 5.38.0 is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5380_20230704_gcc13

@genio - If no issues are reported then we could likely promote those to final for 5.38.0. A 5.36.1 can follow, and perhaps a 5.34 if there is an actual need and the patches we are applying to 5.36 apply cleanly.

[shawnlaffan]

Discussion on this topic has generalised to any release of Strawberry perl, and now that 5.36.1 and 5.38.0 are out the original intent (release a new version) would seem to be satisfied.

I've opened #124 for a 5.34 specific release.

I will close this issue in a few days unless there is a need to keep it open.

[PhilterPaper]

That's wonderful news! I look forward to both versions showing up on StrawberryPerl.com for download and trying out one or both (at least, 5.38). I might wait a few days to see if 5.38.1 comes out; usually .0 releases of any software need to be updated fairly quickly. Thanks for all the effort you've put in!

[shawnlaffan]

5.38.1 is likely quite some time off. For example, 5.36.1 was about 11 months after 5.36.0.

If you want to avoid .0 releases then Strawberry Perl 5.36.1.1 is available.

The website is a work in progress as we don't yet have access. In the meantime you can download from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/SP_5380_5361

[gregoa]

On Thu, 27 Jul 2023 16:01:42 -0700, shawnlaffan wrote: The website is a work in progress as we don't yet have access. In the meantime you can download from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/SP_5380_5361

That's great, thank you! Are there any plans to release a 32-bit version as well? Cheers, gregor

…

-- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `-

[genio]

We haven't made any plans to build and test a 32-bit version at this point in time. That being said, are there real world scenarios out there of 32-bit Windows in the wild? Note that this isn't intended to come off as a criticism or plain snide in any way, but a genuine question.

I mean, I know there are some "can't ever ever ever upgrade that windows 7 machine that's running my CNC machine" and stuff like that, but are those edge cases actually running/using Perl? Even if so, are those users brave enough to update anything on those machines?

I'm asking because it's quite a bit of work. We'd need to rebuild everything with a 32-bit compiler and go through all the testing and whatnot again. That's a lot of work if it's not something used but just kind of, sort of available on someone's testing rig.

[shawnlaffan]

A cut down 32-bit version (fewer external libs and perl modules) would likely be sufficient for most needs but it does still involve time on our part.

Discussion for a possible 32 bit release is under #96

[kenneth-olwing]

Let me first say:

===

Thanks to everyone involved for the incredible amount of work I suspect has been necessary in getting this on track again!

===

It is hard to overstate the importance of having a Strawberry dist for me to build on in terms of serving several hundred users in my organization, on both Windows and Linux where I try to make available Perl installations that are as "identical" as possible, from Perl base version to collection of modules. This in order to be as sure as possible that the multiplatform apps and tools we have can be expected to work "everywhere".

I have for the past month been busy getting my build system on track for creating the above with 5.38 as the base (building from source on Linux, but "cheating" on Windows by leveraging Strawberry, especially for getting the toolchain and many other libs sorted out). The last I did this with was 5.30 a few years back.

Mostly this has been working fine, but I note some modules that I know installed on 5.30 but not now. I'm not organized yet to track things down and properly compare whys and hows, but I expect to report a few later here for a first diagnosis.

Again, kudos!

ken1

[shawnlaffan]

You're welcome.

In terms of the modules, we do have several open issues for those we know to have errors. There is also the Task::Kensho list which is part of a large issue thread but the status of modules is in this comment: #103 (comment)

[gregoa]

On Thu, 27 Jul 2023 18:19:11 -0700, Chase Whitener wrote: We haven't made any plans to build and test a 32-bit version at this point in time. That being said, are there real world scenarios out there of 32-bit Windows in the wild?

I'm running a CPAN smoke tester on an old decommissioned laptop running Windows 10 32-bit. I guess the machine has a 64-bit CPU but originally it had a 32-bit Vista or something … and I haven't reinstalled since. Not a common scenario, and not a typical "production environment". And this also means that I wouldn't need a full-blown all-batteries-included Strawberry Perl but just a perl core Windows 32-bit build. Cheers, gregor

…

-- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `-

[sisyphus]

And this also means that I wouldn't need a full-blown all-batteries-included Strawberry Perl but just a perl core Windows
32-bit build

The Strawberry Perl developers (of which I am not one) might be too polite to point out that it's really quite easy and straightforward to build such a perl yourself.

To put it bluntly and simply, all you need is the gmake utility, the perl-5.38.0 source, and a 32-bit mingw-w64 port of gcc such as
CORRECTION: At this point I originally provided a link to a winlibs UCRT compiler - but for 5.38.0 we need an MSVCRT compiler.
Here is the corrected link
https://github.com/brechtsanders/winlibs_mingw/releases/download/13.1.0-16.0.5-11.0.0-msvcrt-r5/winlibs-i686-posix-dwarf-gcc-13.1.0-mingw-w64msvcrt-11.0.0-r5.zip (That's what SP will probably be using for any 32-bit builds they do release ... though, of course, that's not my call.)

UPDATE: One other thing I initially failed to mention in this post is that the changes made by Perl/perl5#21142 need to be made, (The diff is at Perl/perl5@89d9037.)
Having unzipped the source and applied PR21142, it's then just a matter of cd'ing to the win32 directory (in the cmd.exe shell) and running gmake args test followed by gmake args install

If you need assistance with specifying "args" correctly, just post back here (or even contact me directly) and help will be forthcoming.
We'd need to know:

1. do you want 32-bit or 64-bit integer (IV);
2. what nvtype do you want (double/long double/__float128);
3. into what location is perl to be installed;
4. what is the full path to your mingw-w64 "bin" folder.

Mind you, I'm hoping that SP does release a 32-bit perl - because perl should still support such architectures, and having 32-bit SP builds out and about will help keep 'em honest and on their toes.
AFAICT, the only point in having a 64-bit perl is if you need to have 64-bit pointers. (AFAIK, all scripts that I've ever written work just as well if pointers are 32 bits.)

UPDATE: The PR at Perl/perl5#21324 resolves the following issue with winlibs compilers. With those patches in place, winlibs compilers produce a 32-bit quadmath build of perl, with all tests passing - and the next paragraph can be ignored.

In the unlikely event that you want to build a 32-bit perl with nvtype of __float128, don't use the winlibs compilers (as they fail numerous threads-related tests).
For 32-bit quadmath builds, I've found that https://gcc-mcf.lhmouse.com/mingw-w64-gcc-mcf_20230419_13.0.1_x86-msvcrt_92844459e964d9e5547d3e3662188bda9b664e4e.7z builds a 32-bit quadmath perl-5.38.0 that passes all tests.

Disclaimer: I don't really want to make any claims about Windows Vista's capability to build perl. I think it should be ok, but I was unfortunate enough to once have a Vista system, and I well remember that it was a shitheap.

Cheers,
Rob

'

[shawnlaffan]

I'll close this issue now, as promised in #39 (comment)

If there are additional problems then please open a new issue. If there is a general question then the discussion forum can be used. https://github.com/StrawberryPerl/Perl-Dist-Strawberry/discussions