Roadmap for 2.6.0 release

[wbhart]

• Check the APRCL implementation is correct
• Check new Spinx documentation
• Fix test failures on Windows
• Univariate polynomial factorisation fails for large coefficients (see also fmpz_poly_CLD_bound still buggy #576)
• Elliptic curve method needs extensive testing and bug fixes
• Quadratic sieve needs to be made faster in the 53-128 bit range
• Quadratic sieve has segfaults
• OpenMP code needs replacing with pthreads code using new threadpool
• Implement smooth integer factoring and tuning
• There are numerous minor build issues and bugs in the tickets
• Inconsistent naming (see fmpz_poly_is_x vs fq_nmod_poly_is_gen #578)
• FFT precomputation
• Speed up inverse of matrices over Z

The following are optional for a release, but highly desirable:

• Deal with non-square matrices in everything that uses LU factorisation, esp. solve
• Speed up more linear algebra functions up to point everything can be done using slongs
• Coefficient splitting for polys over Z with very large coeffs, for use with FFT
• Fast polynomial division using FFT precomputation and other tricks
• Memory manager needs a threaded pooled allocator
• Quadratic sieve I/O needs parallelising (better still, Bill Allombert's experiments clearly indicate it should be done entirely in memory with no file I/O)
• Improve Zassenhaus factorisation (pruning algorithms)
• Speed up charpoly over Z when dimension > 200
• Speed up determinant over Z
• Reduce as much linear algebra as possible to fast matrix multiplication
• Small primes FFT
• Use BLAS in nmod_mat
• flesh out fmpz_mod_mat module

[isuruf]

FYI, here are the windows failures,

• MinGW-w64 64-bit

fmpq/test/t-cmp_ui

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
FAIL
x = 33477875922062297407124210257081144831707807591277002752000/118937304180895415593582671402914857770571093
y = 551784335844911
cmp(x,y) = -1, cmp(X,Y) = 1

• MinGW-w64 32-bit

Tests pass

• MSVC 32-bit

fmpz_mpoly/test/t-divides_array .......................................Exit code 0xc0000409
***Exception:   1.15 sec
divides_array....

fmpz/test/t-is_perfect_power ..........................................***Exception: SegFault  1.15 sec
is_perfect_power....

• MSVC 64-bit
  fmpz_poly_factor/test/t-factor hangs for a long time even with 1 testcase

1047 - nmod_poly_mat-test-t-concat_vertical (SEGFAULT)
	1690 - fmpz_mpoly-test-t-divides_array (Exit code 0xc0000409
)
	1693 - fmpz_mpoly-test-t-divrem_array (Exit code 0xc0000409
)
	1694 - fmpz_mpoly-test-t-divrem_ideal_monagan_pearce (Exit code 0xc0000409
)

[wbhart]

Thanks. This will be very helpful, though I won't be able to do anything about it until I get back from holidays in a little over a month.

[wbhart]

@isuruf Do you agree the Windows problems are now resolved? If so, I will tick that off the roadmap.

[isuruf]

@wbhart, no, the failing tests are simply disabled in appveyor. They still fail last I checked

[wbhart]

@isuruf Ah, I was not aware of that. Glad I checked!

[slel]

Will #585 (flush standard output in flint_memory_error) be in FLINT 2.6?

[wbhart]

Someone suggested we should just write to stderr instead. So I'm not sure at this point.

[wbhart]

@slel yes.

[wbhart]

@isuruf I have fixed the cmp_ui test. However, I am getting nowhere with t-factor. The appveyor --output-on-failure is broken. It doesn't display all lines of output when a test times out. I tried putting printf's in to see where the hang is, and if I remove earlier printf's some later printf's begin to show up. If I remove a couple more earlier printf's no printf's at all show up in the appveyor output!

Do you know how to make all output show up? I can't find any documentation of the ctest option at all.

[wbhart]

It's ok, I figured it out. ctest is the cmake command, not an appveyor command, and --verbose is the right option. However, no output is showing up for that test. Not even a print of the test iteration shows up.

I can't imagine what is causing this.

[isuruf]

@wbhart, maybe flush the stdout?

[isuruf]

I see that it works now. If you would like to, you can login to an appveyor node for an hour (including the time it takes to run the CI) using RDP.

[wbhart]

Yeah that did it. I now think the hang is in fmpz_poly_CLD_bound. We had a hang here on Linux which was fixed. I'm now wondering if MPIR-2.7.0 might be the issue. Those functions rely on the new versions in MPIR-3.0.0 I think.

[wbhart]

On Windows 64 I mean. 2.7.2 is fine on Linux, obviously. On the other hand the gcc version passes. I don't see why that would be as a long should still be 32 bits there I think.

[isuruf]

3.0.0 is problematic because of the issue https://groups.google.com/forum/#!search/Flint$2C$20MPIR$20and$20Windows$20compatibility/mpir-devel/htQR38wtrxQ/-BC1LeAmAQAJ

[wbhart]

Can also reduce the timeout to 1min.

[wbhart]

I did fix this issue. Unfortunately I don't remember where I fixed it, whether in Flint or MPIR. If the latter then only a build of mpir from master is going to work. I guess we'll see in a few minutes.

[wbhart]

By the way, Brian Gladman reports that on his box MSVC passes the t-factor test on Win64, but a different test fails (not the one I fixed). He uses his own visual solution files.

[wbhart]

Apparently @thofma has been using GMP-6.1.2 and MPFR-4.0.2 with Flint on Windows 64. We could try those instead.

[wbhart]

By the way, I disabled the login again. I couldn't find the source code once in there, and don't know how to run the build anyway, since I'm not familiar with this development platform. Also, just doing dir caused it to crash and print weird things, so it's not really stable enough to use.

[isuruf]

Here are the instructions, if you are interested.
In a command prompt,

call C:\\Miniconda36-x64\Scripts\activate.bat
call "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Auxiliary\\Build\\vcvarsall.bat" x64
cd C:\projects\flint2\build
cmake --build . --target fmpz_poly_factor-test-t-factor -- -j3
bin\fmpz_poly_factor-test-t-factor.exe

[wbhart]

Thanks, I might actually give it a go. It's just hanging in random places which makes no sense at all.

I'm beginning to wonder if there is interference from a virus scanner, though I still think that's a less likely possibility.

[isuruf]

I'm beginning to wonder if there is interference from a virus scanner, though I still think that's a less likely possibility.

Just to make sure, I ran the tests in Azure Pipeline CI. Same issue.

[wbhart]

I randomly found the bug while inserting traces. It was actually a pretty serious bug, but I've no idea why it was only showing up on MSVC. Anyhow, that has probably saved a few days of very painful debugging that probably wouldn't have gotten anywhere.

[wbhart]

I'll close this now as all the ideas have been put into separate tickets, and we have the Milestone for keeping track of progress in the actual release.