What is the difference between the release build workflow and solution build? (Windows VS)?

[Thell]

Hopefully this is a simple question...

If I download the pre-built release for Cbc-releases.2.10.8-w64-msvc17-md and check the libs they have exports (for example: lib.exe /exports libCbcSolver.lib) but if I build them myself using the v17 solution in VS2022 the produced libs do not have exports.

I'm attempting to get a Release-Parallel configuration built where the libs have exports in order to work with the good_lp crate in Rust on Windows. I have the local Release and Release-Parallel builds fine for cbc.exe and I can use the Cbc-releases.2.10.8-w64-msvc17-md download libs just fine in Rust but I can't get either of the builds to work in place of the downloaded libs.

Any ideas?

Also, the main readme states if there is a build we'd like to see in the releases to let you know on the mailing list. While this isn't a mailing list can this post suffice as a request for a Release-Parallel build in the releases?

[tkralphs]

The binary releases for Windows are being built using coinbrew in an MSys shell, but using the Visual Studio compiler rather than gcc. The autotools make it simple to swap out gcc and swap in cl and that takes care of the details of how to build libraries properly. I have to admit that I don't know that much about how the autotools does this or how/why they have exports, etc. I know a lot more about how builds work on pretty much every other platform but those with cl on Windows. It shouldn't be terribly difficult to replicate these builds on your local machine. The documentation is here.

As for requesting other builds, yes, this is the right place to do it (although this should probably be a Discussion, not an Issue). When you say Release-Parallel, do you mean built with multi-threading support? That is the one thing that may be a bit tricky. Do you need Visual Studio compatible binaries? Or just a parallel version of Cbc that can run under Windows? Do you need it built from a release? All of this is much easier with the code in the master branch.

[Thell]

Thanks for the quick response and thanks for the pointer to the Building on Windows. I guess it is time for me to install msys2 which I haven't used for years because of the pains of custom R/Rcpp environments on Windows. 😊

As for the Release-Parallel; yes, just built with multi-threading support as long as it runs under windows and the libs have their export sections. As for a particular release 🤷 I don't think it'd matter. Just something to take the pain out of setting up the usage of libCbc (and family).

To relate my recent experience, I knew from usage on linux years ago that I wanted to use cbc for a project I'm working on.

I use windows now and do my prototyping in Python, so I quickly installed or-tools and whipped up a model then setup multi-processing and saw my processing time decrease from ~55 minutes down to under 5 minutes for the solution but my post processing time was still horrendous (because of python) and since OR-Tools isn't compatible with pypy but there are rust crates for cbc I figured "Perfect, I love Rust!"... that was all on day 1... now I am on day 4 and still fighting getting a parallel version of the libs on Windows with the exports so as to make the rust linking happy. I honestly can't imagine people not using all the hardware capabilities they have but it is what it is and it could be sooo much easier if pre-built parallel builds got included in the releases.

Actually it makes me wonder that if they did get included would they end up being more popular than the single threaded ones. Anyhow, from what I understand, as long as pthreads is available and found during the build then using Release-Parallel would pick it up (?) so if that's doable that'd be fantastic.

[Thell]

The msys2 setup walk-through is pretty good. I still haven't succeeded in getting a successful parallel build though. I found this old issue #264 but still no love. Everything proceeded up until the gamsTest.exe.

./gamsTest.exe
make[1]: Leaving directory '/home/thell/coinbrew/build/Cbc/2.10.8/test'
make[1]: *** [Makefile:819: gamstests] Error 127
make: *** [Makefile:988: test] Error 2

My command line was

./coinbrew build Cbc --verbosity=4 --enable-cbc-parallel --with-pthreadsw32-incdir=~/pthreads/include --with-pthreadsw32-lib=~/pthreads/lib/pthreadVC2.lib --no-prompt --build=x86_64-w64-mingw32 ADD_CXXFLAGS=-DCBC_THREAD_SAFE

(with or without CBC_THREAD_SAFE and with with swapping pthreadVC2.lib with pthread.lib)

When I tried with pthread.lib (from my locally built pthread libs from ) instead of pthreadVC2.lib with --enable-msvc the build still failed even though that is the same build I used on my successful cbc build in Visual Studio (the one that works but doesn't export symbols).

Also, if I ran without any test and then attempted to link against the libs the unresolved external symbols all came from CbcThread.obj (like pthread_create) so I'm not quite sure what to make of this.

[jhmgoossens]

I've been using the VS2022 (v17) Visual Studio solutions for some time now (without coinbrew) to build the multi-threaded versions using ReleaseParallel. I typically use the https://github.com/GerHobbelt/pthread-win32 library for the native windows pthread library. I have to admit that the Cbc documentation in the README.md about building the parallel version in Visual Studio is quite old, since it refers to Visual Studio 2013.
The main reason the native windows pthread parallel build is not included (until now) in the binary packages is that these windows pthread libs have a different license (Apache License v2.0 for the library above)

@tkralphs Would we consider building and including these pthread-win32 libraries anyway and simply include something like a third-party-license.txt file? This is what I do for Sonnet in the GitHub Action.

The stable/2.10 branch doesn't have the v17 solutions since I only maintain these for master branch, so I presume you're using the master v17 solution with the stable/2.10 source code--which could well work.

As you can see in the Visual Studio 2022 Cbc project (and similar executables like the gamsTest project):

• The compiler expects include files for pthread-win32 are to be found in "..\..\..\..\..\pthreads" folder, so one level higher than the root folder with the Cbc, Clp, CoinUtils etc. projects.
• The linker expects the file pthread.lib in the Output Directory, so v17\x64\ReleaseParallel (x64) or v17\ReleaseParallel (x86).
  When building pthread-win32 manually for the matching Platform, simply copy the file to that folder.
• Any necessary defines like CBC_THREAD are already included in the proper places in the ReleaseParallel configurations.

And then the tests, including gamsTest, etc. pass.
Does this help?

But, of course, if you've switched from building using the VS solution to building using coinbrew, then this is irrelevant.

[Thell]

@jhmgoossens Would be be willing to check and see if your built cbc static libs are actually exporting functions? One of the things I noted between the VS solution and the coinbrew is that with the VS solution "/MD" is being used where as with coinbrew "/MT" is being used.

The 10 undefined ones I am getting with my coinbrew build are pthread functions. Can you check using a Developer Command Prompt?

> dumpbin /exports /symbols libCbc.lib | find "thread"
31D 00000000 UNDEF  notype ()    External     | pthread_create
31E 00000000 UNDEF  notype ()    External     | pthread_exit
31F 00000000 UNDEF  notype ()    External     | pthread_join
320 00000000 UNDEF  notype ()    External     | pthread_mutex_init
321 00000000 UNDEF  notype ()    External     | pthread_mutex_destroy
322 00000000 UNDEF  notype ()    External     | pthread_mutex_lock
323 00000000 UNDEF  notype ()    External     | pthread_mutex_unlock
324 00000000 UNDEF  notype ()    External     | pthread_cond_init
325 00000000 UNDEF  notype ()    External     | pthread_cond_timedwait
326 00000000 UNDEF  notype ()    External     | pthread_cond_signal
328 00000000 SECT39 notype ()    External     | ??0CbcSpecificThread@@QEAA@PEAV0@PEAPEAUpthread_mutex_t_@@@Z (public: __cdecl CbcSpecificThread::CbcSpecificThread(class CbcSpecificThread *,struct pthread_mutex_t_ * *))
595 00000000 SECT18D notype       Static      | $unwind$??0CbcSpecificThread@@QEAA@PEAV0@PEAPEAUpthread_mutex_t_@@@Z
598 00000000 SECT18E notype       Static      | $pdata$??0CbcSpecificThread@@QEAA@PEAV0@PEAPEAUpthread_mutex_t_@@@Z
7E8 00000000 SECT253 notype       External    | ??_C@_0M@MAHHBDIJ@Main?5thread@ (`string')
797 00000000 SECT248 notype       External    | ??_C@_0M@LNCAHNFJ@Use?5threads@ (`string')
79D 00000000 SECT24A notype       External    | ??_C@_0BO@FMKBDGOP@Using?5?$CFd?5threads?5in?5analysis?6@ (`string')
E1A 00000000 UNDEF  notype ()    External     | pthread_create
E1B 00000000 UNDEF  notype ()    External     | pthread_join
250E 00000000 SECTB45 notype       External    | ??_C@_0FE@IJECIIBL@?$CK?$CK?$CK?5threaded?5error?5?9?5numberRows@ (`string')

With my VS build they aren't any symbols being exported.

[Thell]

While I still do not know why I couldn't get a "/MT" build to completely work with rust at least I now can muddle my way through building a workable parallel cbc.exe using either coinbrew or Visual Studio. While studying the differences in the build and link commands I also took a closer look at OR-Tools cmake setup since it is a working parallel solve.exe with a solver of cbc and shows that it is exporting the functions and I learned that the reason I was getting unresolved symbols from it was because it has a dependency on 'zlib.lib' as well. Once I added that the Rust coin_cbc crate worked in parallel. Woot! 🥳

I don't think I'd be of much help in trying to make a Release-Parallel release a reality other than to point out that OR-Tools has it working reliably with cmake so maybe that logic would be transferable to coinbrew and the github-workflow. Also, I can point out that having a parallel build being available could have a notable downstream benefit. For instance, currently the Rust coin_cbc crate has a global lock on calls to the cbc solver because of the possibility of memory corruption from calling the solver in parallel using the default releases of Cbc (KardinalAI/coin_cbc#9) so even though I now have a working parallel setup it is still 2x slower than using Python to call the same Cbc libraries. Perhaps once a parallel build is part of the normal release cycle downstream projects wouldn't feel the need to use work-arounds/restrictions to maintain reliability.

[tkralphs]

Yeah, I'm aware of these issues of thread safety and the workaround in the rust interface. As I indicated, if you don't really care about having an official release, then you are probably better off with master. The code there is undergoing some major refactoring, with one of the goals being to improve thread safety. The refactoring has been a lot of work and is not really done yet, but the changes related to thread safety are done and what's there should work well for your purposes (I think). I'm working to get this out into a release, but my bandwidth has been severely limited and no one is stepping up to help. What's needed doesn't require a deep knowledge of Cbc's internals and would improve things immensely for exactly this use case.

I'll try to have a look at or-tools, but the issues around thread safety are pretty deeply entrenched in the current release version, so I can't really imagine what they're doing unless they are working with master.

By the way, parallel MILP is highly non-deterministic so I would be careful about drawing conclusions regarding performance improvement until you've done a pretty thorough benchmarking. It is difficult to get reliable speedups and it is entirely possible to see 10x speedup on one instance, then an actual slowdown on another. On the whole, even with a pretty beefy machine, you probably can't expect more than a modest speedup with parallelization. Of course, if you're solving a set of instances and these can be solved in parallel, you'll probably do much better.

[jhmgoossens]

Would be be willing to check and see if your built cbc static libs are actually exporting functions?
With my VS build they aren't any symbols being exported.

I have to admit I had to google what it means to 'export functions'. I tried dumpbin. No functions are exported by the libs built in VS. Is this relevant when we're creating Static Libs? (or "so what"?) Within the VS2022 solution, all projects (other than executables) build static libraries that will be linked as the projects Reference each other, and then It Just Works..

One of the things I noted between the VS solution and the coinbrew is that with the VS solution "/MD" is being used where as with coinbrew "/MT" is being used.

The pthread library that I use in VS (see above) also uses "/MD", just like the VS Cbc projects. And all modules passed to a given invocation of the linker must have been compiled with the same run-time library compiler option (/MD, /MT, /LD). For Visual Studio projects, using "/MD" is default. That said, if the coinbrew build consistently used "/MT" in all projects, I don't see why that wouldn't work.

Many thanks to @tkralphs for all the hard work on refactoring Cbc master, and apologies for not helping.

[tkralphs]

I never managed to fully understand the difference between /MT and /MD, despite having looked at Microsoft's documentation many times. I believe that in the current releases, which use the old BuildTools, the default is /MD, not /MT, as per here, and this is why the tarballs containing the artifacts end in -md. But perhaps there is something I've overlooked, since I don't use the Windows artifacts at all. If I'm not mistaken, /MD is also what one needs to produce C extensions for Python. But in any case, it is possible with coinbrew to specify what run-time you want. If you want /MT, then just add --enable-msvc=MT.

The master version does all of this "the right way" by declaring which functions should be exported with all the declspec stuff (take a look here). But I thought that really only applied to DLLs.

[Thell]

--enable-msvc=MT

Ahhh, I didn't see that option and will give it a try.

[tkralphs]

--enable-msvc=MT

Ahhh, I didn't see that option and will give it a try.

Sorry, it may not be well-documented :(.

[wozhendeshuai]

Why is the CBC built using the master branch in mid-September 2024 6 times slower than the 2.10.11 release version to solve MIP problems with the same scale of millions? @tkralphs

[tkralphs]

Six times slower on what instance(s)? Performance on individual instances can vary dramatically, even with seemingly inconsequential changes to the input or parameters (see Section 5 here). You can only expect relative stability on large sets of test instances.

But with that said, part of the the reason, master has not been released is because there are some regressions and the parameters need to be better tuned. The master version is not very different from what is in 2.10 overall, but some additional methodologies that were added and are important for some classes of problems probably slow down performance. There should be parameters settings for master that would result in performance similar to 2.10, but @h-g-s is probably more knowledgeable on this.

Since this isn't really related to the original issue (which is stale anyway, I'm going to close this issue. If you don't mind, you can continue discussion in another issue if needed.