Installer: 32 bit support

[bpintea]

Evaluate the effort and safety of a 32bit build of the installer.

Note: some preliminary findings out of a quick check:

• code-wise, a s/-windows-x86_64/-windows-x86 in installer/src/Installer/Program.cs:56 would be enough to get a 32bit .msi;
• the target installation should be changed from Program Files to Program Files (x86) [1];
• the installation eventually fails with error 13, potentially related to how the ODBC setup routines are called?
  

[1] The 32bit .msi find the 64bit driver and so it refuses to install -- not sure if the install path might be the only cause -- but this reveals that the name of the driver itself should bear the 32 tag (??).

The code itself can simply be made to work with 32-bit builders, resulting in a 32-bit installer (vs. having a 64-bit installer deliver both 64-bit and 32-bit drivers). Reasons for that could be:

• low effort of having a 32-bit installer (no significant changes required between the builds);
• low RM integration effort (besides the current build.bat setup 64 package we'd add build.bat setup 32 package);
• testing could be run similarly irrespective of underlying platform;
• the 32-bit installer could theoretically be used on 32-bit OSes (small chance of this occurrence, though);
• better user-experience: the user would need to make a choice right from the start and thus be made aware of the split (and of otherwise hidden potential issues later).

[codebrain]

I imagine that the error regarding the filename is that the total length of the filename needs to be 8 characters (excluding extension) - esodbc7u32.dll is 10 characters and it throws the conversion off.

Could we go with es7u32.dll?

[bpintea]

the total length of the filename needs to be 8 characters

We could rename it to es7u32.dll, but not "cleanly" and maybe it wouldn't be that intuitive either.
Ideally, the 32-bit driver would be named just like the 64-bit one, plus the 32 suffix.

In the list of installed 32-bit ODBC drivers on my machine I see most driver names being indeed 8 chars long, but not all, so I'm wondering where is the 8 characters naming requirement actually stemming from, is it at at an API level?

[droberts195]

It looks like the problem is that an MSI file entry that contains both short and long names, syzh8wlk.dll|esodbc7u32.dll has been passed directly to an ODBC API that requires a single file name. I'm sure there's a way to ask the MSI for just one of these file names rather than using the raw database entry.

[codebrain]

It looks like the problem is that an MSI file entry that contains both short and long names, syzh8wlk.dll|esodbc7u32.dll has been passed directly to an ODBC API that requires a single file name. I'm sure there's a way to ask the MSI for just one of these file names rather than using the raw database entry.

Yes, the simple solution was to keep the files under 8 characters, but I will look into how it is possible to overwrite the default filename generation.

[codebrain]

After a discussion with @russcam about 32bit and 64bit installers, we have arrived at the following.

We will need two installers - one for 32 bit and one for 64 bit. The MSIs have to be built to target a particular platform as noted here - http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/single-msi-for-32-bit-and-64-bit-td6358749.html#a6358844.

The question now is around what the contents of the 64 bit MSI should be?

Initially, we should simplify the build/development process by shipping the 32 bit driver in the 32 bit installer and the 64 bit driver in the 64 bit installer. There are a number of helpful tips/tricks on how to do this here: https://stackoverflow.com/questions/471424/wix-tricks-and-tips

Following this, we could consider shipping the 32 bit driver into the 64 bit installer (alongside the 64 bit driver) and make it an optional feature (perhaps default checkbox ticked). This could deliver a better user experience for those users on 64 bit platforms who still use 32 bit programs, notably the Microsoft Office Suite of applications. It does, however, introduce some questions around how to label each driver with independent product codes to align to each type of driver. As an example, what happens when a customer installs the 64 bit and 32 bit drivers via the 64 bit installer, and then attempts to install the 32 bit driver via the 32 bit installer - situations like this will require additional thought.

In the interim, suggest we notate the driver download page with a notice for 64 bit platform users to consider downloading and installing both MSIs, something like:

64 bit platform users, who are still using 32 bit applications (Microsoft Access, Microsoft Excel...), should consider downloading both the 64 bit installer *and* the 32 bit installer and install both in order to maximise compatibility across installed applications.

[sophiec20]

I think that the only use case that we should consider supporting would be users on 64 bit platforms who still use 32 bit programs. Was the above written with this in mind? Would be good to clarify reasoning as it may just be an option that is easier to implement. (If it is just an easier to implement choice, then we'll have a 32-bit MSI that sounds like it will be almost immediately deprecated).

Because a 64-bit MSI can include both 32-bit and 64-bit components, and it seems to be the ideal end goal to have a single MSI for both 32/64-bit driver software ... then it seems that the limiting factors to achieving this is time and complexity in WiX. This to me suggests we should not be adding this to the scope of the 6.6 beta.

I propose we focus on a solid and stable 64-bit odbc driver for this release, and we schedule time for 32-bit installer for a following one.

[bpintea]

Thanks, @codebrain, @sophiec20.

Just had a look at what other vendors do:

• MySQL, PostgreSQL, MariaDB, MS SQL, MongoDB, Redshift, all provide separate 32 and 64 bit installers.
• MySQL, PostgreSQL and Redshift don't install 32-bit drivers with their 64 bit installers; I didn't try the other vendors.

We could indeed check if it's feasible with current installer to provide both 32 and 64 bit drivers. I know for sure that with a separate/installer_external 32-bit program to do the 32-bit driver configuration past ODBC files copying, this would be feasible, but not positive if this cross configuration would work with the now used C# ODBC support.

My guess is that users typically have one application they would use with Elasticsearch and that few would have (1) more than one and (2) with different bitness.

So my recommendation would still remain to follow suite of the other vendors.

[sophiec20]

So my recommendation would still remain to follow suite of the other vendors.

Useful to know and I see the logic in following suite. Would be good not to try to do both.

I am still not convinced we have the time to do both installers for this current FF. I would prefer to focus on stability in 6.6 for 64-bit and to allow more time for 32-bit option.

[bpintea]

It looks like the problem is that an MSI file entry that contains both short and long names, syzh8wlk.dll|esodbc7u32.dll has been passed directly to an ODBC API that requires a single file name.

Indeed, this is the issue, as revealed by the MSI installation logs:
MSI (s) (FC:AC) [00:08:46:676]: Executing op: ODBCInstallDriver(DriverKey=Elasticsearch Driver,Component={EF6B65E0-20C3-43E3-A5E3-24E2F3DEEC22},Folder=C:\Program Files (x86)\Elastic\ODBCDriver\7.0.0\,Attribute_=Driver,Value_=jeyxpojs.dll|esodbc7u32.dll)

However, not sure if/how this could be fixed. The ODBCDriver table reference points to Microsoft's documentation suggesting that the only SFN names are accepted.

PostgreSQL's ODBC x86 driver's name has more than 8 chars and also installed with WiX-generated installer. However, its installer won't use the ODBCInstallDriver function, but write directly to the registry:
MSI (s) (54:E4) [01:15:41:920]: Executing op: RegAddValue(Name=Setup,Value=C:\Program Files (x86)\psqlODBC\1100\bin\psqlodbc35w.dll,).

Which might all suggest that there is indeed a limitation here.
Since the x64 and x86 driver files should never need to be placed in same folder (Program Files vs Program Files (x86)), it should be safe to drop for now the 32 suffix from the x86 build (35f4f74). With that, the resulting installer works fine.

[codebrain]

++ to using the short file name form. As you rightly point out the ODBCDriver entry in the MSI table does not support the long form | short form format:

The DLL file for drivers listed in the Driver column. The File_ column is an external key into the File table. The filename entered in the Filename column of that File table record must be in the short filename format. The SFN|LFN syntax cannot be used.

Dropping the suffix makes sense when the installation folders are different for 32 and 64 bit installs. I think continuing to use the ODBCDriver is preferable to "manually" creating the registry keys.

[bpintea]

Closed by #76 (#74).