Feature/9 add appveyor support

[shanmukhananda]

@paul-michalik appveyor support is added.
enable orb-salm2 project from your appveyor account
Modify readme.md with build status.
Appveyor > project > settings > badges > sample badge.

[paul-michalik]

I am not really happy about the solution with vcpkg binaries. But until we've found a better solution let's use this.

[ras0219-msft]

Hey, I noticed you're implementing something very similar to our built-in vcpkg export capability.

I think using vcpkg export --raw could help to simplify your process if we add the ability to specify the output directory; you could then just run something like:

vcpkg export <libraries> --raw --raw-target=target
rm target/installed/*/bin/*.pdb
7z target

Would that work for you?

[paul-michalik]

Hey, Robert nice to have you around...🙂

Yes, a more flexible export capabilities of vcpkg would be of great help! We have tried all export variants of vcpkg which unfortunately all turned out to be dysfunctional at various levels. The raw or zip export variants seem to be currently the most usable ones if the consumption from cmake controlled projects is intended. But still there are a lot of glitches which need to be sorted out, e.g. hard coded paths in *-config.cmake files or missing scripts from scripts/buildsystems/*.

We removed pdb's only for the appveyor scenario in order to save time and space.

[ras0219-msft]

dysfunctional at various levels

Please let us know the details (file issues, etc)! We build these features to be used and need to fix them if they don't work 🙂. If you prefer email, vcpkg contact lists our email address.

We're aware that there are occasional hardcoded paths in *-config.cmake -- these are definitely bugs and we fix them when we find them. We've made incremental improvements to vcpkg_fixup_cmake_targets() over the last several months which fixes all instances I'm currently aware of. If you have a specific package that's still misbehaving, I'd be happy to fix it! Also, as far as I can tell, your current approach should still be affected by that, right?

I'm not currently aware of any missing scripts in the export. If you file an issue, I'd be happy to fix it as well :)

more flexible export capabilities of vcpkg would be of great help

Would you mind going into some more specific details of what would help? Either here, email, or an issue on vcpkg would work great for us. It also really helps to know the context around it (e.g. removed pdbs to save space on appveyor).

[paul-michalik]

Great, yes, we'll provide a detailed error report and formulate the requirements towards vcpkg. You'll get a checklist with red and green hooks, I promise... 😉 We are also looking at possibilities to better integrate vcpkg with Conan.io and NuGet and also at extending vcpkg for cross compilation scenarios. Unfortunately this all costs time so many ideas exist only on paper.

Basically all export formats have to support the inverse of an export = an import. Currently this is only guaranteed for export -nuget AFAIK. We could add direct support for more package managers. i.e. export -conan.io or export -cpack or create a specification for a generic import command from raw archives. However this might be non-trivial considering all the possibilities regarding triplets, versions and other compatibility issues.

@shanmukhananda Can you please enlist the issues you ran into while trying to use vcpkg export for the appveyor scenario?

[shanmukhananda]

vcpkg is hard-coding the paths to .lib which are present under installed/ and packages/ folder.
This was seen using Pangolin

set_target_properties(pangolin PROPERTIES
  IMPORTED_IMPLIB_RELEASE "C:/Software/vcpkg/vcpkg/packages/pangolin_x86-windows-v141/lib/pangolin.lib"
  IMPORTED_LINK_INTERFACE_LIBRARIES_RELEASE "glu32;opengl32;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/glew32.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/avcodec.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/avformat.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/avutil.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/swscale.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/libpng16.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/zlib.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/jpeg.lib;C:/Software/vcpkg/vcpkg/installed/x86-windows-v141/lib/tiff.lib"
  IMPORTED_LOCATION_RELEASE "C:/Software/vcpkg/vcpkg/packages/pangolin_x86-windows-v141/bin/pangolin.dll"
  )

list(APPEND _IMPORT_CHECK_TARGETS pangolin )
list(APPEND _IMPORT_CHECK_FILES_FOR_pangolin "C:/Software/vcpkg/vcpkg/packages/pangolin_x86-windows-v141/lib/pangolin.lib" "C:/Software/vcpkg/vcpkg/packages/pangolin_x86-windows-v141/bin/pangolin.dll" )

So I had to keep both packages and installed folders together this was my initial problem.
Looks like this is a specific problem only to Pangolin, might be a bug in port

---

In general the problems of setting up windows build using vcpkg along with CI/CD set up like appveyor are,

• Time
  vcpkg takes lot of time to build depending on the number of packages, but in free build servers like appveyor we have only 60mins.
  Of-course we can cache the build artifacts in server, but many times even first setup itself will not succeed due to time limit.
  vcpkg could help solve this problem by providing options for release only build.
  Usually in build server its not required to build in debug mode. at-least true for this project.
• Size
  vckpg export consumes lot of memory, in appveyor cache limit is 1gb. vcpkg consumes lot of space by generating both debug and release binaries + pdb file
  vcpkg could provide options to disable pdb files

Note: The limits on appveyor are for free plans on open-source projects.

---

Must have features:

• controlling pdb generation
• build-type(debug/release) would really be helpful. Saves both time and space
• cotire support might help improving compile time
• installed folder is actually a collated copy of packages folder. Would be better if we can use links (something similar ln -s)

---

My wish-list 😃

• vcpkg should support importing pre-built binaries from cloud, standard packages provided by Microsoft/vcpkg should be compiled and uploaded to vcpkg cloud
• User could create, publish his/her own packages too. (user can always break packages, one can always refer to official microsoft/ binaries)
• This way user can avoid huge initial build times, user can easily switch between different package versions, and also save lot of space.
• Easy version switch, boost=1.60
• Some examples are Anaconda cloud(for python), homebrew or apt-get, which helps us set up development environment very fast.
• Once vcpkg supports binary installs from cloud, there should also be options where a package is installed from source and its dependencies as binary, install both package and dependencies from source, or both package and dependencies as binary. This is in similar lines with homebrew.

vcpkg install Microsoft/qt5=5.9.0<triplet> --config=release --build=0  # Imports official pre-built qt5 libs from Microsoft cloud
vcpkg install shanmukha/pangolin --config=debug --build=0              # Imports user built binaries from cloud
vcpkg install boost --config=release --build=1                         # Builds from source

[ras0219-msft]

a specific problem only to Pangolin, might be a bug in port

This is definitely a bug in our port, thanks for letting us know! I'll try to look into this tomorrow.

providing options for release only build.

Your wish is my command: microsoft/vcpkg#143 (comment). It is very recent, so some ports may encounter issues. Please report these as you find them (or better, send PRs) and we'll make everything work!

Essentially, just add set(VCPKG_BUILD_TYPE Release) to your triplet file.

we can cache the build artifacts in server

Yeah, caching binaries is definitely important! We intend for open source projects on time-limited machines to use our export mechanism to produce binaries ahead of time (say, on a lead developer's machine) which can be trivially uploaded to GitHub or NuGet.org. Then, the appveyor CI can easily download this archive containing exactly the needed bits. It might also be useful for users to have the exact "tested" libraries available if they want.

This helps with PDBs as well, because although they are large they compress really well.

We hope to write a blog post soon that walks through this workflow!

Would be better if we can use links (something similar ln -s)

We would like to use hardlinks here in the future!