Add cudatoolkit #6240

xmnlab · 2018-07-05T19:31:03Z

Added recipe for cudatoolkit for linux.
next step would be to add recipes for win and osx platform

conda-forge-linter · 2018-07-05T19:31:12Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/cudatoolkit) and found it was in an excellent condition.

xmnlab · 2018-07-05T19:32:44Z

added @scopatz @andersy005 as recipe-maintainers.

xmnlab · 2018-07-05T19:56:39Z

it seems it is broken because some perl issue. I am taking a look on that.

scopatz · 2018-07-05T19:57:30Z

recipes/cudatoolkit/meta.yaml

+  recipe-maintainers:
+    - xmnlab
+    - scopatz
+    - andersy005


Thanks @xmnlab! @andersy005 are you OK with being a maintainer?

@scopatz, Yes!

scopatz · 2018-07-05T20:03:41Z

recipes/cudatoolkit/build.sh

@@ -0,0 +1,57 @@
+#!/bin/bash
+


Can you add a set -ex line here please, so that we can debug this? Thanks!

sure! thanks!

scopatz · 2018-07-05T20:04:33Z

Thanks for putting this in @xmnlab!

scopatz · 2018-07-05T20:36:44Z

it seems it is broken because some perl issue. I am taking a look on that.

This is probably because they used /usr/bin/perl in one of their scripts rather than /usr/bin/env perl

scopatz · 2018-07-05T21:01:43Z

Also, you may be able to use a yum_requirements.txt file to install perl into the system. Here is an example feedstock: https://github.com/conda-forge/imagepy-feedstock/blob/master/recipe/yum_requirements.txt

This is an alternative to doing a sed replace on whatever file is running the perl is.

xmnlab · 2018-07-05T23:29:52Z

thanks @scopatz !

I will try yum_requirements approach. normally I use sed to replace some code to work around some issue to build the package .. but in this case it seems that the runfile extract and run perl scripts ... I have no time to apply a sed command before run scrips are called.

scopatz · 2018-07-06T02:05:14Z

I have no time to apply a sed command before run scrips are called

Yeah, it would seem that you'd need to use --extract in order to use sed. yum_requirements.txt seems to be working though.

scopatz · 2018-07-06T02:07:05Z

Since this seems to be working for Linux, I'd likt o bring it to the attention of @conda-forge/staged-recipes @conda-forge/core for further discussion and review.

scopatz · 2018-07-06T15:58:04Z

Ok! This looks great @xmnlab! Did you want to try to tackle Mac and or Windows here in staged-recipes or instead should we try to add them in the feedstock?

isuruf · 2018-07-06T16:00:08Z

Let's keep the discussion at conda-forge/conda-forge.github.io#63 and not merge until it is resolved

scopatz · 2018-07-06T16:20:20Z

Let's keep the discussion at conda-forge/conda-forge.github.io#63 and not merge until it is resolved

Hmmm OK. The current discussion there doesn't seem super relate to the contents of this PR, which really just mirrors what is on defaults

xmnlab · 2018-07-06T17:52:37Z

@scopatz so would be better working on here to support osx and win until conda-forge/conda-forge.github.io#63 is resolved?

scopatz · 2018-07-06T17:55:52Z

Yes @xmnlab - I think it is better to work here for the time being.

xmnlab · 2018-07-06T19:53:58Z

basically there are 2 runfiles for windows:

https://developer.nvidia.com/compute/cuda/9.2/Prod/local_installers/cuda_9.2.88_win10 (10, Server 2016)
https://developer.nvidia.com/compute/cuda/9.2/Prod/local_installers/cuda_9.2.88_windows (8, 7, Server 2012 R2)

how to treat this case?

scopatz · 2018-07-06T20:01:51Z

I think you want to use https://developer.nvidia.com/compute/cuda/9.2/Prod/local_installers/cuda_9.2.88_win10 based on this page: https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal

The assumption should be to support Windows 10

xmnlab · 2018-07-06T21:56:43Z

thanks @scopatz !

scopatz · 2018-08-07T18:52:08Z

Ahh OK, that is pretty easy to solve, I think you need to add the following dependncies:

build:
  - {{ compiler('cxx') }}   # [osx]
host:
  - ncurses  # [osx]
run:
  - ncurses  # [osx]

scopatz · 2018-08-07T18:53:08Z

made a slight edit to add run dep.

xmnlab · 2018-08-07T18:55:44Z

great! thanks @scopatz ! I am going to add that!

xmnlab · 2018-08-08T23:33:35Z

[win] for some reason there is no "NVIDIA GPU Computing Toolkit" folder [1] .. I will mix installation and extraction approach.

[1] https://ci.appveyor.com/project/conda-forge/staged-recipes/build/1.0.26980#L755

xmnlab · 2018-08-09T01:22:40Z

@scopatz it seems it worked for windows now. maybe we could improve that in the future .. but at least it is working now.

xmnlab · 2018-08-09T23:28:51Z

@scopatz I changed the tests for osx to check the files there.

karl616 · 2018-08-10T12:44:56Z

Could it have something to do with DYLD_LIBRARY_PATH? Wild guess and I found this in the documentation:

Set up the required environment variables:

export PATH=/Developer/NVIDIA/CUDA-9.2/bin
${PATH:+:${PATH}}
export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-9.2/lib\
                         ${DYLD_LIBRARY_PATH:+:${DYLD_LIBRARY_PATH}

It is a wild guess, the path above is probably wrong... and I thought conda normally handled things like this on it's own.

Perhaps add an `echo ${DYLD_LIBRARY_PATH} to the build script... Hope someone with more osx experience know better...

xmnlab · 2018-08-10T19:41:26Z

thanks @karl616 for the feedback!

we probably should avoid to change DYLD_LIBRARY_PATH to avoid conflicts with conda. but it seems to be a good point and we should have this in mind. Thanks so much!

scopatz · 2018-08-10T19:44:11Z

This is now passing on all platforms - and is ready for review by @conda-forge/staged-recipes!

jjhelmus · 2018-08-21T22:34:01Z

This looks to be including both the dynamic and static libraries (at least on linux) which results in a really large package, ~770 MB vs 350 MB for just the dynamic libraries. Is there a reason for including the static libraries? Would it be reasonable to split these into their own package, cudatoolkit-static?

scopatz · 2018-08-22T16:12:13Z

This is a great suggestion @jjhelmus! This package should include only the dynamic libraries.

hadim · 2018-11-04T02:31:47Z

Any chance to get this merged?

hadim · 2018-11-04T02:37:34Z

Ok, I just realized there is some licensing issue at conda-forge/conda-forge.github.io#63.

leofang · 2019-10-30T07:28:24Z

just curious, is this PR superseded by something else or abandoned?

andersy005 · 2019-10-30T14:43:30Z

@leofang

just curious, is this PR superseded by something else or abandoned?

https://github.com/conda-forge/cudatoolkit-dev-feedstock

xmnlab · 2019-10-30T18:47:09Z

@scopatz should we keep it opened? or should we close it?

scopatz · 2019-10-30T21:54:06Z

It would be nice to have this still, so we don't rely on defaults, but we need permission from Nvidia.

stale · 2020-03-28T22:31:17Z

Hi friend!

We really, really, really appreciate that you have taken the time to make a PR on conda-forge/staged-recipes! conda-forge only exists because people like you donate their time to build and maintain conda recipes for use by the community.

In an effort to maintain this repository and increase the signal-to-noise for open PRs, the maintainers of staged-recipes close excessively old PRs after six months. This PR will remain open for another month, and then will be closed.

If you'd like to keep it open, please comment/push and we will be happy to oblige! Note that very old PRs will likely need to be rebased on master so that they can be rebuilt with the most recent CI scripts. If you have any trouble, or we missed reviewing this PR in the first place (sorry!), feel free to ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team.

Cheers and thank you for contributing to this community effort!

stale · 2020-04-27T22:39:04Z

Hi again! About a month ago, we commented on this PR saying it would be closed in another month if it was still inactive. It has been a month and so now it is being closed. Thank you so much for making it in the first place and contributing to the community project that is conda-forge. If you'd like to reopen this PR, please feel free to do so at any time!

Cheers and have a great day!

xmnlab added 2 commits July 5, 2018 15:24

Added cudatoolkit

fd7f2a9

Removed comments

6fbf017

Added perl as requirement

d943b69

scopatz reviewed Jul 5, 2018

View reviewed changes

xmnlab added 2 commits July 5, 2018 19:31

Added perl inside yum_requirements

b65f8fe

Rebuild travisci

21e21d2

scopatz mentioned this pull request Jul 6, 2018

GPU Support? conda-forge/conda-forge.github.io#63

Closed

Added osx support

a2c4bda

changed from sh to ./

1950231

xmnlab added 2 commits July 6, 2018 21:43

Added win script

7cf9cb4

Removed skip

ce08813

jchodera mentioned this pull request Jul 10, 2018

CUDA 9.2 has been released openmm/openmm#2084

Closed

xmnlab added 3 commits August 8, 2018 16:50

Added host section

41b6348

Get information from ProgramFiles var

d663452

Added quotes to paths

545856e

xmnlab added 2 commits August 8, 2018 20:04

Added dir /s

55c9706

Added extract files approach

f1d90ea

xmnlab added 2 commits August 9, 2018 18:20

Changed tests for osx

5a8ae4c

Add return True for osx test skip

62e1c92

This was referenced Nov 4, 2018

Package in conda-forge? pytorch/builder#190

Closed

Add pytorch recipe #6990

Closed

stale bot added the stale will be closed in 30 days label Mar 28, 2020

stale bot closed this Apr 27, 2020

Add cudatoolkit #6240

Add cudatoolkit #6240

Conversation

xmnlab commented Jul 5, 2018

conda-forge-linter commented Jul 5, 2018

xmnlab commented Jul 5, 2018

xmnlab commented Jul 5, 2018

scopatz Jul 5, 2018

Choose a reason for hiding this comment

andersy005 Jul 5, 2018

Choose a reason for hiding this comment

scopatz Jul 5, 2018 • edited Loading

Choose a reason for hiding this comment

xmnlab Jul 5, 2018

Choose a reason for hiding this comment

scopatz commented Jul 5, 2018

scopatz commented Jul 5, 2018

scopatz commented Jul 5, 2018

xmnlab commented Jul 5, 2018

scopatz commented Jul 6, 2018

scopatz commented Jul 6, 2018

scopatz commented Jul 6, 2018

isuruf commented Jul 6, 2018

scopatz commented Jul 6, 2018

xmnlab commented Jul 6, 2018

scopatz commented Jul 6, 2018

xmnlab commented Jul 6, 2018

scopatz commented Jul 6, 2018

xmnlab commented Jul 6, 2018

scopatz commented Aug 7, 2018 • edited Loading

scopatz commented Aug 7, 2018

xmnlab commented Aug 7, 2018

xmnlab commented Aug 8, 2018

xmnlab commented Aug 9, 2018

xmnlab commented Aug 9, 2018

karl616 commented Aug 10, 2018

xmnlab commented Aug 10, 2018

scopatz commented Aug 10, 2018

jjhelmus commented Aug 21, 2018 • edited Loading

scopatz commented Aug 22, 2018

hadim commented Nov 4, 2018

hadim commented Nov 4, 2018

leofang commented Oct 30, 2019

andersy005 commented Oct 30, 2019

xmnlab commented Oct 30, 2019

scopatz commented Oct 30, 2019

stale bot commented Mar 28, 2020

stale bot commented Apr 27, 2020

scopatz Jul 5, 2018 •

edited

Loading

scopatz commented Aug 7, 2018 •

edited

Loading

jjhelmus commented Aug 21, 2018 •

edited

Loading