Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to supply a custom SDK and source-built artifacts #100

Merged
merged 1 commit into from
Feb 4, 2020

Conversation

omajid
Copy link
Member

@omajid omajid commented Jan 30, 2020

This change tries to implement something analogous to dotnet/source-build#1455 for source-build-reference-packages.

With this change, I can build source-build-reference-packages completely offline on linux-x64 against a source-built SDK by running:

./build.sh --with-sdk /usr/lib64/dotnet \
           --with-packages /usr/lib64/dotnet/source-built-artifacts/*.tar.gz

This adds a dependency on the jq tool for editing json.

@omajid omajid requested a review from dseefeld January 30, 2020 16:24
@omajid
Copy link
Member Author

omajid commented Feb 3, 2020

cc @adaggarwal @crummel @dseefeld

@adaggarwal
Copy link
Member

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

build.sh Outdated
cp -a "${CUSTOM_SDK_DIR}" $__scriptpath/.dotnet
sdk_version="$(basename "$(find "${CUSTOM_SDK_DIR}/sdk" -maxdepth 1 -type d -printf "%p\n" | sort -rn | head -1)")"
echo "Found SDK version $sdk_version in ${CUSTOM_SDK_DIR}"
jq ".tools.dotnet = \"$sdk_version\"" global.json > global.json.tmp && mv global.json.tmp global.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the CI images that we use have jq installed on them. Can you use sed here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will give it a shot. But it wont be as reliable or robust as parsing json using something that understands json. Is a scripting language like perl or python available in the CI images?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arcade uses grep: https://github.com/dotnet/arcade/blob/3dec733f38264d7068ae75b3bb85e4c43979bcc7/eng/common/tools.sh#L76-L90

There's some discussion about changing to something else that actually understands json at dotnet/arcade#2510, but in core infra that we want to work when bringing up new platforms, depending on stuff we may not need to is a very hard sell.

Copy link
Contributor

@dseefeld dseefeld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @omajid. Once this is in, I will use it to remove the checked in binaries in the source-built directory and download the source-built-packages instead.

@adaggarwal
Copy link
Member

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

This change tries to implement something analogous to
dotnet/source-build#1455 for
source-build-reference-packages.

With this change, I can build source-build-reference-packages completely
offline on linux-x64 against a source-built SDK by running:

    ./build.sh --with-sdk /usr/lib64/dotnet \
               --with-packages /usr/lib64/dotnet/source-built-artifacts/*.tar.gz
@omajid omajid force-pushed the provide-local-bits branch from 14ce71b to 93e4ef4 Compare February 4, 2020 18:12
@dseefeld
Copy link
Contributor

dseefeld commented Feb 4, 2020

@omajid ready to merge?

@omajid
Copy link
Member Author

omajid commented Feb 4, 2020

@dseefeld Yes, looks good on my end.

@dseefeld dseefeld merged commit 125b8d6 into dotnet:master Feb 4, 2020
Copy link
Member

@dagood dagood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple general sh review notes, sorry I missed the merge though.

build.sh Show resolved Hide resolved
build.sh Show resolved Hide resolved
build.sh Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants