Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improved Docker & NGC #2557

Merged
merged 3 commits into from
Oct 18, 2018
Merged

Improved Docker & NGC #2557

merged 3 commits into from
Oct 18, 2018

Conversation

ax3l
Copy link
Member

@ax3l ax3l commented Mar 22, 2018 •
edited
Loading

This is a WIP PR documenting the changes to make PIConGPU

latest change:

  • make MPI CUDA aware

As before, the corresponding spack recipe is uploaded here.

  • future: add to install docs as possible deployment site

@ax3l ax3l added documentation regarding documentation or wiki discussions component: third party third party libraries that are shipped and/or linked labels Mar 22, 2018
@ax3l ax3l added this to the 0.4.0 / 1.0.0: Next Stable milestone Mar 22, 2018
@ax3l ax3l force-pushed the topic-NGCandGTC18 branch from 97fa31d to 7457a5c Compare April 5, 2018 09:52
@ax3l ax3l force-pushed the topic-NGCandGTC18 branch 3 times, most recently from 813671b to f4d7b40 Compare May 24, 2018 15:53
ax3l
ax3l commented Jun 5, 2018
View reviewed changes
share/picongpu/dockerfiles/ubuntu-1604/Dockerfile

COPY start_lwfa.sh /usr/bin/lwfa
COPY start_lwfa_4.sh /usr/bin/lwfa4
COPY start_lwfa_8.sh /usr/bin/lwfa8
COPY start_khi_4.sh /usr/bin/bench4

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

share/picongpu/examples/FoilLCT/etc/picongpu/4_isaac.cfg
## Section: Optional Variables ##
#################################

TBG_isaac="--isaac.period 1 --isaac.name foil --isaac.url hypnos5 --isaac.quality 99"

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

@ax3l ax3l force-pushed the topic-NGCandGTC18 branch 2 times, most recently from 736ec5d to c0913c1 Compare June 7, 2018 09:44
ax3l
ax3l commented Jun 7, 2018
View reviewed changes
share/picongpu/dockerfiles/ubuntu-1604/modules.yaml
^openmpi::
environment:
set:
OMPI_MCA_btl: '^vader'

This comment was marked as off-topic.

Sign in to view

@ax3l ax3l force-pushed the topic-NGCandGTC18 branch from c0913c1 to d0f79e7 Compare June 18, 2018 07:35
ax3l
ax3l commented Jun 19, 2018
View reviewed changes
share/picongpu/dockerfiles/ubuntu-1604/start_foil_8.sh Outdated
-s "bash -l" \
-c etc/picongpu/8.cfg \
-t etc/picongpu/bash/mpirun.tpl \
/tmp/foil8_001
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possible improvement: add an optional parameter to the scripts to control the output path

@ax3l ax3l mentioned this pull request Jun 26, 2018
Open
@ax3l ax3l mentioned this pull request Aug 11, 2018
Merged
4 tasks
@ax3l ax3l force-pushed the topic-NGCandGTC18 branch 3 times, most recently from d088c03 to d848852 Compare October 17, 2018 15:03
@ax3l ax3l changed the title [WIP] NGC NGC Oct 17, 2018
@ax3l ax3l force-pushed the topic-NGCandGTC18 branch from d848852 to 1f8d9a9 Compare October 17, 2018 15:05
@ax3l ax3l requested a review from psychocoderHPC October 17, 2018 15:05
@ax3l ax3l changed the title NGC Improved Docker & NGC Oct 17, 2018
@ax3l ax3l mentioned this pull request Oct 17, 2018
Merged
21 tasks
psychocoderHPC
psychocoderHPC requested changes Oct 18, 2018
View reviewed changes
share/picongpu/examples/LaserWakefield/etc/picongpu/8.cfg Outdated
TBG_steps="4000"

# leave TBG_movingWindow empty to disable moving window
TBG_movingWindow="-m"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please re-add the sliding window. Maybe you should add again 4 GPUs in Y direction to have a larger window.

share/picongpu/examples/LaserWakefield/etc/picongpu/4.cfg
TBG_devices_z=1

TBG_gridSize="128 2048 128"
TBG_steps="2048"

This comment was marked as resolved.

Sign in to view

ax3l added 3 commits October 18, 2018 14:08
@ax3l
Examples
803b40d 
Live Visualization:
- 4 & 8 GPU LWFA (e- acc.)
- 4 & 8 GPU Foil (p+ acc.)

Benchmark:
- 1, 4 & 8 GPU KHI
@ax3l
TBG: bash tpl - less verbose
6914bbb 
remove mapping display and tag output from startup.
make output less verbose
@ax3l
Docker: More Examples, NGC & Benchmark
b35e608 
OpenMPI: Use 2.1.2

Multi-Rank MPI support and thus multi-GPU support requires for
in-node communication a BTL such as "sm" which was replaced with
"vader" in OpenMPI 3.0.0.

"vader" requires CMA support from the kernel which seems to be
something that is not working in a docker container. We therefore
switch back to an older release (pre-3.0) of OpenMPI that still has
the (slower) "sm" transport.

Build for SM_70 (V100)

Docker Readme: MPI BTL shared Mem

Overwrite exising output with tbg -f

Configurable output directory and print to user.

Use non-ISAAC examples for now.
Add _live LWFA examples for later testing of ISAAC.
@ax3l ax3l force-pushed the topic-NGCandGTC18 branch from 1f8d9a9 to b35e608 Compare October 18, 2018 12:08
@psychocoderHPC psychocoderHPC merged commit b218848 into ComputationalRadiationPhysics:dev Oct 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@theZiz theZiz theZiz left review comments

@psychocoderHPC psychocoderHPC psychocoderHPC approved these changes

Assignees

@psychocoderHPC psychocoderHPC

Labels
component: third party third party libraries that are shipped and/or linked documentation regarding documentation or wiki discussions
Projects
None yet
Milestone
0.4.0: CPU Support, Filter & Merging
Development

Successfully merging this pull request may close these issues.
3 participants
@ax3l @theZiz @psychocoderHPC