Skip to content
This repository has been archived by the owner on Jul 1, 2024. It is now read-only.

Parameters updates, default to Broadwell #1

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

michalhabera
Copy link

@michalhabera michalhabera commented Nov 25, 2020

Compiling with -march=native on skylake machines made it impossible to use on broadwell.
Some updates for latest JIT parameters as well.

env-fenics.sh Outdated
export DOLFINX_JIT_CACHE_DIR=${PREFIX}/cache
export DOLFINX_JIT_CFLAGS="-O3 -march=native"
mkdir -p ${HOME}/.config/dolfinx
echo '{"cache_dir":"'${PREFIX}/.cache/fenics'", "cffi_extra_compile_args": ["-O3", "-march=haswell"]}' > ${HOME}/.config/dolfinx/dolfinx_jit_parameters.json
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens here if this file is sourced on multiple nodes?

Copy link
Author

@michalhabera michalhabera Nov 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect all are touching it. I didn't see any lock or hang, this must be somehow handled in the way how nodes access the $SCRATCH $HOME. Feel free to add a condition and echo on first node only.

@michalhabera michalhabera changed the title Parameters updates, default to Haswell Parameters updates, default to Broadwell Nov 25, 2020
@michalhabera
Copy link
Author

michalhabera commented Nov 25, 2020

So I am getting Illegal instruction even for -march=broadwell. UNILU HPC docs says

Skylake processors (iris-[109-196] nodes) carry on 32 DP ops/cycle and support the new AVX-512 instruction set.
Broadwell processors (iris-[1-108] nodes) carry on 16 DP ops/cycle and support AVX2/FMA3.

which is not entirely true. E.g. iris-066 has Intel(R) Xeon(R) CPU E5-2680 for which I get -march=core-avx2. I will make this as default.

EDIT: Getting confused now as that doesn't work as well.

@michalhabera
Copy link
Author

Changed the approach now. Kept -march=native, but asking for Broadwell node in builder. Seems to work.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants