Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up cputopology initialization global communication pattern #3449

Merged
merged 4 commits into from
Aug 16, 2021

Conversation

evan-charmworks
Copy link
Contributor

@evan-charmworks evan-charmworks commented Aug 12, 2021

  • Replaces error-prone locking and improper scheduler driving with well-tested pattern from Isomalloc-Sync
  • Replaces bespoke "empty reduction" with CmiBarrier
  • Replaces Broadcast with NodeBroadcast and moves send outside of handler
  • Replaces unnecessary privatization of Converse handler index variables with CmiAssignOnce

@evan-charmworks
Copy link
Contributor Author

@trquinn Does this resolve the hang on init from #2850?

@evan-charmworks
Copy link
Contributor Author

evan-charmworks commented Aug 12, 2021

Spurious and unrelated:

/Applications/Xcode_12.4.app/Contents/Developer/usr/bin/make -C method_templates smptest OPTS='-g -build-shared -optimize -production' TESTOPTS='++local +setcpuaffinity'
########################################################################################
../../../bin/testrun   +p4 ./testTemplateEP  ++ppn 2 ++local +setcpuaffinity
Charmrun> scalable start enabled. 
Charmrun> started all node programs in 0.017 seconds.
Charm++> Running in SMP mode: 2 processes, 2 worker threads (PEs) + 1 comm threads per process, 4 PEs total
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: 264b00b
Isomalloc> Synchronized global address space.
Charm++> scheduler running in netpoll mode.
CharmLB> Load balancer assumes all CPUs are same.
sched_setaffinity() is not supported, +setcpuaffinity disabled.
Charm++> Running on 1 hosts (1 sockets x 3 cores x 1 PUs = 3-way SMP)
Charmrun> error on request socket to node 0 '127.0.0.1'--
Socket closed before recv.

real	10m0.184s
user	4m25.177s
sys	4m39.497s
make[3]: *** [smptest] Error 1
make[2]: *** [smptest-method_templates] Error 2
make[1]: *** [test] Error 2
make: *** [test-charm++] Error 2
==> Error: ProcessError: Command exited with status 2:
    'make' '-C' '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/runner/spack-stage/spack-stage-charmpp-main-hb6ii4az3aa2ry6fozgur65x3u6wplpw/spack-src/tests' 'test' 'TESTOPTS=++local +setcpuaffinity'

4 errors found in build log:
     22985    Charmrun> error on request socket to node 0 '127.0.0.1'--
     22986    Socket closed before recv.
     22987    
     22988    real	10m0.184s
     22989    user	4m25.177s
     22990    sys	4m39.497s
  >> 22991    make[3]: *** [smptest] Error 1
  >> 22992    make[2]: *** [smptest-method_templates] Error 2
  >> 22993    make[1]: *** [test] Error 2
  >> 22994    make: *** [test-charm++] Error 2

See build log for details:
  /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/runner/spack-stage/spack-stage-charmpp-main-hb6ii4az3aa2ry6fozgur65x3u6wplpw/spack-build-out.txt

@evan-charmworks evan-charmworks enabled auto-merge (squash) August 12, 2021 23:08
ericjbohm
ericjbohm previously approved these changes Aug 13, 2021
Copy link
Contributor

@ericjbohm ericjbohm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit comment should note motive for switching from locked Cpv to atomic.

@evan-charmworks evan-charmworks enabled auto-merge (squash) August 13, 2021 19:44
@evan-charmworks
Copy link
Contributor Author

Commit comment should note motive for switching from locked Cpv to atomic.

Updated.

@evan-charmworks
Copy link
Contributor Author

Needs one more review.

@evan-charmworks evan-charmworks merged commit 018b5ce into main Aug 16, 2021
@evan-charmworks evan-charmworks deleted the evan/cputopology-cleanup-comm branch August 16, 2021 18:53
evan-charmworks added a commit that referenced this pull request Aug 19, 2021
- Replaces error-prone locking and improper scheduler driving with well-tested pattern from Isomalloc-Sync
- Replaces bespoke "empty reduction" with CmiBarrier
- Replaces Broadcast with NodeBroadcast and moves send outside of handler
- Replaces unnecessary privatization of Converse handler index variables with CmiAssignOnce
evan-charmworks added a commit that referenced this pull request Aug 19, 2021
- Replaces error-prone locking and improper scheduler driving with well-tested pattern from Isomalloc-Sync
- Replaces bespoke "empty reduction" with CmiBarrier
- Replaces Broadcast with NodeBroadcast and moves send outside of handler
- Replaces unnecessary privatization of Converse handler index variables with CmiAssignOnce
stwhite91 pushed a commit that referenced this pull request May 11, 2023
- Replaces error-prone locking and improper scheduler driving with well-tested pattern from Isomalloc-Sync
- Replaces bespoke "empty reduction" with CmiBarrier
- Replaces Broadcast with NodeBroadcast and moves send outside of handler
- Replaces unnecessary privatization of Converse handler index variables with CmiAssignOnce
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants