Code

Topic/monitoring#3109

Code

Merged

bosilca merged 83 commits intoopen-mpi:masteropen-mpi/ompi:masterfrom bosilca:topic/monitoringbosilca/ompi:topic/monitoringCopy head branch name to clipboardJun 26, 2017

+8,216-684

Commits

Commits on Jun 26, 2017

• Add a monitoring PML. This PML track all data exchanges by the processes

  Show description for 5e5f269
  bosilca

  committed
  5e5f269

• Fix a convertion problem and add a comment about the lack of component

  Show description for 673c766
  bosilca

  committed
  673c766

• add ability to querry pml monitorinting results with MPI Tools interface

  Show description for a4a9b39
  

  gpapaure

  authored and

  bosilca

  committed
  a4a9b39

• Allow the pvar to be written by invoking the associated callback.

  Show description for 7316eb2
  bosilca

  committed
  7316eb2

• Various fixes for the monitoring.

  Show description for d50394b
  bosilca

  committed
  d50394b

• Cleanup for the monitoring module.

  Show description for 617e3c5
  bosilca

  committed
  617e3c5

• Adding documentation about how to use pml_monitoring component.

  Show description for 8b0af75
  

  clementFoyer

  authored and

  bosilca

  committed
  8b0af75

• Change rank into MPI_COMM_WORLD and size(MPI_COMM_WORLD) to global variables in pml_monitoring.c.

  Show description for c3dbc7e
  

  clementFoyer

  authored and

  bosilca

  committed
  c3dbc7e

• Improve monitoring support (including integration with MPI_T)

  Show description for 3689a01
  

  clementFoyer

  authored and

  bosilca

  committed
  3689a01

• Add overhead benchmark, with script to use data and create graphs out of the results

  Show description for 462e59a
  

  clementFoyer

  authored and

  bosilca

  committed
  462e59a

• Fix segfault error at end when not loading pml

  Show description for 30ac0a3
  

  clementFoyer

  authored and

  bosilca

  committed
  30ac0a3

• Start create common monitoring module. Factorise version numbering

  Show description for c9c9739
  

  clementFoyer

  authored and

  bosilca

  committed
  c9c9739

• Fix microbenchmarks script

  Show description for e1b35a6
  

  clementFoyer

  authored and

  bosilca

  committed
  e1b35a6

• Resolve brutal segfault when double freeing filename

  Show description for dbbd563
  

  clementFoyer

  authored and

  bosilca

  committed
  dbbd563

• Improve readability of code

  Show description for 0cfa1d3
  

  clementFoyer

  authored and

  bosilca

  committed
  0cfa1d3

• Add osc monitoring component

  Show description for 5d2725f
  

  clementFoyer

  authored and

  bosilca

  committed
  5d2725f

• Add error checking if running out of memory in osc_monitoring

  Show description for 9894ba6
  

  clementFoyer

  authored and

  bosilca

  committed
  9894ba6

• Moving to ompi/mca/common the proper parts of the monitoring system

  Show description for ef0ccca
  

  clementFoyer

  authored and

  bosilca

  committed
  ef0ccca

• Fix linking library with mca/common

  Show description for e63532a
  

  clementFoyer

  authored and

  bosilca

  committed
  e63532a

• Add calls to record monitored data from osc. Use common function to translate ranks.

  Show description for 886b180
  

  clementFoyer

  authored and

  bosilca

  committed
  886b180

• Fix test_overhead benchmark script distribution

  Show description for 8ca765e
  

  clementFoyer

  authored and

  bosilca

  committed
  8ca765e

• Add passive operations in monitoring_test

  Show description for 8458be7
  

  clementFoyer

  authored and

  bosilca

  committed
  8458be7

• Fix from rank calculation. Add more detailed error messages

  Show description for 7ad29cf
  

  clementFoyer

  authored and

  bosilca

  committed
  7ad29cf

• Fix alignments. Fix common_monitoring_get_world_rank function. Remove useless trailing new lines

  Show description for 9ec0be4
  

  clementFoyer

  authored and

  bosilca

  committed
  9ec0be4

• Fix osc_monitoring mget_message_count function call

  Show description for ce6a7cb
  

  clementFoyer

  authored and

  bosilca

  committed
  ce6a7cb

• Change common_monitoring function names to respect the naming convention. Move to common_finalize the common parts of finalization. Add some comments.

  Show description for 3ff6e7c
  

  clementFoyer

  authored and

  bosilca

  committed
  3ff6e7c

• Add monitoring common output system

  Show description for ec6c426
  

  clementFoyer

  authored and

  bosilca

  committed
  ec6c426

• Add error message when trying to flush to a file, and open fails. Remove erroneous info message when flushing wereas the monitoring is already disabled.

  Show description for 88835f6
  

  clementFoyer

  authored and

  bosilca

  committed
  88835f6

• Consistent output file name (with and without MPI_T).

  Show description for 7ddc0bc
  

  clementFoyer

  authored and

  bosilca

  committed
  7ddc0bc

• Always output to a file when flushing at pvar_stop(flush).

  Show description for edc6863
  

  clementFoyer

  authored and

  bosilca

  committed
  edc6863

• Update the monitoring documentation.

  Show description for f944638
  

  clementFoyer

  authored and

  bosilca

  committed
  f944638

• Use the world_rank for printf's.

  Show description for f6e1ba8
  

  clementFoyer

  authored and

  bosilca

  committed
  f6e1ba8

• Clean potential previous runs, but keep the results at the end in order to potentially reprocess the data. Add comments.

  Show description for a4014ec
  

  clementFoyer

  authored and

  bosilca

  committed
  a4014ec

• Add security check for unique initialization for osc monitoring

  Show description for ed6f1fd
  

  clementFoyer

  authored and

  bosilca

  committed
  ed6f1fd

• Clean the amout of symbols available outside mca/common/monitoring

  Show description for de5577f
  

  clementFoyer

  authored and

  bosilca

  committed
  de5577f

• Remove use of __sync_* built-ins. Use opal_atomic_* instead.

  Show description for c6dd8e0
  

  clementFoyer

  authored and

  bosilca

  committed
  c6dd8e0

• Add histogram ditribution of message sizes

  Show description for 52dfb1c
  

  clementFoyer

  authored and

  bosilca

  committed
  52dfb1c

• Allocate the hashtable on common/monitoring component initialization. Define symbols to set the values for error/warning/info verbose output. Use opal_atomic instead of built-in function in osc/mon…

  Show description for 549d2ef
  

  clementFoyer

  authored and

  bosilca

  committed
  549d2ef

• Deleting now useless file : moved to common/monitoring

  Show description for 545f64c
  

  clementFoyer

  authored and

  bosilca

  committed
  545f64c

• Add histogram array of 2-based log of message sizes. Use simple call to reset/allocate arrays in common_monitoring.c

  Show description for d1339da
  

  clementFoyer

  authored and

  bosilca

  committed
  d1339da

• Add informations in dumping file. Separate per category (pt2pt/osc/coll (to come)) monitored data

  Show description for e95ea72
  

  clementFoyer

  authored and

  bosilca

  committed
  e95ea72

• Add coll component for collectives communications monitoring

  Show description for 55ebb61
  

  clementFoyer

  authored and

  bosilca

  committed
  55ebb61

• Fix warning messages : use c_name as the magic id is not always defined. Moreover, there was a % missing. Add call to release underlying modules. Add debug info messages. Add warning which may lead…

  Show description for 003c92e
  

  clementFoyer

  authored and

  bosilca

  committed
  003c92e

• Fix log10_2 constant initialization. Fix index calculation for histogram array.

  Show description for b5becc0
  

  clementFoyer

  authored and

  bosilca

  committed
  b5becc0

• Add debug info messages to follow more easily initialization steps.

  Show description for 56063c9
  

  clementFoyer

  authored and

  bosilca

  committed
  56063c9

• Group all the var/pvar definitions to common_monitoring. Separate initial filename from the current on, to ease its lifetime management. Add verifications to ensure common is initialized once only.…

  Show description for 3c96586
  

  clementFoyer

  authored and

  bosilca

  committed
  3c96586

• Fix invalid memory allocation. Initialize initial_filename to empty string to avoid invalid read in mca_base_var_register.

  Show description for 76dac4d
  

  clementFoyer

  authored and

  bosilca

  committed
  76dac4d

• Fix missing procs in hashtable. Cache coll monitoring data.

  Show description for 2f1c06b
  

  clementFoyer

  authored and

  bosilca

  committed
  2f1c06b

• Don't install the test scripts.

  Show description for f31cf18
  bosilca

  committed
  f31cf18

• Use intermediate variable to avoid invalid write while retrieving ranks in hashtable.

  Show description for 0d9235e
  

  clementFoyer

  authored and

  bosilca

  committed
  0d9235e

• Add missing release of the last element in flush_all. Add release of the hashtable in finalize.

  Show description for 32c90eb
  

  clementFoyer

  authored and

  bosilca

  committed
  32c90eb

• Use a linked list instead of a hashtable to keep tracks of communicator data. Add release of the structure at finalize time.

  Show description for 070ef41
  

  clementFoyer

  authored and

  bosilca

  committed
  070ef41

• Set world_rank from hashtable only if found

  Show description for 08fa473
  

  clementFoyer

  authored and

  bosilca

  committed
  08fa473

• Use predefined symbol from opal system to print int

  Show description for 8f7cbb4
  

  clementFoyer

  authored and

  bosilca

  committed
  8f7cbb4

• Move collective monitoring data to a hashtable. Add pvar to access the monitoring_coll_data. Move functions header to a private file only to be used in ompi/mca/common/monitoring

  Show description for d8c95e6
  

  clementFoyer

  authored and

  bosilca

  committed
  d8c95e6

• Fix pvar registration. Use OMPI_ERROR isntead of -1 as returned error value. Fix releasing of coll_data_t objects. Affect value only if data is found in the hashtable.

  Show description for ce76a15
  

  clementFoyer

  authored and

  bosilca

  committed
  ce76a15

• Add automated check (with MPI_Tools) of monitoring.

  Show description for dd3480e
  

  clementFoyer

  authored and

  bosilca

  committed
  dd3480e

• Fix procs list caching in common_monitoring_coll_data_t

  Show description for 5d59aa8
  

  clementFoyer

  authored and

  bosilca

  committed
  5d59aa8

• Add linking to Fortran applications for LD_PRELOAD usage of monitoring_prof

  Show description for dd51986
  

  clementFoyer

  authored and

  bosilca

  committed
  dd51986

• Add PVAR's handles. Clean up code (visibility, add comments...). Start updating the documentation

  Show description for a059c17
  

  clementFoyer

  authored and

  bosilca

  committed
  a059c17

• Fix coll operations monitoring. Update check_monitoring accordingly to the added pvar. Fix monitoring array allocation.

  Show description for 6ae15ea
  

  clementFoyer

  authored and

  bosilca

  committed
  6ae15ea

• Documentation update.

  Show description for 770d1e0
  

  clementFoyer

  authored and

  bosilca

  committed
  770d1e0

• Aggregate monitoring COLL data to the generated matrix. Update documentation accordingly.

  Show description for 6ee2d62
  

  clementFoyer

  authored and

  bosilca

  committed
  6ee2d62

• Fix monitoring_prof (bad variable.vector used, and wrong array in PMPI_Gather).

  Show description for 9530757
  

  clementFoyer

  authored and

  bosilca

  committed
  9530757

• Add reduce_scatter and reduce_scatter_block monitoring. Reduce memory footprint of monitoring_prof. Unify OSC related outputs.

  Show description for 145a52f
  

  clementFoyer

  authored and

  bosilca

  committed
  145a52f

• Add the use of a machine file for overhead benchmark

  Show description for c365c30
  

  clementFoyer

  authored and

  bosilca

  committed
  c365c30

• Check for out-of-bound write in histogram

  Show description for 8376ad3
  

  clementFoyer

  authored and

  bosilca

  committed
  8376ad3

• Fix common_monitoring_cache object init for MPI_COMM_WORLD

  Show description for b8d6825
  

  clementFoyer

  authored and

  bosilca

  committed
  b8d6825

• Add RDMA benchmarks to test_overhead

  Show description for 08fa850
  

  clementFoyer

  authored and

  bosilca

  committed
  08fa850

• Add computation of average and median of overheads. Add comments and copyrigths to the test_overhead script

  Show description for 2914158
  

  clementFoyer

  authored and

  bosilca

  committed
  2914158

• Add dumping histogram in edge case

  Show description for cbf9bf3
  

  clementFoyer

  authored and

  bosilca

  committed
  cbf9bf3

• Add technical documentation

  Show description for 716dbb8
  

  clementFoyer

  authored and

  bosilca

  committed
  716dbb8

• Update expected output in test/monitoring/monitoring_test.c

  Show description for 5dbf25d
  

  clementFoyer

  authored and

  bosilca

  committed
  5dbf25d

• Adding a reduce(pml_monitoring_messages_count, MPI_MAX) example

  Show description for db3f3c4
  

  clementFoyer

  authored and

  bosilca

  committed
  db3f3c4

• Adapt to the new definition of communicators

  Show description for e7b154f
  

  clementFoyer

  authored and

  bosilca

  committed
  e7b154f

• Add consistency in header inclusion.

  Show description for ee939b9
  

  clementFoyer

  authored and

  bosilca

  committed
  ee939b9

• misc monitoring fixes

  Show description for b139f2b
  

  ggouaillardet

  authored and

  bosilca

  committed
  b139f2b

• Cleanups.

  Show description for 4ac5ac9
  bosilca

  committed
  4ac5ac9

• Changing int64_t to size_t.

  Show description for b93f217
  

  clementFoyer

  authored and

  bosilca

  committed
  b93f217

• Add parameter for RMA test case

  Show description for 00c896b
  

  clementFoyer

  authored and

  bosilca

  committed
  00c896b

• Clean the maximum bound computation for proc list dump.

  Show description for 65884bf
  

  clementFoyer

  authored and

  bosilca

  committed
  65884bf

• Add communicator-specific monitored collective data reset

  Show description for a0c31de
  

  clementFoyer

  authored and

  bosilca

  committed
  a0c31de

• Add monitoring scripts to the 'make dist'

  Show description for d87cba3
  bosilca

  committed
  d87cba3