0.8.0 Release

[grondo]

We have a new toss release coming up, so we should probably tag 0.8.0 soon (this week?). I'll take a stab at the release notes and paste it here.

I think there is a bunch of issues tied to the 0.8.0 milestone. We should promote most of those to 0.9.0, and also decide if we want the pending KVS work to land in 0.8.0 or 0.9.0.

flux-core version 0.8.0 (2017-08-xx)

Fixes

• libflux: remove calls to functions that exit on error (avoid use of xzmalloc and oom in libflux API sources #1060)
• fix flux_reactor_run() to return active watcher count (flux_reactor_run() should return active watcher count #1085)
• fix flux path detection when install path contains symlinks (Fix flux path detection when install path contains symlinks #1122)
• lua: fix refcount bug in kvs bindings (KVS lua refcount fix, drop unused deprecated functions #1116)
• kvs: oom() fixes (KVS oom() and xzmalloc() cleanup #1124, KVS more oom() fixes #1128)
• kvs: Fix forced dirty bit clear error (modules/kvs: Fix forced dirty bit clear error #1133)
• kvs: fix invalid memory read (fix invalid memory read in kvs module #1065)
• kvs: directory walk return error fixes (KVS directory walk return error fixes #1058)
• kvs_classic: fix kvs(dir)_put_double (kvs_classic: fix kvs(dir)_put_double #1114)
• fix memory leaks detected by valgrind (Valgrind fixes #1076)
• avoid deadlock when unloading connector-local module (avoid deadlock when unloading connector-local module #1027)
• fix several arm7l portability issues (fix several arm7l portability issues #1023)
• optparse: test and allow adjustment of posixly-correct behavior (optparse: test and allow adjustment of posixly-correct behavior #1049)
• Small improvements for systemd unit file and install paths (Small improvements for systemd unit file and install paths #1037)
• fix small leak in flux cmd driver (Change flux_handle_destroy prototype to avoid potential memory corruption, fix small leak in flux cmd help #1067)

New Features

• add FLUX_MSGFLAG_PRIVATE and allow guests to content load/store (add FLUX_MSGFLAG_PRIVATE and allow guests to content load/store #1032)
• allow guests to access hwloc topology (hwloc: open topology request to user role, add simple "info" subcommand #1043)
• libflux: new flux_future_t API (implement flux_future_t API #1083)
• libflux: implement RPCs in terms of futures (reimplement RPC in terms of futures #1089)
• kvs: implement transaction objects (implement KVS transaction object #1107)
• connector-local: Fix compiler warning (modules/connector-local: Fix compiler warning #1031)
• add optional initial program timeout, for test scripts (add optional initial program timeout, for test scripts #1129)
• libutil: new dirwalk interface (soak: Update for recent flux changes #1072, Add fixes to dirwalk tests #1061, Add dirwalk interface to libutil, update readdir callers #1059)
• connector-local: add exponential backoff to connect retry count (connector/local: add exponential backoff to connect retry count #1148)
• support tbon.endpoint and mcast.endpoint attributes (Issue385: Support tbon.endpoint and mcast.endpoint attributes #1030)
• content: allow hash type to be configured (allow hash type to be configured #1051)

Cleanup

• update many broker attribute names (Issue1040: Update broker attribute names #1042)
• consolidate installed libraries and source tree cleanup (consolidate installed libraries and source tree cleanup #1095)
• convert broker from json-c to jansson (convert broker from json-c to jansson (cleanup) #1050)
• libflux: rename jansson pack/unpack-based Flux API functions (rename jansson pack/unpack-based Flux API functions #1104)
• kvs: various code cleanup (Few miscellaneous KVS cleanups #1057, KVS Cleanup #1073, modules/kvs: Cleanup content_load_completion() #1079, clean up KVS API implementation #1099, Kvs cleanup error paths #1119, [cleanup] convert KVS users to new API and jansson, drop some old API functions #1123, Misc Refactoring & Cleanup in libkvs #1153)
• kvs: refactor kvs commit, lookup, and walk logic (KVS refactor of lookup() / walk() functions #1066, KVS commit refactor #1105)
• kvs: drop unused, legacy and deprecated functions (reduce legacy KVS API #1100, KVS lua refcount fix, drop unused deprecated functions #1116)
• kvs: switch from json-c to jansson (KVS Jansson Updates #1108, Misc Refactoring & Cleanup in libkvs #1153)
• Misc Cleanup/Minor Fixes from KVS TreeObject Work (Misc Cleanup/Minor Fixes from KVS TreeObject Work #1152)
• cron: avoid use of json-c and xzmalloc (cron: avoid use of json-c and xzmalloc #1143)
• Change void * to void ** in flux_msg_get_payload (Flux request decode raw fix #1144)
• python: make bindings compatible with newer versions of pylint (python: make bindings compatible with newer versions of pylint #1113)
• barrier: cleanup (Barrier cleanup #1092)
• tweak watcher structure, add external watcher construction interface
  (tweak watcher structure, add external watcher construction interface #1082)
• drop coprocess programming model (drop coprocess programming model #1081)
• split flux_mrpc() out to its own class (split flux_mrpc() out to its own class #1080)
• deprecate some libutil classes (deprecate some libutil classes (cleanup) #1047)
• cleanup of flux_msg_copy(), flux_rpc_aux_set() etc. (cleanup of flux_msg_copy(), flux_rpc_aux_set() etc. and tests #1056)

Testing

• update sharness version to upstream 1.0.0 version (Issue 433: Update sharness version to upstream 1.0.0 version #1035)
• cleanup kvs tests (Cleanup KVS tests #1149)
• mitigate slow builds in Travis-CI (travis-ci: mitigate slow builds #1142)
• fix --chain-lint option in sharness tests (Fix testsuite --chain-lint option, add timeout to some tests, and other testing fixes #1125)
• t2000-wreck.t: fix intermittent failures (Fix intermittent failure of t2000-wreck.t #1102, t2000-wreck.t: fix logic in check for archive job #1109)
• kvs: Add json_util unit tests (modules/kvs: Add json_util unit tests #1106)
• run valgrind if available as part of make check (Valgrind fixes #1076, only run valgrind tests if valgrind/valgrind.h was found #1098)
• add FLUX_PMI_SINGLETON env variable to avoid SLURMs libpmi in valgrind
  test (add FLUX_PMI_SINGLETON env variable to avoid SLURM's libpmi in valgrind test #1091)
• other test improvements (Minor test improvements #1087)
• update soak test for recent flux changes (soak: Update for recent flux changes #1072)
• test/security: Fix test corner case (test/security: Fix test corner case #1029)

Documentation

• add missing manpages, minor manpage fixes (Issue998: Add missing manpages, minor manpage fixes #1045)
• improve reactor documentation (improve reactor documentation #1086)
• Code comments and documentation cleanup (common/libutil: Fix base64 API documentation #1138)

[garlick]

I would say it might be a bit bold to drop those KVS changes in right before a release tag.

We should definitely try to get sched tagged and pushed out in that TOSS release though.

Can #925 (ppc64le buildfarm issue) wait until I'm back next week? I'm not sure I can get much done on that one remotely.

[garlick]

Er #918 might be the one? #925 is closed.

[garlick]

I would say it might be a bit bold to drop those KVS changes in right before a release tag.

Oh then again, it's not like we're in production with flux. Maybe moving forward is the right thing. If we can get #1154 merged early in the week we'll have a few days to discover any fallout.

(Sorry for my chatty comments, spamming everybody)

[grondo]

Oh then again, it's not like we're in production with flux.

Yes, though I think we'll definitely want something stable to make it into that release. If we have a set of KVS changes (including the utility changes) we could assign those to 0.9.0 which could be tagged in a week or a month.

I'm fine with including kvs updates though, just wanted to argue the other side.

I could investigate #918 while you're out if that's helpful. Seems like a straightforward buildfarm problem.

[chu11]

I would lean towards #1154 being in 0.9.0 as well. While PRs like #1158 and #1159 aren't a necessity or requirement, they do feel somewhat important to be in the release as well.

[garlick]

OK, works for me, and I certainly don't mind "sharing" #918 - just didn't want to dump it on anybody since I had volunteered.

[grondo]

@chu11, just to clarify, you are proposing we tag 0.8.0 without #1154, and assign that PR and #1158 and #1159 to 0.9.0, right?

[chu11]

@grondo yes, correct

[grondo]

I added a first cut of the NEWS.md additions to the first comment above.

[garlick]

Thanks for putting together the notes @grondo!