Replace FSMonitor v2 with v4

[dscho]

We already skipped v3, but it is time to take (tentative) v4 of the patch series. "Tentative" because it has technically not been contributed to the Git mailing list, but it will be contributed soon after v2.33.0.

[derrickstolee]

I'm happy to approve this to unblock it, but is it possible to create a range-diff with the existing implementation in main?

[jeffhostetler]

We should do the MAX_LONG_PATH fixup on this file too.

[dscho]

I just force-pushed a fix.

[dscho]

is it possible to create a range-diff with the existing implementation in main?

Certainly! But I must warn you: It's not an easy-to-read range-diff, nor is it short. I used the command-line git range-diff --creation-factor=99 906cca96e73~2..906cca96e73 bcf4d40110f~2..bcf4d40110f to generate it.

 1:  323a6ae8acd <  -:  ----------- git-artifacts: add workaround for GCM Core on ARM64
 2:  5c2c31d5968 =  1:  cafc71c8d7d simple-ipc: preparations for supporting binary messages.
 3:  06a65fd7f2f =  2:  5db2c0390a6 fsmonitor--daemon: man page
 4:  fc19ed00b3f =  3:  86413bfe347 fsmonitor--daemon: update fsmonitor documentation
 5:  b71b5dede8a !  4:  dfcd3e5ac97 fsmonitor-ipc: create client routines for git-fsmonitor--daemon
    @@ fsmonitor-ipc.c (new)
     +	struct ipc_client_connection *connection = NULL;
     +	struct ipc_client_connect_options options
     +		= IPC_CLIENT_CONNECT_OPTIONS_INIT;
    ++	const char *tok = since_token ? since_token : "";
    ++	size_t tok_len = since_token ? strlen(since_token) : 0;
     +
     +	options.wait_if_busy = 1;
     +	options.wait_if_not_found = 0;
     +
     +	trace2_region_enter("fsm_client", "query", NULL);
    -+
    -+	trace2_data_string("fsm_client", NULL, "query/command",
    -+			   since_token);
    ++	trace2_data_string("fsm_client", NULL, "query/command", tok);
     +
     +try_again:
     +	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
    @@ fsmonitor-ipc.c (new)
     +	switch (state) {
     +	case IPC_STATE__LISTENING:
     +		ret = ipc_client_send_command_to_connection(
    -+			connection, since_token, strlen(since_token), answer);
    ++			connection, tok, tok_len, answer);
     +		ipc_client_close_connection(connection);
     +
     +		trace2_data_intmax("fsm_client", NULL,
    @@ fsmonitor-ipc.c (new)
     +		= IPC_CLIENT_CONNECT_OPTIONS_INIT;
     +	int ret;
     +	enum ipc_active_state state;
    ++	const char *c = command ? command : "";
    ++	size_t c_len = command ? strlen(command) : 0;
     +
     +	strbuf_reset(answer);
     +
    @@ fsmonitor-ipc.c (new)
     +		return -1;
     +	}
     +
    -+	ret = ipc_client_send_command_to_connection(connection,
    -+						    command, strlen(command),
    ++	ret = ipc_client_send_command_to_connection(connection, c, c_len,
     +						    answer);
     +	ipc_client_close_connection(connection);
     +
     +	if (ret == -1) {
    -+		die("could not send '%s' command to fsmonitor--daemon",
    -+		    command);
    ++		die("could not send '%s' command to fsmonitor--daemon", c);
     +		return -1;
     +	}
     +
 6:  d2ec6458bba !  5:  0aaca2f9390 help: include fsmonitor--daemon feature flag in version info
    @@ help.c: void get_version_info(struct strbuf *buf, int show_build_options)
      	}
      }
      
    +
    + ## t/test-lib.sh ##
    +@@ t/test-lib.sh: test_lazy_prereq REBASE_P '
    + # Tests that verify the scheduler integration must set this locally
    + # to avoid errors.
    + GIT_TEST_MAINT_SCHEDULER="none:exit 1"
    ++
    ++# Does this platform support `git fsmonitor--daemon`
    ++#
    ++test_lazy_prereq FSMONITOR_DAEMON '
    ++	git version --build-options | grep "feature:" | grep "fsmonitor--daemon"
    ++'
 7:  4c4e3551ed1 !  6:  0a756b2a254 config: FSMonitor is repository-specific
    @@
      ## Metadata ##
    -Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
    +Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    config: FSMonitor is repository-specific
    +    fsmonitor: config settings are repository-specific
     
    -    This commit refactors `git_config_get_fsmonitor()` into the `repo_*()`
    -    form that takes a parameter `struct repository *r`.
    +    Move FSMonitor config settings to a new `struct fsmonitor_settings`
    +    structure.  Add a lazily-loaded pointer to `struct repo_settings`.
    +    Create `fsm_settings__get_*()` getters to lazily look up fsmonitor-
    +    related config settings.
     
    -    That change prepares for the upcoming `core.useBuiltinFSMonitor` flag which
    -    will be stored in the `repo_settings` struct.
    +    Get rid of the `core_fsmonitor` global variable, and add support for
    +    the new `core.useBuiltinFSMonitor` config setting.  Move config code
    +    to lookup the existing `core.fsmonitor` value to `fsmonitor-settings.[ch]`.
     
    -    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    The `core_fsmonitor` global variable was used to store the pathname to
    +    the FSMonitor hook and it was used as a boolean to see if FSMonitor
    +    was enabled.  This dual usage will lead to confusion when we add
    +    support for a builtin FSMonitor based on IPC, since the builtin
    +    FSMonitor doesn't need the hook pathname.
    +
    +    Replace the boolean usage with an `enum fsmonitor_mode` to represent
    +    the state of FSMonitor.  And only set the pathname when in HOOK mode.
    +
    +    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    +
    + ## Makefile ##
    +@@ Makefile: LIB_OBJS += fmt-merge-msg.o
    + LIB_OBJS += fsck.o
    + LIB_OBJS += fsmonitor.o
    + LIB_OBJS += fsmonitor-ipc.o
    ++LIB_OBJS += fsmonitor-settings.o
    + LIB_OBJS += gettext.o
    + LIB_OBJS += gpg-interface.o
    + LIB_OBJS += graph.o
     
      ## builtin/update-index.c ##
     @@ builtin/update-index.c: int cmd_update_index(int argc, const char **argv, const char *prefix)
    @@ builtin/update-index.c: int cmd_update_index(int argc, const char **argv, const
      
      	if (fsmonitor > 0) {
     -		if (git_config_get_fsmonitor() == 0)
    -+		if (repo_config_get_fsmonitor(r) == 0)
    ++		enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
    ++
    ++		if (fsm_mode == FSMONITOR_MODE_DISABLED) {
    ++			warning(_("core.useBuiltinFSMonitor is unset; "
    ++				"set it if you really want to enable the "
    ++				"builtin fsmonitor"));
      			warning(_("core.fsmonitor is unset; "
    - 				"set it if you really want to "
    - 				"enable fsmonitor"));
    +-				"set it if you really want to "
    +-				"enable fsmonitor"));
    ++				"set it if you really want to enable the "
    ++				"hook-based fsmonitor"));
    ++		}
      		add_fsmonitor(&the_index);
      		report(_("fsmonitor enabled"));
      	} else if (!fsmonitor) {
     -		if (git_config_get_fsmonitor() == 1)
    -+		if (repo_config_get_fsmonitor(r) == 1)
    ++		enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
    ++		if (fsm_mode == FSMONITOR_MODE_IPC)
    ++			warning(_("core.useBuiltinFSMonitor is set; "
    ++				"remove it if you really want to "
    ++				"disable fsmonitor"));
    ++		if (fsm_mode == FSMONITOR_MODE_HOOK)
      			warning(_("core.fsmonitor is set; "
      				"remove it if you really want to "
      				"disable fsmonitor"));
     
    + ## cache.h ##
    +@@ cache.h: extern int core_preload_index;
    + extern int precomposed_unicode;
    + extern int protect_hfs;
    + extern int protect_ntfs;
    +-extern const char *core_fsmonitor;
    + 
    + extern int core_apply_sparse_checkout;
    + extern int core_sparse_checkout_cone;
    +
      ## config.c ##
     @@ config.c: int git_config_get_max_percent_split_change(void)
      	return -1; /* default value */
      }
      
     -int git_config_get_fsmonitor(void)
    -+int repo_config_get_fsmonitor(struct repository *r)
    - {
    +-{
     -	if (git_config_get_pathname("core.fsmonitor", &core_fsmonitor))
    -+	if (repo_config_get_pathname(r, "core.fsmonitor", &core_fsmonitor))
    - 		core_fsmonitor = getenv("GIT_TEST_FSMONITOR");
    - 
    - 	if (core_fsmonitor && !*core_fsmonitor)
    +-		core_fsmonitor = getenv("GIT_TEST_FSMONITOR");
    +-
    +-	if (core_fsmonitor && !*core_fsmonitor)
    +-		core_fsmonitor = NULL;
    +-
    +-	if (core_fsmonitor)
    +-		return 1;
    +-
    +-	return 0;
    +-}
    +-
    + int git_config_get_index_threads(int *dest)
    + {
    + 	int is_bool, val;
     
      ## config.h ##
     @@ config.h: int git_config_get_index_threads(int *dest);
    @@ config.h: int git_config_get_index_threads(int *dest);
      int git_config_get_split_index(void);
      int git_config_get_max_percent_split_change(void);
     -int git_config_get_fsmonitor(void);
    -+int repo_config_get_fsmonitor(struct repository *r);
      
      /* This dies if the configured or default date is in the future */
      int git_config_get_expiry(const char *key, const char **output);
     
    + ## environment.c ##
    +@@ environment.c: int protect_hfs = PROTECT_HFS_DEFAULT;
    + #define PROTECT_NTFS_DEFAULT 1
    + #endif
    + int protect_ntfs = PROTECT_NTFS_DEFAULT;
    +-const char *core_fsmonitor;
    + 
    + /*
    +  * The character that begins a commented line in user-editable file
    +
    + ## fsmonitor-settings.c (new) ##
    +@@
    ++#include "cache.h"
    ++#include "config.h"
    ++#include "repository.h"
    ++#include "fsmonitor-settings.h"
    ++
    ++/*
    ++ * We keep this structure defintion private and have getters
    ++ * for all fields so that we can lazy load it as needed.
    ++ */
    ++struct fsmonitor_settings {
    ++	enum fsmonitor_mode mode;
    ++	char *hook_path;
    ++};
    ++
    ++void fsm_settings__set_ipc(struct repository *r)
    ++{
    ++	struct fsmonitor_settings *s = r->settings.fsmonitor;
    ++
    ++	s->mode = FSMONITOR_MODE_IPC;
    ++}
    ++
    ++void fsm_settings__set_hook(struct repository *r, const char *path)
    ++{
    ++	struct fsmonitor_settings *s = r->settings.fsmonitor;
    ++
    ++	s->mode = FSMONITOR_MODE_HOOK;
    ++	s->hook_path = strdup(path);
    ++}
    ++
    ++void fsm_settings__set_disabled(struct repository *r)
    ++{
    ++	struct fsmonitor_settings *s = r->settings.fsmonitor;
    ++
    ++	s->mode = FSMONITOR_MODE_DISABLED;
    ++	FREE_AND_NULL(s->hook_path);
    ++}
    ++
    ++static int check_for_ipc(struct repository *r)
    ++{
    ++	int value;
    ++
    ++	if (!repo_config_get_bool(r, "core.usebuiltinfsmonitor", &value) &&
    ++	    value) {
    ++		fsm_settings__set_ipc(r);
    ++		return 1;
    ++	}
    ++
    ++	return 0;
    ++}
    ++
    ++static int check_for_hook(struct repository *r)
    ++{
    ++	const char *const_str;
    ++
    ++	if (repo_config_get_pathname(r, "core.fsmonitor", &const_str))
    ++		const_str = getenv("GIT_TEST_FSMONITOR");
    ++
    ++	if (const_str && *const_str) {
    ++		fsm_settings__set_hook(r, const_str);
    ++		return 1;
    ++	}
    ++
    ++	return 0;
    ++}
    ++
    ++static void lookup_fsmonitor_settings(struct repository *r)
    ++{
    ++	struct fsmonitor_settings *s;
    ++
    ++	CALLOC_ARRAY(s, 1);
    ++
    ++	r->settings.fsmonitor = s;
    ++
    ++	if (check_for_ipc(r))
    ++		return;
    ++
    ++	if (check_for_hook(r))
    ++		return;
    ++
    ++	fsm_settings__set_disabled(r);
    ++}
    ++
    ++enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
    ++{
    ++	if (!r->settings.fsmonitor)
    ++		lookup_fsmonitor_settings(r);
    ++
    ++	return r->settings.fsmonitor->mode;
    ++}
    ++
    ++const char *fsm_settings__get_hook_path(struct repository *r)
    ++{
    ++	if (!r->settings.fsmonitor)
    ++		lookup_fsmonitor_settings(r);
    ++
    ++	return r->settings.fsmonitor->hook_path;
    ++}
    +
    + ## fsmonitor-settings.h (new) ##
    +@@
    ++#ifndef FSMONITOR_SETTINGS_H
    ++#define FSMONITOR_SETTINGS_H
    ++
    ++struct repository;
    ++
    ++enum fsmonitor_mode {
    ++	FSMONITOR_MODE_DISABLED = 0,
    ++	FSMONITOR_MODE_HOOK = 1, /* core.fsmonitor */
    ++	FSMONITOR_MODE_IPC = 2,  /* core.useBuiltinFSMonitor */
    ++};
    ++
    ++void fsm_settings__set_ipc(struct repository *r);
    ++void fsm_settings__set_hook(struct repository *r, const char *path);
    ++void fsm_settings__set_disabled(struct repository *r);
    ++
    ++enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
    ++const char *fsm_settings__get_hook_path(struct repository *r);
    ++
    ++struct fsmonitor_settings;
    ++
    ++#endif /* FSMONITOR_SETTINGS_H */
    +
      ## fsmonitor.c ##
    +@@
    + #include "dir.h"
    + #include "ewah/ewok.h"
    + #include "fsmonitor.h"
    ++#include "fsmonitor-ipc.h"
    + #include "run-command.h"
    + #include "strbuf.h"
    + 
    +@@ fsmonitor.c: void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate)
    + /*
    +  * Call the query-fsmonitor hook passing the last update token of the saved results.
    +  */
    +-static int query_fsmonitor(int version, const char *last_update, struct strbuf *query_result)
    ++static int query_fsmonitor_hook(struct repository *r,
    ++				int version,
    ++				const char *last_update,
    ++				struct strbuf *query_result)
    + {
    + 	struct child_process cp = CHILD_PROCESS_INIT;
    + 	int result;
    + 
    +-	if (!core_fsmonitor)
    ++	if (fsm_settings__get_mode(r) != FSMONITOR_MODE_HOOK)
    + 		return -1;
    + 
    +-	strvec_push(&cp.args, core_fsmonitor);
    ++	strvec_push(&cp.args, fsm_settings__get_hook_path(r));
    + 	strvec_pushf(&cp.args, "%d", version);
    + 	strvec_pushf(&cp.args, "%s", last_update);
    + 	cp.use_shell = 1;
    +@@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
    + 	struct strbuf last_update_token = STRBUF_INIT;
    + 	char *buf;
    + 	unsigned int i;
    ++	struct repository *r = istate->repo ? istate->repo : the_repository;
    ++	enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
    + 
    +-	if (!core_fsmonitor || istate->fsmonitor_has_run_once)
    ++	if (fsm_mode <= FSMONITOR_MODE_DISABLED ||
    ++	    istate->fsmonitor_has_run_once)
    + 		return;
    + 
    +-	hook_version = fsmonitor_hook_version();
    +-
    + 	istate->fsmonitor_has_run_once = 1;
    + 
    + 	trace_printf_key(&trace_fsmonitor, "refresh fsmonitor");
    ++
    ++	if (fsm_mode == FSMONITOR_MODE_IPC) {
    ++		/* TODO */
    ++		return;
    ++	}
    ++
    ++	assert(fsm_mode == FSMONITOR_MODE_HOOK);
    ++
    ++	hook_version = fsmonitor_hook_version();
    ++
    + 	/*
    +-	 * This could be racy so save the date/time now and query_fsmonitor
    ++	 * This could be racy so save the date/time now and query_fsmonitor_hook
    + 	 * should be inclusive to ensure we don't miss potential changes.
    + 	 */
    + 	last_update = getnanotime();
    +@@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
    + 		strbuf_addf(&last_update_token, "%"PRIu64"", last_update);
    + 
    + 	/*
    +-	 * If we have a last update token, call query_fsmonitor for the set of
    ++	 * If we have a last update token, call query_fsmonitor_hook for the set of
    + 	 * changes since that token, else assume everything is possibly dirty
    + 	 * and check it all.
    + 	 */
    + 	if (istate->fsmonitor_last_update) {
    + 		if (hook_version == -1 || hook_version == HOOK_INTERFACE_VERSION2) {
    +-			query_success = !query_fsmonitor(HOOK_INTERFACE_VERSION2,
    ++			query_success = !query_fsmonitor_hook(
    ++				r, HOOK_INTERFACE_VERSION2,
    + 				istate->fsmonitor_last_update, &query_result);
    + 
    + 			if (query_success) {
    +@@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
    + 		}
    + 
    + 		if (hook_version == HOOK_INTERFACE_VERSION1) {
    +-			query_success = !query_fsmonitor(HOOK_INTERFACE_VERSION1,
    ++			query_success = !query_fsmonitor_hook(
    ++				r, HOOK_INTERFACE_VERSION1,
    + 				istate->fsmonitor_last_update, &query_result);
    + 		}
    + 
    +-		trace_performance_since(last_update, "fsmonitor process '%s'", core_fsmonitor);
    +-		trace_printf_key(&trace_fsmonitor, "fsmonitor process '%s' returned %s",
    +-			core_fsmonitor, query_success ? "success" : "failure");
    ++		trace_performance_since(last_update, "fsmonitor process '%s'",
    ++					fsm_settings__get_hook_path(r));
    ++		trace_printf_key(&trace_fsmonitor,
    ++				 "fsmonitor process '%s' returned %s",
    ++				 fsm_settings__get_hook_path(r),
    ++				 query_success ? "success" : "failure");
    + 	}
    + 
    + 	/* a fsmonitor process can return '/' to indicate all entries are invalid */
     @@ fsmonitor.c: void remove_fsmonitor(struct index_state *istate)
      void tweak_fsmonitor(struct index_state *istate)
      {
      	unsigned int i;
     -	int fsmonitor_enabled = git_config_get_fsmonitor();
    -+	int fsmonitor_enabled = repo_config_get_fsmonitor(istate->repo ? istate->repo : the_repository);
    ++	struct repository *r = istate->repo ? istate->repo : the_repository;
    ++	int fsmonitor_enabled = (fsm_settings__get_mode(r) > FSMONITOR_MODE_DISABLED);
      
      	if (istate->fsmonitor_dirty) {
      		if (fsmonitor_enabled) {
    +@@ fsmonitor.c: void tweak_fsmonitor(struct index_state *istate)
    + 		istate->fsmonitor_dirty = NULL;
    + 	}
    + 
    +-	switch (fsmonitor_enabled) {
    +-	case -1: /* keep: do nothing */
    +-		break;
    +-	case 0: /* false */
    +-		remove_fsmonitor(istate);
    +-		break;
    +-	case 1: /* true */
    ++	if (fsmonitor_enabled)
    + 		add_fsmonitor(istate);
    +-		break;
    +-	default: /* unknown value: do nothing */
    +-		break;
    +-	}
    ++	else
    ++		remove_fsmonitor(istate);
    + }
    +
    + ## fsmonitor.h ##
    +@@
    + 
    + #include "cache.h"
    + #include "dir.h"
    ++#include "fsmonitor-settings.h"
    + 
    + extern struct trace_key trace_fsmonitor;
    + 
    +@@ fsmonitor.h: int fsmonitor_is_trivial_response(const struct strbuf *query_result);
    +  */
    + static inline int is_fsmonitor_refreshed(const struct index_state *istate)
    + {
    +-	return !core_fsmonitor || istate->fsmonitor_has_run_once;
    ++	struct repository *r = istate->repo ? istate->repo : the_repository;
    ++	enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
    ++
    ++	return fsm_mode <= FSMONITOR_MODE_DISABLED ||
    ++		istate->fsmonitor_has_run_once;
    + }
    + 
    + /*
    +@@ fsmonitor.h: static inline int is_fsmonitor_refreshed(const struct index_state *istate)
    +  */
    + static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache_entry *ce)
    + {
    +-	if (core_fsmonitor && !(ce->ce_flags & CE_FSMONITOR_VALID)) {
    ++	struct repository *r = istate->repo ? istate->repo : the_repository;
    ++	enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
    ++
    ++	if (fsm_mode > FSMONITOR_MODE_DISABLED &&
    ++	    !(ce->ce_flags & CE_FSMONITOR_VALID)) {
    + 		istate->cache_changed = 1;
    + 		ce->ce_flags |= CE_FSMONITOR_VALID;
    + 		trace_printf_key(&trace_fsmonitor, "mark_fsmonitor_clean '%s'", ce->name);
    +@@ fsmonitor.h: static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache
    +  */
    + static inline void mark_fsmonitor_invalid(struct index_state *istate, struct cache_entry *ce)
    + {
    +-	if (core_fsmonitor) {
    ++	struct repository *r = istate->repo ? istate->repo : the_repository;
    ++	enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
    ++
    ++	if (fsm_mode > FSMONITOR_MODE_DISABLED) {
    + 		ce->ce_flags &= ~CE_FSMONITOR_VALID;
    + 		untracked_cache_invalidate_path(istate, ce->name, 1);
    + 		trace_printf_key(&trace_fsmonitor, "mark_fsmonitor_invalid '%s'", ce->name);
    +
    + ## repo-settings.c ##
    +@@ repo-settings.c: void prepare_repo_settings(struct repository *r)
    + 	UPDATE_DEFAULT_BOOL(r->settings.commit_graph_read_changed_paths, 1);
    + 	UPDATE_DEFAULT_BOOL(r->settings.gc_write_commit_graph, 1);
    + 
    ++	r->settings.fsmonitor = NULL; /* lazy loaded */
    ++
    + 	if (!repo_config_get_int(r, "index.version", &value))
    + 		r->settings.index_version = value;
    + 	if (!repo_config_get_maybe_bool(r, "core.untrackedcache", &value)) {
    +
    + ## repository.h ##
    +@@
    + #include "path.h"
    + 
    + struct config_set;
    ++struct fsmonitor_settings;
    + struct git_hash_algo;
    + struct index_state;
    + struct lock_file;
    +@@ repository.h: struct repo_settings {
    + 	int gc_write_commit_graph;
    + 	int fetch_write_commit_graph;
    + 
    ++	struct fsmonitor_settings *fsmonitor; /* lazy loaded */
    ++
    + 	int index_version;
    + 	enum untracked_cache_setting core_untracked_cache;
    + 
    +
    + ## t/README ##
    +@@ t/README: every 'git commit-graph write', as if the `--changed-paths` option was
    + passed in.
    + 
    + GIT_TEST_FSMONITOR=$PWD/t7519/fsmonitor-all exercises the fsmonitor
    +-code path for utilizing a file system monitor to speed up detecting
    +-new or changed files.
    ++code path for utilizing a (hook based) file system monitor to speed up
    ++detecting new or changed files.
    + 
    + GIT_TEST_INDEX_VERSION=<n> exercises the index read/write code path
    + for the index version specified.  Can be set to any valid version
 8:  0cc85cb3ded !  7:  d8f36842d89 fsmonitor: introduce `core.useBuiltinFSMonitor` to call the daemon via IPC
    @@
      ## Metadata ##
    -Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
    +Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor: introduce `core.useBuiltinFSMonitor` to call the daemon via IPC
    +    fsmonitor: use IPC to query the builtin FSMonitor daemon
     
         Use simple IPC to directly communicate with the new builtin file
    -    system monitor daemon.
    -
    -    Define a new config setting `core.useBuiltinFSMonitor` to enable the
    -    builtin file system monitor.
    +    system monitor daemon when `core.useBuiltinFSMonitor` is set.
     
         The `core.fsmonitor` setting has already been defined as a HOOK
         pathname.  Historically, this has been set to a HOOK script that will
    @@ Commit message
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
     
    - ## config.c ##
    -@@ config.c: int git_config_get_max_percent_split_change(void)
    - 
    - int repo_config_get_fsmonitor(struct repository *r)
    - {
    -+	if (r->settings.use_builtin_fsmonitor > 0) {
    -+		core_fsmonitor = "(built-in daemon)";
    -+		return 1;
    -+	}
    -+
    - 	if (repo_config_get_pathname(r, "core.fsmonitor", &core_fsmonitor))
    - 		core_fsmonitor = getenv("GIT_TEST_FSMONITOR");
    - 
    -
      ## fsmonitor.c ##
    -@@
    - #include "dir.h"
    - #include "ewah/ewok.h"
    - #include "fsmonitor.h"
    -+#include "fsmonitor-ipc.h"
    - #include "run-command.h"
    - #include "strbuf.h"
    - 
    -@@ fsmonitor.c: static void fsmonitor_refresh_callback(struct index_state *istate, char *name)
    - 
    - void refresh_fsmonitor(struct index_state *istate)
    - {
    -+	struct repository *r = istate->repo ? istate->repo : the_repository;
    - 	struct strbuf query_result = STRBUF_INIT;
    - 	int query_success = 0, hook_version = -1;
    - 	size_t bol = 0; /* beginning of line */
     @@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
    - 	istate->fsmonitor_has_run_once = 1;
    - 
      	trace_printf_key(&trace_fsmonitor, "refresh fsmonitor");
    -+
    -+	if (r->settings.use_builtin_fsmonitor > 0) {
    + 
    + 	if (fsm_mode == FSMONITOR_MODE_IPC) {
    +-		/* TODO */
    +-		return;
     +		query_success = !fsmonitor_ipc__send_query(
     +			istate->fsmonitor_last_update ?
     +			istate->fsmonitor_last_update : "builtin:fake",
    @@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
     +			strbuf_addstr(&last_update_token, "builtin:fake");
     +		}
     +
    -+		/*
    -+		 * Regardless of whether we successfully talked to a
    -+		 * fsmonitor daemon or not, we skip over and do not
    -+		 * try to use the hook.  The "core.useBuiltinFSMonitor"
    -+		 * config setting ALWAYS overrides the "core.fsmonitor"
    -+		 * hook setting.
    -+		 */
     +		goto apply_results;
    -+	}
    -+
    - 	/*
    - 	 * This could be racy so save the date/time now and query_fsmonitor
    - 	 * should be inclusive to ensure we don't miss potential changes.
    + 	}
    + 
    + 	assert(fsm_mode == FSMONITOR_MODE_HOOK);
     @@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
    - 			core_fsmonitor, query_success ? "success" : "failure");
    + 				 query_success ? "success" : "failure");
      	}
      
     +apply_results:
      	/* a fsmonitor process can return '/' to indicate all entries are invalid */
      	if (query_success && query_result.buf[bol] != '/') {
      		/* Mark all entries returned by the monitor as dirty */
    -
    - ## repo-settings.c ##
    -@@ repo-settings.c: void prepare_repo_settings(struct repository *r)
    - 		r->settings.core_multi_pack_index = value;
    - 	UPDATE_DEFAULT_BOOL(r->settings.core_multi_pack_index, 1);
    - 
    -+	if (!repo_config_get_bool(r, "core.usebuiltinfsmonitor", &value) && value)
    -+		r->settings.use_builtin_fsmonitor = 1;
    -+
    - 	if (!repo_config_get_bool(r, "feature.manyfiles", &value) && value) {
    - 		UPDATE_DEFAULT_BOOL(r->settings.index_version, 4);
    - 		UPDATE_DEFAULT_BOOL(r->settings.core_untracked_cache, UNTRACKED_CACHE_WRITE);
    -
    - ## repository.h ##
    -@@ repository.h: enum fetch_negotiation_setting {
    - struct repo_settings {
    - 	int initialized;
    - 
    -+	int use_builtin_fsmonitor;
    -+
    - 	int core_commit_graph;
    - 	int commit_graph_read_changed_paths;
    - 	int gc_write_commit_graph;
 9:  747f767cdad =  8:  c92b348a7cd fsmonitor--daemon: add a built-in fsmonitor daemon
10:  e1c1624c77a !  9:  b137b625001 fsmonitor--daemon: implement client command options
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor--daemon: implement client command options
    +    fsmonitor--daemon: implement 'stop' and 'status' commands
     
         Implement `stop` and `status` client commands to control and query the
         status of a `fsmonitor--daemon` server process (and implicitly start a
    @@ builtin/fsmonitor--daemon.c
     +
     +	switch (state) {
     +	case IPC_STATE__LISTENING:
    -+		printf(_("The built-in file system monitor is active\n"));
    ++		printf(_("fsmonitor-daemon is watching '%s'\n"),
    ++		       the_repository->worktree);
     +		return 0;
     +
     +	default:
    -+		printf(_("The built-in file system monitor is not active\n"));
    ++		printf(_("fsmonitor-daemon is not watching '%s'\n"),
    ++		       the_repository->worktree);
     +		return 1;
     +	}
     +}
11:  6774837ccc7 ! 10:  4e81f9bac1f t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon
    @@ t/helper/test-fsmonitor-client.c (new)
     +#include "test-tool.h"
     +#include "cache.h"
     +#include "parse-options.h"
    -+//#include "fsmonitor.h"
     +#include "fsmonitor-ipc.h"
    -+//#include "compat/fsmonitor/fsmonitor-fs-listen.h"
    -+//#include "fsmonitor--daemon.h"
    -+//#include "simple-ipc.h"
     +
     +#ifndef HAVE_FSMONITOR_DAEMON_BACKEND
     +int cmd__fsmonitor_client(int argc, const char **argv)
12:  08f094608c8 ! 11:  91b26cc2981 fsmonitor-fs-listen-win32: stub in backend for Windows
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor-fs-listen-win32: stub in backend for Windows
    +    fsm-listen-win32: stub in backend for Windows
     
    -    Stub in empty backend for fsmonitor--daemon on Windows.
    +    Stub in empty filesystem listener backend for fsmonitor--daemon on Windows.
     
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
     
    @@ Makefile: all::
      #
     +# If your platform supports a built-in fsmonitor backend, set
     +# FSMONITOR_DAEMON_BACKEND to the "<name>" of the corresponding
    -+# `compat/fsmonitor/fsmonitor-fs-listen-<name>.c` that implements the
    -+# `fsmonitor_fs_listen__*()` routines.
    ++# `compat/fsmonitor/fsm-listen-<name>.c` that implements the
    ++# `fsm_listen__*()` routines.
     +#
      # Define DEVELOPER to enable more compiler warnings. Compiler version
      # and family are auto detected, but could be overridden by defining
    @@ Makefile: ifdef NEED_ACCESS_ROOT_HANDLER
      
     +ifdef FSMONITOR_DAEMON_BACKEND
     +	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
    -+	COMPAT_OBJS += compat/fsmonitor/fsmonitor-fs-listen-$(FSMONITOR_DAEMON_BACKEND).o
    ++	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
     +endif
     +
      ifeq ($(TCLTK_PATH),)
    @@ Makefile: GIT-BUILD-OPTIONS: FORCE
      	@echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst ','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+
      endif
     
    - ## compat/fsmonitor/fsmonitor-fs-listen-win32.c (new) ##
    + ## compat/fsmonitor/fsm-listen-win32.c (new) ##
     @@
     +#include "cache.h"
     +#include "config.h"
     +#include "fsmonitor.h"
    -+#include "fsmonitor-fs-listen.h"
    ++#include "fsm-listen.h"
     +
    -+void fsmonitor_fs_listen__stop_async(struct fsmonitor_daemon_state *state)
    ++void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
     +{
     +}
     +
    -+void fsmonitor_fs_listen__loop(struct fsmonitor_daemon_state *state)
    ++void fsm_listen__loop(struct fsmonitor_daemon_state *state)
     +{
     +}
     +
    -+int fsmonitor_fs_listen__ctor(struct fsmonitor_daemon_state *state)
    ++int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
     +{
     +	return -1;
     +}
     +
    -+void fsmonitor_fs_listen__dtor(struct fsmonitor_daemon_state *state)
    ++void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
     +{
     +}
     
    - ## compat/fsmonitor/fsmonitor-fs-listen.h (new) ##
    + ## compat/fsmonitor/fsm-listen.h (new) ##
     @@
    -+#ifndef FSMONITOR_FS_LISTEN_H
    -+#define FSMONITOR_FS_LISTEN_H
    ++#ifndef FSM_LISTEN_H
    ++#define FSM_LISTEN_H
     +
     +/* This needs to be implemented by each backend */
     +
    @@ compat/fsmonitor/fsmonitor-fs-listen.h (new)
     + * Returns 0 if successful.
     + * Returns -1 otherwise.
     + */
    -+int fsmonitor_fs_listen__ctor(struct fsmonitor_daemon_state *state);
    ++int fsm_listen__ctor(struct fsmonitor_daemon_state *state);
     +
     +/*
     + * Cleanup platform-specific data for the fsmonitor listener thread.
     + * This will be called from the main thread AFTER joining the listener.
     + */
    -+void fsmonitor_fs_listen__dtor(struct fsmonitor_daemon_state *state);
    ++void fsm_listen__dtor(struct fsmonitor_daemon_state *state);
     +
     +/*
     + * The main body of the platform-specific event loop to watch for
    @@ compat/fsmonitor/fsmonitor-fs-listen.h (new)
     + * It should set `state->error_code` to -1 if the daemon should exit
     + * with an error.
     + */
    -+void fsmonitor_fs_listen__loop(struct fsmonitor_daemon_state *state);
    ++void fsm_listen__loop(struct fsmonitor_daemon_state *state);
     +
     +/*
     + * Gently request that the fsmonitor listener thread shutdown.
     + * It does not wait for it to stop.  The caller should do a JOIN
     + * to wait for it.
     + */
    -+void fsmonitor_fs_listen__stop_async(struct fsmonitor_daemon_state *state);
    ++void fsm_listen__stop_async(struct fsmonitor_daemon_state *state);
     +
     +#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
    -+#endif /* FSMONITOR_FS_LISTEN_H */
    ++#endif /* FSM_LISTEN_H */
     
      ## config.mak.uname ##
     @@ config.mak.uname: ifeq ($(uname_S),Windows)
    @@ contrib/buildsystems/CMakeLists.txt: else()
      
     +if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
     +	add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
    -+	list(APPEND compat_SOURCES compat/fsmonitor/fsmonitor-fs-listen-win32.c)
    ++	list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
     +endif()
     +
      set(EXE_EXTENSION ${CMAKE_EXECUTABLE_SUFFIX})
      
      #header checks
    +
    + ## repo-settings.c ##
    +@@
    + #include "config.h"
    + #include "repository.h"
    + #include "midx.h"
    ++#include "compat/fsmonitor/fsm-listen.h"
    + 
    + #define UPDATE_DEFAULT_BOOL(s,v) do { if (s == -1) { s = v; } } while(0)
    + 
13:  5667e8d7fe6 ! 12:  59bf7463526 fsmonitor-fs-listen-macos: stub in backend for MacOS
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor-fs-listen-macos: stub in backend for MacOS
    +    fsm-listen-darwin: stub in backend for Darwin
     
         Stub in empty implementation of fsmonitor--daemon
    -    backend for MacOS.
    +    backend for Darwin (aka MacOS).
     
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
     
    - ## compat/fsmonitor/fsmonitor-fs-listen-macos.c (new) ##
    + ## compat/fsmonitor/fsm-listen-darwin.c (new) ##
     @@
     +#include "cache.h"
     +#include "fsmonitor.h"
    -+#include "fsmonitor-fs-listen.h"
    ++#include "fsm-listen.h"
     +
    -+int fsmonitor_fs_listen__ctor(struct fsmonitor_daemon_state *state)
    ++int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
     +{
     +	return -1;
     +}
     +
    -+void fsmonitor_fs_listen__dtor(struct fsmonitor_daemon_state *state)
    ++void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
     +{
     +}
     +
    -+void fsmonitor_fs_listen__stop_async(struct fsmonitor_daemon_state *state)
    ++void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
     +{
     +}
     +
    -+void fsmonitor_fs_listen__loop(struct fsmonitor_daemon_state *state)
    ++void fsm_listen__loop(struct fsmonitor_daemon_state *state)
     +{
     +}
     
    @@ config.mak.uname: ifeq ($(uname_S),Darwin)
      			MSGFMT = /usr/local/opt/gettext/bin/msgfmt
      		endif
      	endif
    -+	FSMONITOR_DAEMON_BACKEND = macos
    ++	FSMONITOR_DAEMON_BACKEND = darwin
     +	BASIC_LDFLAGS += -framework CoreServices
      endif
      ifeq ($(uname_S),SunOS)
    @@ contrib/buildsystems/CMakeLists.txt
     @@ contrib/buildsystems/CMakeLists.txt: endif()
      if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
      	add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
    - 	list(APPEND compat_SOURCES compat/fsmonitor/fsmonitor-fs-listen-win32.c)
    + 	list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
     +elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin")
     +	add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
    -+	list(APPEND compat_SOURCES compat/fsmonitor/fsmonitor-fs-listen-macos.c)
    ++	list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
      endif()
      
      set(EXE_EXTENSION ${CMAKE_EXECUTABLE_SUFFIX})
14:  4be31abcdda ! 13:  7215c5ad3dc fsmonitor--daemon: implement daemon command options
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor--daemon: implement daemon command options
    +    fsmonitor--daemon: implement 'run' command
     
    -    Implement `run` and `start` commands to try to
    -    begin listening for file system events.
    +    Implement `run` command to try to begin listening for file system events.
     
    -    This version defines the thread structure with a single
    -    fsmonitor_fs_listen thread to watch for file system events
    -    and a simple IPC thread pool to wait for connections from
    -    Git clients over a well-known named pipe or Unix domain
    -    socket.
    +    This version defines the thread structure with a single fsmonitor_fs_listen
    +    thread to watch for file system events and a simple IPC thread pool to
    +    watch for connection from Git clients over a well-known named pipe or
    +    Unix domain socket.
     
    -    This version does not actually do anything yet because the
    +    This commit does not actually do anything yet because the platform
         backends are still just stubs.
     
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    @@ builtin/fsmonitor--daemon.c
      #include "parse-options.h"
      #include "fsmonitor.h"
      #include "fsmonitor-ipc.h"
    -+#include "compat/fsmonitor/fsmonitor-fs-listen.h"
    ++#include "compat/fsmonitor/fsm-listen.h"
     +#include "fsmonitor--daemon.h"
      #include "simple-ipc.h"
      #include "khash.h"
      
      static const char * const builtin_fsmonitor__daemon_usage[] = {
    -+	N_("git fsmonitor--daemon start [<options>]"),
     +	N_("git fsmonitor--daemon run [<options>]"),
      	N_("git fsmonitor--daemon stop"),
      	N_("git fsmonitor--daemon status"),
    @@ builtin/fsmonitor--daemon.c
     +#define FSMONITOR__IPC_THREADS "fsmonitor.ipcthreads"
     +static int fsmonitor__ipc_threads = 8;
     +
    -+#define FSMONITOR__START_TIMEOUT "fsmonitor.starttimeout"
    -+static int fsmonitor__start_timeout_sec = 60;
    -+
     +static int fsmonitor_config(const char *var, const char *value, void *cb)
     +{
     +	if (!strcmp(var, FSMONITOR__IPC_THREADS)) {
    @@ builtin/fsmonitor--daemon.c
     +		return 0;
     +	}
     +
    -+	if (!strcmp(var, FSMONITOR__START_TIMEOUT)) {
    -+		int i = git_config_int(var, value);
    -+		if (i < 0)
    -+			return error(_("value of '%s' out of range: %d"),
    -+				     FSMONITOR__START_TIMEOUT, i);
    -+		fsmonitor__start_timeout_sec = i;
    -+		return 0;
    -+	}
    -+
     +	return git_default_config(var, value, cb);
     +}
     +
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +	return result;
     +}
     +
    -+static void *fsmonitor_fs_listen__thread_proc(void *_state)
    ++static void *fsm_listen__thread_proc(void *_state)
     +{
     +	struct fsmonitor_daemon_state *state = _state;
     +
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +		trace_printf_key(&trace_fsmonitor, "Watching: gitdir '%s'",
     +				 state->path_gitdir_watch.buf);
     +
    -+	fsmonitor_fs_listen__loop(state);
    ++	fsm_listen__loop(state);
     +
     +	trace2_thread_exit();
     +	return NULL;
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +	 * events.
     +	 */
     +	if (pthread_create(&state->listener_thread, NULL,
    -+			   fsmonitor_fs_listen__thread_proc, state) < 0) {
    ++			   fsm_listen__thread_proc, state) < 0) {
     +		ipc_server_stop_async(state->ipc_server_data);
     +		ipc_server_await(state->ipc_server_data);
     +
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +	 * event from the IPC thread pool, but it doesn't hurt to tell
     +	 * it again.  And wait for it to shutdown.
     +	 */
    -+	fsmonitor_fs_listen__stop_async(state);
    ++	fsm_listen__stop_async(state);
     +	pthread_join(state->listener_thread, NULL);
     +
     +	return state->error_code;
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +	state.nr_paths_watching = 1;
     +
     +	/*
    -+	 * We create/delete cookie files inside the .git directory to
    -+	 * help us keep sync with the file system.  If ".git" is not a
    -+	 * directory, then <gitdir> is not inside the cone of
    -+	 * <worktree-root>, so set up a second watch for it.
    ++	 * We create and delete cookie files somewhere inside the .git
    ++	 * directory to help us keep sync with the file system.  If
    ++	 * ".git" is not a directory, then <gitdir> is not inside the
    ++	 * cone of <worktree-root>, so set up a second watch to watch
    ++	 * the <gitdir> so that we get events for the cookie files.
     +	 */
     +	strbuf_init(&state.path_gitdir_watch, 0);
     +	strbuf_addbuf(&state.path_gitdir_watch, &state.path_worktree_watch);
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +	 * Confirm that we can create platform-specific resources for the
     +	 * filesystem listener before we bother starting all the threads.
     +	 */
    -+	if (fsmonitor_fs_listen__ctor(&state)) {
    ++	if (fsm_listen__ctor(&state)) {
     +		err = error(_("could not initialize listener thread"));
     +		goto done;
     +	}
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +
     +done:
     +	pthread_mutex_destroy(&state.main_lock);
    -+	fsmonitor_fs_listen__dtor(&state);
    ++	fsm_listen__dtor(&state);
     +
     +	ipc_server_free(state.ipc_server_data);
     +
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +	 * common error case.
     +	 */
     +	if (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING)
    -+		die("fsmonitor--daemon is already running.");
    -+
    -+	return !!fsmonitor_run_daemon();
    -+}
    -+
    -+#ifndef GIT_WINDOWS_NATIVE
    -+/*
    -+ * This is adapted from `daemonize()`.  Use `fork()` to directly create
    -+ * and run the daemon in a child process.  The fork-parent returns the
    -+ * child PID so that we can wait for the child to startup before exiting.
    -+ */
    -+static int spawn_background_fsmonitor_daemon(pid_t *pid)
    -+{
    -+	*pid = fork();
    -+
    -+	switch (*pid) {
    -+	case 0:
    -+		if (setsid() == -1)
    -+			error_errno(_("setsid failed"));
    -+		close(0);
    -+		close(1);
    -+		close(2);
    -+		sanitize_stdfds();
    -+
    -+		return !!fsmonitor_run_daemon();
    -+
    -+	case -1:
    -+		return error_errno(_("could not spawn fsmonitor--daemon in the background"));
    -+
    -+	default:
    -+		return 0;
    -+	}
    -+}
    -+#else
    -+/*
    -+ * Conceptually like `daemonize()` but different because Windows does not
    -+ * have `fork(2)`.  Spawn a normal Windows child process but without the
    -+ * limitations of `start_command()` and `finish_command()`.
    -+ */
    -+static int spawn_background_fsmonitor_daemon(pid_t *pid)
    -+{
    -+	char git_exe[MAX_PATH];
    -+	struct strvec args = STRVEC_INIT;
    -+	int in, out;
    -+
    -+	GetModuleFileNameA(NULL, git_exe, MAX_PATH);
    -+
    -+	in = open("/dev/null", O_RDONLY);
    -+	out = open("/dev/null", O_WRONLY);
    -+
    -+	strvec_push(&args, git_exe);
    -+	strvec_push(&args, "fsmonitor--daemon");
    -+	strvec_push(&args, "run");
    ++		die("fsmonitor--daemon is already running '%s'",
    ++		    the_repository->worktree);
     +
    -+	*pid = mingw_spawnvpe(args.v[0], args.v, NULL, NULL, in, out, out);
    -+	close(in);
    -+	close(out);
    ++	printf(_("running fsmonitor-daemon in '%s'\n"),
    ++	       the_repository->worktree);
    ++	fflush(stdout);
     +
    -+	strvec_clear(&args);
    -+
    -+	if (*pid < 0)
    -+		return error(_("could not spawn fsmonitor--daemon in the background"));
    -+
    -+	return 0;
    -+}
    -+#endif
    -+
    -+/*
    -+ * This is adapted from `wait_or_whine()`.  Watch the child process and
    -+ * let it get started and begin listening for requests on the socket
    -+ * before reporting our success.
    -+ */
    -+static int wait_for_background_startup(pid_t pid_child)
    -+{
    -+	int status;
    -+	pid_t pid_seen;
    -+	enum ipc_active_state s;
    -+	time_t time_limit, now;
    -+
    -+	time(&time_limit);
    -+	time_limit += fsmonitor__start_timeout_sec;
    -+
    -+	for (;;) {
    -+		pid_seen = waitpid(pid_child, &status, WNOHANG);
    -+
    -+		if (pid_seen == -1)
    -+			return error_errno(_("waitpid failed"));
    -+		else if (pid_seen == 0) {
    -+			/*
    -+			 * The child is still running (this should be
    -+			 * the normal case).  Try to connect to it on
    -+			 * the socket and see if it is ready for
    -+			 * business.
    -+			 *
    -+			 * If there is another daemon already running,
    -+			 * our child will fail to start (possibly
    -+			 * after a timeout on the lock), but we don't
    -+			 * care (who responds) if the socket is live.
    -+			 */
    -+			s = fsmonitor_ipc__get_state();
    -+			if (s == IPC_STATE__LISTENING)
    -+				return 0;
    -+
    -+			time(&now);
    -+			if (now > time_limit)
    -+				return error(_("fsmonitor--daemon not online yet"));
    -+		} else if (pid_seen == pid_child) {
    -+			/*
    -+			 * The new child daemon process shutdown while
    -+			 * it was starting up, so it is not listening
    -+			 * on the socket.
    -+			 *
    -+			 * Try to ping the socket in the odd chance
    -+			 * that another daemon started (or was already
    -+			 * running) while our child was starting.
    -+			 *
    -+			 * Again, we don't care who services the socket.
    -+			 */
    -+			s = fsmonitor_ipc__get_state();
    -+			if (s == IPC_STATE__LISTENING)
    -+				return 0;
    -+
    -+			/*
    -+			 * We don't care about the WEXITSTATUS() nor
    -+			 * any of the WIF*(status) values because
    -+			 * `cmd_fsmonitor__daemon()` does the `!!result`
    -+			 * trick on all function return values.
    -+			 *
    -+			 * So it is sufficient to just report the
    -+			 * early shutdown as an error.
    -+			 */
    -+			return error(_("fsmonitor--daemon failed to start"));
    -+		} else
    -+			return error(_("waitpid is confused"));
    -+	}
    -+}
    -+
    -+static int try_to_start_background_daemon(void)
    -+{
    -+	pid_t pid_child;
    -+	int ret;
    -+
    -+	/*
    -+	 * Before we try to create a background daemon process, see
    -+	 * if a daemon process is already listening.  This makes it
    -+	 * easier for us to report an already-listening error to the
    -+	 * console, since our spawn/daemon can only report the success
    -+	 * of creating the background process (and not whether it
    -+	 * immediately exited).
    -+	 */
    -+	if (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING)
    -+		die("fsmonitor--daemon is already running.");
    -+
    -+	/*
    -+	 * Run the actual daemon in a background process.
    -+	 */
    -+	ret = spawn_background_fsmonitor_daemon(&pid_child);
    -+	if (pid_child <= 0)
    -+		return ret;
    -+
    -+	/*
    -+	 * Wait (with timeout) for the background child process get
    -+	 * started and begin listening on the socket/pipe.  This makes
    -+	 * the "start" command more synchronous and more reliable in
    -+	 * tests.
    -+	 */
    -+	ret = wait_for_background_startup(pid_child);
    -+
    -+	return ret;
    ++	return !!fsmonitor_run_daemon();
     +}
     +
      int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix)
    @@ builtin/fsmonitor--daemon.c: static int do_as_client__status(void)
     +		OPT_INTEGER(0, "ipc-threads",
     +			    &fsmonitor__ipc_threads,
     +			    N_("use <n> ipc worker threads")),
    -+		OPT_INTEGER(0, "start-timeout",
    -+			    &fsmonitor__start_timeout_sec,
    -+			    N_("Max seconds to wait for background daemon startup")),
    -+
      		OPT_END()
      	};
      
    @@ builtin/fsmonitor--daemon.c: int cmd_fsmonitor__daemon(int argc, const char **ar
     +		die(_("invalid 'ipc-threads' value (%d)"),
     +		    fsmonitor__ipc_threads);
     +
    -+	if (!strcmp(subcmd, "start"))
    -+		return !!try_to_start_background_daemon();
    -+
     +	if (!strcmp(subcmd, "run"))
     +		return !!try_to_run_foreground_daemon();
      
 -:  ----------- > 14:  f7a8c02b9a7 fsmonitor--daemon: implement 'start' command
 -:  ----------- > 15:  6e796211dc9 fsmonitor--daemon: do not try to operate on bare repos
15:  90ea7a3520e ! 16:  e43cdc6559f fsmonitor--daemon: add pathname classification
    @@ builtin/fsmonitor--daemon.c: static int handle_client(void *data,
     +	return fsmonitor_classify_path_gitdir_relative(rel);
     +}
     +
    - static void *fsmonitor_fs_listen__thread_proc(void *_state)
    + static void *fsm_listen__thread_proc(void *_state)
      {
      	struct fsmonitor_daemon_state *state = _state;
     
16:  db517a09694 = 17:  89f8b1fa14b fsmonitor--daemon: define token-ids
17:  1f06639dd22 ! 18:  30c5a1a8c26 fsmonitor--daemon: create token-based changed path cache
    @@ builtin/fsmonitor--daemon.c: enum fsmonitor_path_type fsmonitor_classify_path_ab
     +	pthread_mutex_unlock(&state->main_lock);
     +}
     +
    - static void *fsmonitor_fs_listen__thread_proc(void *_state)
    + static void *fsm_listen__thread_proc(void *_state)
      {
      	struct fsmonitor_daemon_state *state = _state;
    -@@ builtin/fsmonitor--daemon.c: static void *fsmonitor_fs_listen__thread_proc(void *_state)
    +@@ builtin/fsmonitor--daemon.c: static void *fsm_listen__thread_proc(void *_state)
      
    - 	fsmonitor_fs_listen__loop(state);
    + 	fsm_listen__loop(state);
      
     +	pthread_mutex_lock(&state->main_lock);
     +	if (state->current_token_data &&
18:  187cd52cb1e ! 19:  95ec3cba77d fsmonitor-fs-listen-win32: implement FSMonitor backend on Windows
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor-fs-listen-win32: implement FSMonitor backend on Windows
    +    fsm-listen-win32: implement FSMonitor backend on Windows
     
         Teach the win32 backend to register a watch on the working tree
         root directory (recursively).  Also watch the <gitdir> if it is
    @@ Commit message
     
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
     
    - ## compat/fsmonitor/fsmonitor-fs-listen-win32.c ##
    + ## compat/fsmonitor/fsm-listen-win32.c ##
     @@
      #include "config.h"
      #include "fsmonitor.h"
    - #include "fsmonitor-fs-listen.h"
    + #include "fsm-listen.h"
     +#include "fsmonitor--daemon.h"
     +
     +/*
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +	return strbuf_normalize_path(normalized_path);
     +}
      
    - void fsmonitor_fs_listen__stop_async(struct fsmonitor_daemon_state *state)
    + void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
      {
     +	SetEvent(state->backend_data->hListener[LISTENER_SHUTDOWN]);
     +}
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +	HANDLE hDir;
     +	wchar_t wpath[MAX_PATH];
     +
    -+	if (xutftowcs_long_path(wpath, path) < 0) {
    ++	if (xutftowcs_path(wpath, path) < 0) {
     +		error(_("could not convert to wide characters: '%s'"), path);
     +		return NULL;
     +	}
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +	memset(&watch->overlapped, 0, sizeof(watch->overlapped));
     +	watch->overlapped.hEvent = watch->hEvent;
     +
    -+start_watch:
     +	/*
     +	 * Queue an async call using Overlapped IO.  This returns immediately.
     +	 * Our event handle will be signalled when the real result is available.
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +		watch->hDir, watch->buffer, watch->buf_len, TRUE,
     +		dwNotifyFilter, &watch->count, &watch->overlapped, NULL);
     +
    -+	/*
    -+	 * The kernel throws an invalid parameter error when our buffer
    -+	 * is too big and we are pointed at a remote directory (and possibly
    -+	 * for other reasons).  Quietly set it down and try again.
    -+	 *
    -+	 * See note about MAX_RDCW_BUF at the top.
    -+	 */
    -+	if (!watch->is_active &&
    -+	    GetLastError() == ERROR_INVALID_PARAMETER &&
    -+	    watch->buf_len > MAX_RDCW_BUF_FALLBACK) {
    -+		watch->buf_len = MAX_RDCW_BUF_FALLBACK;
    -+		goto start_watch;
    -+	}
    -+
     +	if (watch->is_active)
     +		return 0;
     +
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +
     +static int recv_rdcw_watch(struct one_watch *watch)
     +{
    ++	DWORD gle;
    ++
     +	watch->is_active = FALSE;
     +
     +	/*
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +				TRUE))
     +		return 0;
     +
    ++	gle = GetLastError();
    ++	if (gle == ERROR_INVALID_PARAMETER &&
    ++	    /*
    ++	     * The kernel throws an invalid parameter error when our
    ++	     * buffer is too big and we are pointed at a remote
    ++	     * directory (and possibly for other reasons).  Quietly
    ++	     * set it down and try again.
    ++	     *
    ++	     * See note about MAX_RDCW_BUF at the top.
    ++	     */
    ++	    watch->buf_len > MAX_RDCW_BUF_FALLBACK) {
    ++		watch->buf_len = MAX_RDCW_BUF_FALLBACK;
    ++		return -2;
    ++	}
    ++
     +	/*
     +	 * NEEDSWORK: If an external <gitdir> is deleted, the above
     +	 * returns an error.  I'm not sure that there's anything that
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +	 */
     +
     +	error("GetOverlappedResult failed on '%s' [GLE %ld]",
    -+	      watch->path.buf, GetLastError());
    ++	      watch->path.buf, gle);
     +	return -1;
     +}
     +
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +
     +		t = fsmonitor_classify_path_gitdir_relative(path.buf);
     +
    -+		trace_printf_key(&trace_fsmonitor, "BBB: %s", path.buf);
    -+
     +		switch (t) {
     +		case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
     +			/* special case cookie files within gitdir */
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +	return LISTENER_HAVE_DATA_GITDIR;
      }
      
    - void fsmonitor_fs_listen__loop(struct fsmonitor_daemon_state *state)
    + void fsm_listen__loop(struct fsmonitor_daemon_state *state)
      {
     +	struct fsmonitor_daemon_backend_data *data = state->backend_data;
     +	DWORD dwWait;
    ++	int result;
     +
     +	state->error_code = 0;
     +
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +						FALSE, INFINITE);
     +
     +		if (dwWait == WAIT_OBJECT_0 + LISTENER_HAVE_DATA_WORKTREE) {
    -+			if (recv_rdcw_watch(data->watch_worktree) == -1)
    ++			result = recv_rdcw_watch(data->watch_worktree);
    ++			if (result == -1) {
    ++				/* hard error */
     +				goto force_error_stop;
    ++			}
    ++			if (result == -2) {
    ++				/* retryable error */
    ++				if (start_rdcw_watch(data, data->watch_worktree) == -1)
    ++					goto force_error_stop;
    ++				continue;
    ++			}
    ++
    ++			/* have data */
     +			if (process_worktree_events(state) == LISTENER_SHUTDOWN)
     +				goto force_shutdown;
     +			if (start_rdcw_watch(data, data->watch_worktree) == -1)
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +		}
     +
     +		if (dwWait == WAIT_OBJECT_0 + LISTENER_HAVE_DATA_GITDIR) {
    -+			if (recv_rdcw_watch(data->watch_gitdir) == -1)
    ++			result = recv_rdcw_watch(data->watch_gitdir);
    ++			if (result == -1) {
    ++				/* hard error */
     +				goto force_error_stop;
    ++			}
    ++			if (result == -2) {
    ++				/* retryable error */
    ++				if (start_rdcw_watch(data, data->watch_gitdir) == -1)
    ++					goto force_error_stop;
    ++				continue;
    ++			}
    ++
    ++			/* have data */
     +			if (process_gitdir_events(state) == LISTENER_SHUTDOWN)
     +				goto force_shutdown;
     +			if (start_rdcw_watch(data, data->watch_gitdir) == -1)
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
     +	cancel_rdcw_watch(data->watch_gitdir);
      }
      
    - int fsmonitor_fs_listen__ctor(struct fsmonitor_daemon_state *state)
    + int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
      {
     +	struct fsmonitor_daemon_backend_data *data;
     +
    @@ compat/fsmonitor/fsmonitor-fs-listen-win32.c
      	return -1;
      }
      
    - void fsmonitor_fs_listen__dtor(struct fsmonitor_daemon_state *state)
    + void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
      {
     +	struct fsmonitor_daemon_backend_data *data;
     +
19:  eef6a035eaa ! 20:  0cb240790ad fsmonitor-fs-listen-macos: add macos header files for FSEvent
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor-fs-listen-macos: add macos header files for FSEvent
    +    fsm-listen-darwin: add macos header files for FSEvent
     
         Include MacOS system declarations to allow us to use FSEvent and
         CoreFoundation APIs.  We need GCC and clang versions because of
    @@ Commit message
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
     
    - ## compat/fsmonitor/fsmonitor-fs-listen-macos.c ##
    + ## compat/fsmonitor/fsm-listen-darwin.c ##
     @@
     +#if defined(__GNUC__)
     +/*
    @@ compat/fsmonitor/fsmonitor-fs-listen-macos.c
     +
      #include "cache.h"
      #include "fsmonitor.h"
    - #include "fsmonitor-fs-listen.h"
    + #include "fsm-listen.h"
20:  a70e9e4a257 ! 21:  f8ef89c867f fsmonitor-fs-listen-macos: implement FSEvent listener on MacOS
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    fsmonitor-fs-listen-macos: implement FSEvent listener on MacOS
    +    fsm-listen-darwin: implement FSEvent listener on MacOS
     
         Implement file system event listener on MacOS using FSEvent,
         CoreFoundation, and CoreServices.
    @@ Commit message
         Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de>
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
     
    - ## compat/fsmonitor/fsmonitor-fs-listen-macos.c ##
    -@@ compat/fsmonitor/fsmonitor-fs-listen-macos.c: void FSEventStreamRelease(FSEventStreamRef stream);
    + ## compat/fsmonitor/fsm-listen-darwin.c ##
    +@@ compat/fsmonitor/fsm-listen-darwin.c: void FSEventStreamRelease(FSEventStreamRef stream);
      #include "cache.h"
      #include "fsmonitor.h"
    - #include "fsmonitor-fs-listen.h"
    + #include "fsm-listen.h"
     +#include "fsmonitor--daemon.h"
     +
     +struct fsmonitor_daemon_backend_data
    @@ compat/fsmonitor/fsmonitor-fs-listen-macos.c: void FSEventStreamRelease(FSEventS
     + * https://developer.apple.com/documentation/coreservices/1443980-fseventstreamcreate
     + */
      
    - int fsmonitor_fs_listen__ctor(struct fsmonitor_daemon_state *state)
    + int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
      {
     +	FSEventStreamCreateFlags flags = kFSEventStreamCreateFlagNoDefer |
     +		kFSEventStreamCreateFlagWatchRoot |
    @@ compat/fsmonitor/fsmonitor-fs-listen-macos.c: void FSEventStreamRelease(FSEventS
      	return -1;
      }
      
    - void fsmonitor_fs_listen__dtor(struct fsmonitor_daemon_state *state)
    + void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
      {
     +	struct fsmonitor_daemon_backend_data *data;
     +
    @@ compat/fsmonitor/fsmonitor-fs-listen-macos.c: void FSEventStreamRelease(FSEventS
     +	FREE_AND_NULL(state->backend_data);
      }
      
    - void fsmonitor_fs_listen__stop_async(struct fsmonitor_daemon_state *state)
    + void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
      {
     +	struct fsmonitor_daemon_backend_data *data;
     +
    @@ compat/fsmonitor/fsmonitor-fs-listen-macos.c: void FSEventStreamRelease(FSEventS
     +	CFRunLoopStop(data->rl);
      }
      
    - void fsmonitor_fs_listen__loop(struct fsmonitor_daemon_state *state)
    + void fsm_listen__loop(struct fsmonitor_daemon_state *state)
      {
     +	struct fsmonitor_daemon_backend_data *data;
     +
21:  f2650302b50 = 22:  c7190622c76 fsmonitor--daemon: implement handle_client callback
 -:  ----------- > 23:  079e66ea828 t/helper/test-chmtime: skip directories on Windows
 -:  ----------- > 24:  15b84ce2c4c t/perf/p7519: speed up test on Windows
29:  169e5592d27 ! 25:  e2756eed26c t/perf: avoid copying builtin fsmonitor files into test repo
    @@ Metadata
      ## Commit message ##
         t/perf: avoid copying builtin fsmonitor files into test repo
     
    -    Do not try to copy a fsmonitor--daemon socket from the current
    -    development directory into the test trash directory.
    +    Do not copy any of the various fsmonitor--daemon files from the .git
    +    directory of the (GIT_PREF_REPO or GIT_PERF_LARGE_REPO) source repo
    +    into the test's trash directory.
     
    -    When we run the perf suite without an explicit source repo set,
    -    we copy of the current $GIT_DIR into the test trash directory.
    -    Unix domain sockets cannot be copied in that manner, so the test
    -    setup fails.
    +    When perf tests start, they copy the contents of the source repo into
    +    the test's trash directory.  If fsmonitor is running in the source repo,
    +    there may be control files, such as the IPC socket and/or fsmonitor
    +    cookie files.  These should not be copied into the test repo.
     
    -    Additionally, omit any other fsmonitor--daemon temp files inside
    -    the $GIT_DIR directory.
    +    Unix domain sockets cannot be copied in the manner used by the test
    +    setup, so if present, the test setup fails.
    +
    +    Cookie files are harmless, but we should avoid them.
    +
    +    The builtin fsmonitor keeps all such control files/sockets in
    +    .git/fsmonitor--daemon*, so it is simple to exclude them.
     
         Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
     
27:  b2ba311e6c3 ! 26:  4260225c2c0 p7519: add fsmonitor--daemon
    @@ Metadata
     Author: Jeff Hostetler <jeffhost@microsoft.com>
     
      ## Commit message ##
    -    p7519: add fsmonitor--daemon
    +    t/perf/p7519: add fsmonitor--daemon test cases
     
         Repeat all of the fsmonitor perf tests using `git fsmonitor--daemon` and
         the "Simple IPC" interface.
    @@ t/perf/p7519-fsmonitor.sh: test_expect_success "setup without fsmonitor" '
     +# Explicitly start the daemon here and before we start client commands
     +# so that we can later add custom tracing.
     +#
    -+
    -+test_lazy_prereq HAVE_FSMONITOR_DAEMON '
    -+	git version --build-options | grep "feature:" | grep "fsmonitor--daemon"
    -+'
    -+
    -+if test_have_prereq HAVE_FSMONITOR_DAEMON
    ++if test_have_prereq FSMONITOR_DAEMON
     +then
     +	USE_FSMONITOR_DAEMON=t
     +
26:  7e4dc68016f ! 27:  b23d8be6b1e t7527: create test for fsmonitor--daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +
     +. ./test-lib.sh
     +
    -+git version --build-options | grep "feature:" | grep "fsmonitor--daemon" || {
    -+	skip_all="The built-in FSMonitor is not supported on this platform"
    ++if ! test_have_prereq FSMONITOR_DAEMON
    ++then
    ++	skip_all="fsmonitor--daemon is not supported on this platform"
     +	test_done
    -+}
    ++fi
     +
    -+kill_repo () {
    ++stop_daemon_delete_repo () {
     +	r=$1
     +	git -C $r fsmonitor--daemon stop >/dev/null 2>/dev/null
     +	rm -rf $1
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	return 0
     +}
     +
    ++# Is a Trace2 data event present with the given catetory and key?
    ++# We do not care what the value is.
    ++#
    ++have_t2_data_event () {
    ++	c=$1
    ++	k=$2
    ++
    ++	grep -e '"event":"data".*"category":"'"$c"'".*"key":"'"$k"'"'
    ++}
    ++
     +test_expect_success 'explicit daemon start and stop' '
    -+	test_when_finished "kill_repo test_explicit" &&
    ++	test_when_finished "stop_daemon_delete_repo test_explicit" &&
     +
     +	git init test_explicit &&
     +	start_daemon test_explicit &&
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'implicit daemon start' '
    -+	test_when_finished "kill_repo test_implicit" &&
    ++	test_when_finished "stop_daemon_delete_repo test_implicit" &&
     +
     +	git init test_implicit &&
     +	test_must_fail git -C test_implicit fsmonitor--daemon status &&
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	# but this test case is only concerned with whether the daemon was
     +	# implicitly started.)
     +
    -+	GIT_TRACE2_EVENT="$PWD/.git/trace" \
    ++	GIT_TRACE2_EVENT="$(pwd)/.git/trace" \
     +		test-tool -C test_implicit fsmonitor-client query --token 0 >actual &&
     +	nul_to_q <actual >actual.filtered &&
     +	grep "builtin:" actual.filtered &&
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	# dependent, just confirm that the foreground command received a
     +	# response from the daemon.
     +
    -+	grep :\"query/response-length\" .git/trace &&
    ++	have_t2_data_event fsm_client query/response-length <.git/trace &&
     +
     +	git -C test_implicit fsmonitor--daemon status &&
     +	git -C test_implicit fsmonitor--daemon stop &&
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'implicit daemon stop (delete .git)' '
    -+	test_when_finished "kill_repo test_implicit_1" &&
    ++	test_when_finished "stop_daemon_delete_repo test_implicit_1" &&
     +
     +	git init test_implicit_1 &&
     +
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	# deleting the .git directory will implicitly stop the daemon.
     +	rm -rf test_implicit_1/.git &&
     +
    -+	# Create an empty .git directory so that the following Git command
    -+	# will stay relative to the `-C` directory.  Without this, the Git
    -+	# command will (override the requested -C argument) and crawl out
    -+	# to the containing Git source tree.  This would make the test
    -+	# result dependent upon whether we were using fsmonitor on our
    -+	# development worktree.
    -+
    ++	# [1] Create an empty .git directory so that the following Git
    ++	#     command will stay relative to the `-C` directory.
    ++	#
    ++	#     Without this, the Git command will override the requested
    ++	#     -C argument and crawl out to the containing Git source tree.
    ++	#     This would make the test result dependent upon whether we
    ++	#     were using fsmonitor on our development worktree.
    ++	#
     +	sleep 1 &&
     +	mkdir test_implicit_1/.git &&
     +
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'implicit daemon stop (rename .git)' '
    -+	test_when_finished "kill_repo test_implicit_2" &&
    ++	test_when_finished "stop_daemon_delete_repo test_implicit_2" &&
     +
     +	git init test_implicit_2 &&
     +
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	# renaming the .git directory will implicitly stop the daemon.
     +	mv test_implicit_2/.git test_implicit_2/.xxx &&
     +
    -+	# Create an empty .git directory so that the following Git command
    -+	# will stay relative to the `-C` directory.  Without this, the Git
    -+	# command will (override the requested -C argument) and crawl out
    -+	# to the containing Git source tree.  This would make the test
    -+	# result dependent upon whether we were using fsmonitor on our
    -+	# development worktree.
    -+
    ++	# See [1] above.
    ++	#
     +	sleep 1 &&
     +	mkdir test_implicit_2/.git &&
     +
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'cannot start multiple daemons' '
    -+	test_when_finished "kill_repo test_multiple" &&
    ++	test_when_finished "stop_daemon_delete_repo test_multiple" &&
     +
     +	git init test_multiple &&
     +
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	test_must_fail git -C test_multiple fsmonitor--daemon status
     +'
     +
    ++# These tests use the main repo in the trash directory
    ++
     +test_expect_success 'setup' '
     +	>tracked &&
     +	>modified &&
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	git config core.useBuiltinFSMonitor true
     +'
     +
    ++# The test already explicitly stopped (or tried to stop) the daemon.
    ++# This is here in case something else fails first.
    ++#
    ++redundant_stop_daemon () {
    ++	git fsmonitor--daemon stop
    ++	return 0
    ++}
    ++
     +test_expect_success 'update-index implicitly starts daemon' '
    ++	test_when_finished redundant_stop_daemon &&
    ++
     +	test_must_fail git fsmonitor--daemon status &&
     +
    -+	GIT_TRACE2_EVENT="$PWD/.git/trace_implicit_1" \
    ++	GIT_TRACE2_EVENT="$(pwd)/.git/trace_implicit_1" \
     +		git update-index --fsmonitor &&
     +
     +	git fsmonitor--daemon status &&
     +	test_might_fail git fsmonitor--daemon stop &&
     +
    -+	grep \"event\":\"start\".*\"fsmonitor--daemon\" .git/trace_implicit_1
    ++	# Confirm that the trace2 log contains a record of the
    ++	# daemon starting.
    ++	test_subcommand git fsmonitor--daemon start <.git/trace_implicit_1
     +'
     +
     +test_expect_success 'status implicitly starts daemon' '
    ++	test_when_finished redundant_stop_daemon &&
    ++
     +	test_must_fail git fsmonitor--daemon status &&
     +
    -+	GIT_TRACE2_EVENT="$PWD/.git/trace_implicit_2" \
    ++	GIT_TRACE2_EVENT="$(pwd)/.git/trace_implicit_2" \
     +		git status >actual &&
     +
     +	git fsmonitor--daemon status &&
     +	test_might_fail git fsmonitor--daemon stop &&
     +
    -+	grep \"event\":\"start\".*\"fsmonitor--daemon\" .git/trace_implicit_2
    ++	# Confirm that the trace2 log contains a record of the
    ++	# daemon starting.
    ++	test_subcommand git fsmonitor--daemon start <.git/trace_implicit_2
     +'
     +
     +edit_files() {
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +}
     +
     +test_expect_success 'edit some files' '
    -+	test_when_finished "clean_up_repo_and_stop_daemon" &&
    ++	test_when_finished clean_up_repo_and_stop_daemon &&
     +
     +	(
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'create some files' '
    -+	test_when_finished "clean_up_repo_and_stop_daemon" &&
    ++	test_when_finished clean_up_repo_and_stop_daemon &&
     +
     +	(
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'delete some files' '
    -+	test_when_finished "clean_up_repo_and_stop_daemon" &&
    ++	test_when_finished clean_up_repo_and_stop_daemon &&
     +
     +	(
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'rename some files' '
    -+	test_when_finished "clean_up_repo_and_stop_daemon" &&
    ++	test_when_finished clean_up_repo_and_stop_daemon &&
     +
     +	(
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'rename directory' '
    -+	test_when_finished "clean_up_repo_and_stop_daemon" &&
    ++	test_when_finished clean_up_repo_and_stop_daemon &&
     +
     +	(
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'file changes to directory' '
    -+	test_when_finished "clean_up_repo_and_stop_daemon" &&
    ++	test_when_finished clean_up_repo_and_stop_daemon &&
     +
     +	(
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +'
     +
     +test_expect_success 'directory changes to a file' '
    -+	test_when_finished "clean_up_repo_and_stop_daemon" &&
    ++	test_when_finished clean_up_repo_and_stop_daemon &&
     +
     +	(
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +# "flush" message to a running daemon and ask it to do a flush/resync.
     +
     +test_expect_success 'flush cached data' '
    -+	test_when_finished "kill_repo test_flush" &&
    ++	test_when_finished "stop_daemon_delete_repo test_flush" &&
     +
     +	git init test_flush &&
     +
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +		GIT_TEST_FSMONITOR_TOKEN=true &&
     +		export GIT_TEST_FSMONITOR_TOKEN &&
     +
    -+		GIT_TRACE_FSMONITOR="$PWD/.git/trace_daemon" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/.git/trace_daemon" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon test_flush
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	git -C wt-base worktree add ../wt-secondary &&
     +
     +	(
    -+		GIT_TRACE2_PERF="$PWD/trace2_wt_secondary" &&
    ++		GIT_TRACE2_PERF="$(pwd)/trace2_wt_secondary" &&
     +		export GIT_TRACE2_PERF &&
     +
    -+		GIT_TRACE_FSMONITOR="$PWD/trace_wt_secondary" &&
    ++		GIT_TRACE_FSMONITOR="$(pwd)/trace_wt_secondary" &&
     +		export GIT_TRACE_FSMONITOR &&
     +
     +		start_daemon wt-secondary
    @@ t/t7527-builtin-fsmonitor.sh (new)
     +	test_must_fail git -C wt-secondary fsmonitor--daemon status
     +'
     +
    ++# NEEDSWORK: Repeat one of the "edit" tests on wt-secondary and
    ++# confirm that we get the same events and behavior -- that is, that
    ++# fsmonitor--daemon correctly watches BOTH the working directory and
    ++# the external GITDIR directory and behaves the same as when ".git"
    ++# is a directory inside the working directory.
    ++
    ++test_expect_success 'cleanup worktrees' '
    ++	stop_daemon_delete_repo wt-secondary &&
    ++	stop_daemon_delete_repo wt-base
    ++'
    ++
     +test_done
22:  ae2f6ae597c ! 28:  87bf95d6e21 fsmonitor--daemon: periodically truncate list of modified files
    @@ builtin/fsmonitor--daemon.c: static void fsmonitor_batch__combine(struct fsmonit
     + * and truncate the list there.  Note that these timestamps are completely
     + * artificial (based on when we pinned the batch item) and not on any
     + * filesystem activity.
    ++ *
    ++ * Return the obsolete portion of the list after we have removed it from
    ++ * the official list so that the caller can free it after leaving the lock.
     + */
     +#define MY_TIME_DELAY_SECONDS (5 * 60) /* seconds */
     +
    -+static void with_lock__truncate_old_batches(
    ++static struct fsmonitor_batch *with_lock__truncate_old_batches(
     +	struct fsmonitor_daemon_state *state,
     +	const struct fsmonitor_batch *batch_marker)
     +{
     +	/* assert current thread holding state->main_lock */
     +
     +	const struct fsmonitor_batch *batch;
    -+	struct fsmonitor_batch *rest;
    -+	struct fsmonitor_batch *p;
    ++	struct fsmonitor_batch *remainder;
     +
     +	if (!batch_marker)
    -+		return;
    ++		return NULL;
     +
     +	trace_printf_key(&trace_fsmonitor, "Truncate: mark (%"PRIu64",%"PRIu64")",
     +			 batch_marker->batch_seq_nr,
    @@ builtin/fsmonitor--daemon.c: static void fsmonitor_batch__combine(struct fsmonit
     +		goto truncate_past_here;
     +	}
     +
    -+	return;
    ++	return NULL;
     +
     +truncate_past_here:
     +	state->current_token_data->batch_tail = (struct fsmonitor_batch *)batch;
     +
    -+	rest = ((struct fsmonitor_batch *)batch)->next;
    ++	remainder = ((struct fsmonitor_batch *)batch)->next;
     +	((struct fsmonitor_batch *)batch)->next = NULL;
     +
    -+	for (p = rest; p; p = fsmonitor_batch__pop(p)) {
    ++	return remainder;
    ++}
    ++
    ++static void free_remainder(struct fsmonitor_batch *remainder)
    ++{
    ++	struct fsmonitor_batch *p;
    ++
    ++	if (!remainder)
    ++		return;
    ++
    ++	for (p = remainder; p; p = fsmonitor_batch__pop(p)) {
     +		trace_printf_key(&trace_fsmonitor,
     +				 "Truncate: kill (%"PRIu64",%"PRIu64")",
     +				 p->batch_seq_nr, (uint64_t)p->pinned_time);
    @@ builtin/fsmonitor--daemon.c: static void fsmonitor_batch__combine(struct fsmonit
      static void fsmonitor_free_token_data(struct fsmonitor_token_data *token)
      {
      	struct fsmonitor_batch *p;
    +@@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state,
    + 	const char *p;
    + 	const struct fsmonitor_batch *batch_head;
    + 	const struct fsmonitor_batch *batch;
    ++	struct fsmonitor_batch *remainder = NULL;
    + 	intmax_t count = 0, duplicates = 0;
    + 	kh_str_t *shown;
    + 	int hash_ret;
     @@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state,
      			 * that work.
      			 */
    @@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon
     +			 * obsolete.  See if we can truncate the list
     +			 * and save some memory.
     +			 */
    -+			with_lock__truncate_old_batches(state, batch);
    ++			remainder = with_lock__truncate_old_batches(state, batch);
      		}
      	}
      
    + 	pthread_mutex_unlock(&state->main_lock);
    + 
    ++	free_remainder(remainder);
    ++
    + 	trace2_data_intmax("fsmonitor", the_repository, "response/length", total_response_len);
    + 	trace2_data_intmax("fsmonitor", the_repository, "response/count/files", count);
    + 	trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates);
23:  82ad1a8d7cc ! 29:  56056fc1f1c fsmonitor--daemon: use a cookie file to sync with file system
    @@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
     +	/*
     +	 * We will write filesystem syncing cookie files into
     +	 * <gitdir>/<fsmonitor-dir>/<cookie-dir>/<pid>-<seq>.
    ++	 *
    ++	 * The extra layers of subdirectories here keep us from
    ++	 * changing the mtime on ".git/" or ".git/foo/" when we create
    ++	 * or delete cookie files.
    ++	 *
    ++	 * There have been problems with some IDEs that do a
    ++	 * non-recursive watch of the ".git/" directory and run a
    ++	 * series of commands any time something happens.
    ++	 *
    ++	 * For example, if we place our cookie files directly in
    ++	 * ".git/" or ".git/foo/" then a `git status` (or similar
    ++	 * command) from the IDE will cause a cookie file to be
    ++	 * created in one of those dirs.  This causes the mtime of
    ++	 * those dirs to change.  This triggers the IDE's watch
    ++	 * notification.  This triggers the IDE to run those commands
    ++	 * again.  And the process repeats and the machine never goes
    ++	 * idle.
    ++	 *
    ++	 * Adding the extra layers of subdirectories prevents the
    ++	 * mtime of ".git/" and ".git/foo" from changing when a
    ++	 * cookie file is created.
     +	 */
     +	strbuf_init(&state.path_cookie_prefix, 0);
     +	strbuf_addbuf(&state.path_cookie_prefix, &state.path_gitdir_watch);
    @@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
      done:
     +	pthread_cond_destroy(&state.cookies_cond);
      	pthread_mutex_destroy(&state.main_lock);
    - 	fsmonitor_fs_listen__dtor(&state);
    + 	fsm_listen__dtor(&state);
      
     @@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
      
24:  b616b7de1e9 = 30:  1ec372a7897 fsmonitor: enhance existing comments
25:  467c99d6291 ! 31:  a7088d5ffb9 fsmonitor: force update index after large responses
    @@ fsmonitor.c: static void fsmonitor_refresh_callback(struct index_state *istate,
     +
      void refresh_fsmonitor(struct index_state *istate)
      {
    - 	struct repository *r = istate->repo ? istate->repo : the_repository;
    + 	struct strbuf query_result = STRBUF_INIT;
     @@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
      		 *
      		 * This updates both the cache-entries and the untracked-cache.
28:  93786f2ee54 ! 32:  7adc66d9d3b t7527: test status with untracked-cache and fsmonitor--daemon
    @@ t/t7527-builtin-fsmonitor.sh: test_expect_success 'setup' '
      	EOF
      
      	git -c core.useBuiltinFSMonitor= add . &&
    -@@ t/t7527-builtin-fsmonitor.sh: test_expect_success 'worktree with .git file' '
    - 	test_must_fail git -C wt-secondary fsmonitor--daemon status
    +@@ t/t7527-builtin-fsmonitor.sh: test_expect_success 'cleanup worktrees' '
    + 	stop_daemon_delete_repo wt-base
      '
      
    -+# TODO Repeat one of the "edit" tests on wt-secondary and confirm that
    -+# TODO we get the same events and behavior -- that is, that fsmonitor--daemon
    -+# TODO correctly listens to events on both the working directory and to the
    -+# TODO referenced GITDIR.
    -+
    -+test_expect_success 'cleanup worktrees' '
    -+	kill_repo wt-secondary &&
    -+	kill_repo wt-base
    -+'
    -+
     +# The next few tests perform arbitrary/contrived file operations and
     +# confirm that status is correct.  That is, that the data (or lack of
     +# data) from fsmonitor doesn't cause incorrect results.  And doesn't
 -:  ----------- > 33:  4005824c909 fsmonitor: handle shortname for .git
 -:  ----------- > 34:  67a44fb0ed9 t7527: test FS event reporing on MacOS WRT case and Unicode
 -:  ----------- > 35:  e1055f3444d t7527: test builtin FSMonitor watching repos with unicode paths
 -:  ----------- > 36:  e024c753c95 t/helper/fsmonitor-client: create stress test
 -:  ----------- > 37:  8956648b5de ipc-win32: add trace2 debugging
 -:  ----------- > 38:  ff00eee55fd fsmonitor-settings: stub in platform-specific incompatibility checking
 -:  ----------- > 39:  d2ed07d07c3 fsmonitor-settings: virtual repos are incompatible with FSMonitor
 -:  ----------- > 40:  cc72760a650 fsmonitor--daemon: background daemon must free the console on windows
 -:  ----------- > 41:  d78e5132eca fsmonitor-settings: stub in platform-specific incompatibility checking on MacOS
 -:  ----------- > 42:  2c799a2bfa3 fsmonitor-settings: remote repos on MacOS are incompatible with FSMonitor
 -:  ----------- > 43:  12625c032ab fsmonitor-settings: remote repos on Windows are incompatible with FSMonitor
30:  29746c6456c = 44:  e78851a7242 fsmonitor: mark the built-in FSMonitor as experimental
31:  69cb641bc31 ! 45:  8a9b342c87f Enable the built-in FSMonitor as an experimental feature
    @@ Commit message
         If `feature.experimental` and `feature.manyFiles` are set, we now start
         the built-in FSMonitor by default.
     
    +    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
     
      ## repo-settings.c ##
    @@ repo-settings.c
      #include "config.h"
      #include "repository.h"
      #include "midx.h"
    +-#include "compat/fsmonitor/fsm-listen.h"
     +#include "fsmonitor-ipc.h"
    ++#include "fsmonitor-settings.h"
      
      #define UPDATE_DEFAULT_BOOL(s,v) do { if (s == -1) { s = v; } } while(0)
      
    @@ repo-settings.c
      
      	if (r->settings.initialized)
     @@ repo-settings.c: void prepare_repo_settings(struct repository *r)
    - 		r->settings.use_builtin_fsmonitor = 1;
    + 	UPDATE_DEFAULT_BOOL(r->settings.core_multi_pack_index, 1);
      
      	if (!repo_config_get_bool(r, "feature.manyfiles", &value) && value) {
     +		feature_many_files = 1;
    @@ repo-settings.c: void prepare_repo_settings(struct repository *r)
     +	if (!repo_config_get_bool(r, "feature.experimental", &value) && value) {
      		UPDATE_DEFAULT_BOOL(r->settings.fetch_negotiation_algorithm, FETCH_NEGOTIATION_SKIPPING);
     +		if (feature_many_files && fsmonitor_ipc__is_supported())
    -+			UPDATE_DEFAULT_BOOL(r->settings.use_builtin_fsmonitor,
    -+					    1);
    ++			fsm_settings__set_ipc(r);
     +	}
      
      	/* Hack for test programs like test-dump-untracked-cache */

[dscho]

For good measure, I rebased the microsoft/git patches (including the Scalar Functional Tests) and pushed it so that we can benefit from the full GitHub workflow run: https://github.com/microsoft/git/actions/runs/1102914546

[dscho]

For good measure, I rebased the microsoft/git patches (including the Scalar Functional Tests) and pushed it so that we can benefit from the full GitHub workflow run: https://github.com/microsoft/git/actions/runs/1102914546

And now I actually enabled the Scalar Functional Tests for tentative/vfs-* branches (d'oh!), so here is the run: https://github.com/microsoft/git/actions/runs/1102956545

[jeffhostetler]

Sorry, but the range-diff is rather large and ugly. There was lots of bike shedding.... And I added code to handle a bunch of edge cases. I'm thinking I'll need to split V4 into 2 parts when I send the real version to the mailing list.

[dscho]

The builds all passed, including the Scalar Functional Tests one. So I am pretty confident that this works, and decided to merge it in time for -rc1.