Updates to thread affinity to allow it for ProxyMutex

[bryancall]

Using this change to track down #5943.

I would like for Continuations with thread affinity to have mutexes that also have thread affinity, but that blew up in the AIO code.

[oknet]

To make Continuation thread affinity, just use EThread::mutex as its mutex.

[oknet]

It is better to rewrite UnixNetVConnection::reenable(VIO *vio) within this PR, because it will also obtain the nh->mutex across EThread.

trafficserver/iocore/net/UnixNetVConnection.cc

Lines 778 to 834 in 0498218

	if (nh->mutex->thread_holding == t) {
	if (vio == &read.vio) {
	ep.modify(EVENTIO_READ);
	ep.refresh(EVENTIO_READ);
	if (read.triggered) {
	nh->read_ready_list.in_or_enqueue(this);
	} else {
	nh->read_ready_list.remove(this);
	}
	} else {
	ep.modify(EVENTIO_WRITE);
	ep.refresh(EVENTIO_WRITE);
	if (write.triggered) {
	nh->write_ready_list.in_or_enqueue(this);
	} else {
	nh->write_ready_list.remove(this);
	}
	}
	} else {
	MUTEX_TRY_LOCK(lock, nh->mutex, t);
	if (!lock.is_locked()) {
	if (vio == &read.vio) {
	int isin = ink_atomic_swap(&read.in_enabled_list, 1);
	if (!isin) {
	nh->read_enable_list.push(this);
	}
	} else {
	int isin = ink_atomic_swap(&write.in_enabled_list, 1);
	if (!isin) {
	nh->write_enable_list.push(this);
	}
	}
	if (likely(nh->thread)) {
	nh->thread->tail_cb->signalActivity();
	} else if (nh->trigger_event) {
	nh->trigger_event->ethread->tail_cb->signalActivity();
	}
	} else {
	if (vio == &read.vio) {
	ep.modify(EVENTIO_READ);
	ep.refresh(EVENTIO_READ);
	if (read.triggered) {
	nh->read_ready_list.in_or_enqueue(this);
	} else {
	nh->read_ready_list.remove(this);
	}
	} else {
	ep.modify(EVENTIO_WRITE);
	ep.refresh(EVENTIO_WRITE);
	if (write.triggered) {
	nh->write_ready_list.in_or_enqueue(this);
	} else {
	nh->write_ready_list.remove(this);
	}
	}
	}
	}

[zwoop]

Seems this needs more work, going to -1 it for now, and move it out to 9.1.x

[bryancall]

[approve ci]

[bryancall]

[approve ci]

Bryan to take a look.

[bryancall]

[approve ci]

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.

[bryancall]

[approve ci]