I've done a series of experiments with ESP_SPIcontrol._wait_spi_char and now I think I thoroughly understand it.

Much of what I wrote in my note 3 days ago is correct, but I misunderstood a few points.  Here is what I have found:

Resolution of timers

I measured the resolution of both time.monotonic() and time.monotonic_ns() which is needed to actually understand the results I found.
On my pyportal (and I suspect all pyportals) the resolution of time.monotonic_ns() is 122.072 microseconds.  So, successive reads of time.monotonic_ns() give a difference of either 0 ns or 122072 ns.  
The resolution of time.monotonic() is dependent on how long the pyportal has run since last reset.  In my case, as I write this, it has been running for 3.5 days and the resolution of time.monotonic() is 127.9 milliseconds.  After being up for a month, the resolution will be a full second.  time.monotonic() is really not suitable for short duration timeouts because the minimum timeout rises with elapsed time.

Modification to _wait_spi_char

I modified _wait_spi_char to use time.monotonic_ns() to do its timing so I could get finer and predictable resolution in the timing.  I kept the timeout at 0.100 seconds (10**8) nanoseconds.
I changed the code that runs if the while loop times out to simply print a message stating it 'would have' generated an exception and then just looped back to try again.  So the code looked like this:

    def _wait_spi_char(self, spi, desired):
        """Read a byte with a time-out, and if we get it, check that its what we expect"""
        for _ in range(4):
            time_start_ns = time.monotonic_ns()
            while (time.monotonic_ns() - time_start_ns) < (10**8):
                r = self._read_byte(spi)
                if r == _ERR_CMD:
                    raise RuntimeError("Error response to command")
                if r == desired:
                    return True
            print("Raise RuntimeError - Timed out waiting for SPI Char")

After that I added code to track how many times we went around the 'while' loop in each call and how many nanoseconds elapsed before we got 'r == desired'.  I couldn't capture this information for each of the millions of calls that I would generate, so I counted how many calls had elapsed times of < 1ms, < 10ms, < 100ms, <1000 ms, and kept a minimum time and maximum time for each of those buckets.  In addition I counted the number of calls that required multiple times around the 'while' loop.

Application

The application causing the calls is a program that simply connects to a socket on another system (a raspberry pi) which then exchange messages over the socket.  The vast majority of the calls to _wait_spi_char are due to the application's calls to socket.connected and socket.available, which were each called in a loop every 50 ms, looking for messages.

Data

The results of 10 hours of running:

2,385,064  calls <    1 ms duration, average 116 us,   
      616  calls <   10 ms duration, minimum 3.662 ms, maximum 3.784 ms
        0  calls <  100 ms duration
       15  calls < 1000 ms duration, minimum 172.85 ms, maximum 177.40 ms
        7  'exceptions' 
        0  number of times while loop re-executed
        
There are some very interesting things to see here.

1. The number of times the while loop re-executed was zero.  This means that each call (except those that caused 'exceptions') called _read_byte(spi) just once.  We always got the byte we expected.  It further means that the 15 times that I measured a ~175 ms duration, all happened with no more than a single call to _read_byte(spi), and even more suprisingly, it means that the 7 'exceptions' were detected at the top of the while loop on entry to the loop.  In those cases, _read_byte(spi) wasn't called at all.

2. The times measured really only had three values: about 116 us (the average of measurements that gave either zero ns or 122.072 ns), about 3.7 ms (either 3.662160 ms or 3.784232 ms measured) and about 175 ms.  There were none in between these values.  3.662160 ms is 30 times the resultion of time.monotonic_ns() and 3.784232 is 31 times.  So the actual time was always somewhere between these values.  I suspect that the 3.7  ms number represents some time in the busio code that sometimes takes a slow path (maybe a retry) of about 3.7 ms but mostly takes a fast path of about 116 us.  

I guessed that the 175 ms time was being caused by garbage collect, so to test that, I added code in _wait_spi_char to disable garbage collect before the first time measurement and re-enable it on exit.  

The results of 24 hours of running with garbage collect disabled in _wait_spi_char:

5,603,225  calls <    1 ms duration, average 111 us,
      197  calls <   10 ms duration, minimum 3.662 ms, maximum 3.784 ms
        0  calls <  100 ms duration
        0  calls < 1000 ms duration
        0  'exceptions'
        0  number of times 'while' loop re-executed

Conclusions
        
The conclusion here is that garbage collect is responsible for the 175 ms times and therefore, for all of the 'exceptions'.  From the data in the first experiment, it appears that about half the 175 ms times caused by garbage collect caused exceptions and half ended with valid data.  Because the while loop never actually looped, the 'exceptions' must have been caused by garbage collect running between the first call to time.monotonic_ns() and the start of the while loop.  The other garbage collects occurred after we were inside the while loop.  Once we entered the busio code to actually do the spi function, no garbage collects could occur.  So it isn't too surprising that half could occur right at the start.

I have seen legitimate timeouts in _wait_spi_char, but those only occurred when my code called socket.connected after inadvertently ignoring a 'ESP32 not responding' exception.  In that case,  we went around the while loop about 50 times and then signalled the exception.  After fixing my application I have seen no more of those.

I think the solution is to abandon using timing here.  Just retry, maybe three times, and then if it still doesn't give the expected answer, signal the exception.  Something like this:

    def _wait_spi_char(self, spi, desired):
        """Read a byte with a time-out, and if we get it, check that its what we expect"""
        for _ in range(3):
            r = self._read_byte(spi)
            if r == _ERR_CMD:
                raise RuntimeError("Error response to command")
            if r == desired:
                return True
        raise RuntimeError("Timed out waiting for SPI char")

This eliminates garbage collect as a factor in whether a 'timeout' occurs and eliminates the issue of using time.monotonic() for short duration timouts.
I am going to run my tests with this code for the next few days.