Do not error on autoreload shutdown

[philippjfr]

Fixes #6934

[codecov]

Codecov Report

Attention: Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 81.19%. Comparing base (d06c1f2) to head (a1ac2ec).

Files	Patch %	Lines
panel/io/server.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6940      +/-   ##
==========================================
- Coverage   81.25%   81.19%   -0.06%     
==========================================
  Files         326      326              
  Lines       47944    47947       +3     
==========================================
- Hits        38958    38932      -26     
- Misses       8986     9015      +29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.