Improve formatting of back-end exceptions

[maxpatiiuk]

Back-end sends a stack trace properly formatted, but front-end glues all lines into one big mess. Need to wrap it in <pre></pre> to fix this. Better yet, @melton-jason is there a way to get back the original django exception page (when in development) - it was quite helpful

See #3315 (review) for another example

[melton-jason]

I believe this issue was not always the case. Some update (maybe the Django/Python update) or change in the code caused this problem.
Even now there are times when I come across the original Django error messages in Debug mode, but I am unsure exactly what causes it.

I have done some looking into this before:

The middleware is properly returning nothing (None) when Debug mode is enabled

specify7/specifyweb/middleware/general.py

Line 45 in 393538b

if not settings.DEBUG:

So Django is sending its own error message to the frontend

Perhaps changing the order in which Django handles the middleware will alleviate the problem (See the Django docs on Middleware layering) or maybe Django has since migrated the error message into another Middleware option (here is a list of all available Middleware)

Currently, our Middleware is layered as:

specify7/specifyweb/settings/__init__.py

Lines 186 to 197 in 393538b

	MIDDLEWARE = [
	# 'middleware.profilemiddleware.ProfileMiddleware',
	'django.middleware.gzip.GZipMiddleware',
	'django.contrib.sessions.middleware.SessionMiddleware',
	'django.middleware.locale.LocaleMiddleware',
	'django.middleware.common.CommonMiddleware',
	'django.middleware.csrf.CsrfViewMiddleware',
	'django.contrib.auth.middleware.AuthenticationMiddleware',
	'specifyweb.context.middleware.ContextMiddleware',
	'specifyweb.permissions.middleware.PermissionsMiddleware',
	'specifyweb.middleware.general.GeneralMiddleware',
	]

I will do some further investigation so we can get back to the Django responses!

[melton-jason]

I have done further investigation into this issue and here is what I have found so far:

• This issue is not caused by the middleware introduced in Improve error response from backend #2702
  • Completely removing the middleware and rebuilding the backend containers still exhibits the same behavior
• Templated views (like the 404 Page Not Found view) still appear as normal, without the weird formatting.

• The backend response is being parsed as HTML by the frontend. This is occurring from the following code snippet

  • specify7/specifyweb/frontend/js_src/lib/components/Errors/FormatError.tsx

    Lines 119 to 124 in a64297b

    	try {
    	const htmlElement = document.createElement('html');
    	htmlElement.innerHTML = error;
    	htmlElement.remove();
    	return <ErrorIframe>{error}</ErrorIframe>;
    	} catch {

I think that Django is not sending proper HTML to the frontend, and rather sending formatted text (with tabs and new lines). I am unsure why Django is doing this suddenly, and have had little luck thus far finding similar experiences online.

[maxpatiiuk]

Could it be because front-end uses Accept: text/plain header when it calls ajax?

[melton-jason]

Yes, I'd say that's worth investigating! I will try and take a look and see if that is the issue soon.

[grantfitzsimmons]

@melton-jason Take a look at this issue when you get a chance (no rush).

[melton-jason]

Could it be because front-end uses Accept: text/plain header when it calls ajax?

Yes, this seems to be the issue.
I did a quick testing of this by changing the Accept header of backbone ajax to be text/html, and the error message was once again properly displayed/formatted.

specify7/specifyweb/frontend/js_src/lib/utils/ajax/backboneAjax.ts

Line 53 in dfbdc2f

Accept: request.type === 'DELETE' ? 'text/plain' : 'application/json',

[emenslin]

Can recreate in edge (7.9.6)