Test that applications return descriptive error messages

[jernst]

(from this discussion: https://mastodon.social/@benpate/111161976484128397 )

Just this: there are tons of ways that an API call can fail, but most Fediverse software just returns something unhelpful, like 500 Internal Server Error.

For production software, that’s probably reasonable, but a well-written test suite should help people debug, not just tell them about the bug.

So if I have a voice, a test suite should really return errors like: “this message was not accepted because it is missing a ‘name’ field.”

Not sure what exactly we can do about this here, but let's record it anyway.

Also the follow-up in the thread:

Nomad returns delivery reports indicating the disposition of a message once it hits the server and let's normal people examine the reports for any message they send. We needed this years ago because Diaspora silently dropped 1/3 of all communications and we needed to be able to prove the issue wasn't our software's fault. The assumption was that site operators of that period were often high school kids that couldn't read or interpret log files, so this let us find federation issues without involving them.

It would be no small feat getting this kind of thing standardised and adopted in ActivityPub, but we do provide feedback of the http error code for all sites -- and if something is stuck in the queue, one can see the results of the individual delivery attempts. Nomad sites provide much more detailed answers of what happened - lack of permission, filter/blocking rule, duplicate, recipient not found, delivered, etc...

[snarfed]

Duplicate of fediverse-devnet/feditest-tests-fediverse#4?

[jernst]

Duh.