Create a well defined charset for namespaces and test them in DAT

[sherifnada]

Tell us about the problem you're trying to solve

issues like #7435 happen because we don't have clear rules for what characters should be allowed in a namespace. In addition, destinations don't have clear guidance around how to handle namespaces which contain special characters e.g: a custom namespace with a - or a space could cause a destination to fail.

Describe the solution you’d like

1. define exactly what characters / rules are allowed in a namespace (both custom and source-defined namespaces).
2. Destinations should be resilient towards all allowed characters
3. DAT should test that all destinations can handle all namespace chars. For this one it is super important that we put on the chaos monkey hat in choosing namespace values in the test to make sure the destination can handle everything we throw at it.

[tuliren]

Sprint summary

Did not have time to look into this yet.

[tuliren]

Sprint summary

• What has been done:
  • Snowflake specific namespace tests.
• What remains:
  • Need to generalize the solution.
• ETA
  • End of this week (Feb 25th).

[tuliren]

TODO

• Publish an initial rule set.

[sherifnada]

Plan

MVP

• Publish initial ruleset. This should describe which characters are allowed, how we perform namespace mappings, how we handle name conflicts if they exist, and boundary conditions (e.g: max length)
• Implement the initial rule set in an opt-in DAT testcase
• Bring Snowflake & Bigquery into compliance with it

Post-MVP

UI

• dynamically render the resulting namespace
• enforce max length limitations
• https://github.com/airbytehq/airbyte-internal-issues/issues/2844

Backend

• implement that test case for the rest of the connectors. Create an epic and handoff to GL or accept community contributions
• make the testcase opt-out in DAT

[tuliren]

Summary

• Destination identifier name spec
• DAT: PR is ready for review

[sherifnada]

Nice work @tuliren!

I imagine the next steps here are:

1. Finalize the DAT PR
2. Publish the spec in our external docs (maybe after review/ratification/etc..)
3. Bring the remaining destinations into compliance with the new Namespace spec

Does that sound right?