[CT-562] Allow for opt in to context method failures

[emmyoop]

toyaml, fromyaml, tojson and fromjson in dbt/context/base.py currently pass back the default (or None) if the value passed in is not serializable/deserializable without ever failing.

dbt-core/core/dbt/context/base.py

Lines 455 to 458 in d1ae9dd

	try:
	return yaml.safe_dump(data=value, sort_keys=sort_keys)
	except (ValueError, yaml.YAMLError):
	return default

It should be possible to get the error out for the case when the value is unexpectedly serializable/deserializable. The existing methods should continue to function as is to prevent breaking current projects. Add new methods that will throw errors.

    @contextmember
    @staticmethod
    def try_toyaml(
        value: Any, sort_keys: bool = False
    ) -> str:
        """The `tojson` context method can be used to serialize a Python
        object primitive, eg. a `dict` or `list` to a yaml string.  If the value 
        cannot be serialized it will raise an exception.
        :param value: The value serialize to yaml
        :param sort_keys: If True, sort the keys.
        Usage:
            {% set my_dict = {"abc": 123} %}
            {% set my_yaml_string = toyaml(my_dict) %}
            {% do log(my_yaml_string) %}
        """
        try:
            return yaml.safe_dump(data=value, sort_keys=sort_keys)
        except (ValueError, yaml.YAMLError):
            raise OurSpecialException()

Open Questions

• What naming standard should we use for the new methods that will raise instead of default?
• What exception should it raise and what information should be in that exception?

[jeremyyeo]

Does a base ContextException make sense in this case? Doesn't appear as if we have such an exception.

[gshank]

I'm not convinced that ContextException is a great category. It's kind of getting into implementation details...

[jtcohen6]

The existing methods should continue to function as is to prevent breaking current projects. Add new methods that will throw errors.

I had missed this piece several months ago.

As mentioned in #5107 (comment), as a person familiar with the try_cast / safe_cast SQL functions, my intuition would be that:

• toyaml(value) will raise an exception if value cannot be converted to valid yaml
• try_toyaml(value, default=None) will return default if value cannot be converted to valid yaml — i.e. the current behavior of the toyaml method

Some options to make that switch while also preserving backwards compatibility:

• Deprecate the default argument of toyaml. (If users specify that argument, raise a deprecation warning and encourage them to switch to try_toyaml, since that's really what they're after.)
• Deprecate toyaml, in favor of to_yaml(value) and try_to_yaml(value, default=None)
• Come up with a different name, e.g. strict_toyaml, for the exception-raising version

[peterallenwebb]

People don't remember what they were even worried about. Closing.