Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example YAML appears to be invalid #94

Closed
kmcquade opened this issue Apr 3, 2022 · 7 comments
Closed

Example YAML appears to be invalid #94

kmcquade opened this issue Apr 3, 2022 · 7 comments

Comments

@kmcquade
Copy link

kmcquade commented Apr 3, 2022

First of all, thanks for such a cool tool! This is awesome.

Description

I was following the tutorial and tinkering a bit and realized that the example YAML output is actually invalid YAML. See the screenshot from the README of the repo below.

image

To Reproduce

It's the main README - but for the sake of conforming to the issue template 🙂 here's the command from the root of the repo:

oasdiff -base data/openapi-test1.yaml -revision data/openapi-test2.yaml

Expected behavior
I would expect the YAML to be valid. I wasn't sure if this was just a diff format thing - but I would assume that we'd want a way to generate valid YAML.

I'm excited to get this working! 😄 Cool stuff!

@reuvenharrison
Copy link
Collaborator

Hi @kmcquade.
This YAML is actually valid but not very common. You can try pasting it into a YAML validator to confirm this.

The YAML spec refers to this as a "complex map key" which we leverage here to denote an Endpoint which is a combination of a Path and a Method.

The YAML spec explains it but not very well unfortunately:
A question mark and space (“? ”) indicate a complex mapping key.

You can also see this related thread.

I hope this clarifies it.
Reuven

@kmcquade
Copy link
Author

kmcquade commented Apr 3, 2022

Oh wow I didn't know that. I guess you do learn something new every day.

I'm trying to load that YAML into Python but I don't think the Python YAML module supports this. Any suggestions? Is there a way for oasdiff to toggle that feature by chance?

@reuvenharrison
Copy link
Collaborator

This seems to work:

from pathlib import Path
import ruamel.yaml

file_name = Path('data/test.yaml')

yaml = ruamel.yaml.YAML()
data = yaml.load(file_name)
yaml.dump(data['endpoints']['modified'] , sys.stdout)

Installation:
pip install ruamel.yaml

@kmcquade
Copy link
Author

kmcquade commented Apr 4, 2022

Sorry, I should have been more specific. I'm trying to load that YAML into a python dictionary. That code snippet above loads data as a CommentedMap and the yaml.dump method prints the same YAML string.

Any ideas on reading it as a dictionary?

@reuvenharrison
Copy link
Collaborator

reuvenharrison commented Apr 4, 2022

How about this?

from yaml import load
import io
import json

try:
    from yaml import CLoader as Loader
except ImportError:
    from yaml import Loader

stream=io.open('data/test.yaml', mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True)
data = load(stream, Loader=Loader)
print(json.dumps(data, indent=4, sort_keys=True))

Edit:
I noticed that this works on some versions of PyYAML but not on others but I'm really not sure how to determine the one that is working properly.

@reuvenharrison
Copy link
Collaborator

I didn't want to leave this mystery unsolved so I did some more digging.
Apparently, it isn't the PyYAML version which determines success or failure but it's the code snippet below that does the magic. I got it from this blog.

from yaml import load
import io
import json

try:
    from yaml import CLoader as Loader
except ImportError:
    from yaml import Loader

class Pairs(list):
    def __repr__(self):
        return f"{type(self).__name__}({super().__repr__()})"

def construct_mapping(self, node):
    value = self.construct_pairs(node)
    try:
        return dict(value)
    except TypeError:
        return Pairs(value)

Loader.construct_mapping = construct_mapping
Loader.add_constructor('tag:yaml.org,2002:map', Loader.construct_mapping)

stream = io.open('data/test.yaml', mode='r', buffering=-1,
                 encoding=None, errors=None, newline=None, closefd=True)
data = load(stream, Loader=Loader)
print(json.dumps(data, indent=4, sort_keys=True))

@kmcquade
Copy link
Author

kmcquade commented Apr 4, 2022

Yes!!! Thank you so much. I never would have figured all that out on my own. You're awesome. I really appreciate it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants