-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incorproate linkml-runtime, linkml-model/linkml-files.py changes from linkml-runtime #201
base: main
Are you sure you want to change the base?
Incorproate linkml-runtime, linkml-model/linkml-files.py changes from linkml-runtime #201
Conversation
…pdate in linkml-runtime stays the same
…s the change in paths to expect here. noting the 'new' paths do resolve.
GITHUB_PATH_FOR(Source.META, Format.NATIVE_JSONLD, "v0.0.1") | ||
current_loc = re.sub(r'linkml-model/[0-9a-f]*/', 'linkml-model/SHA/', GITHUB_PATH_FOR(Source.TYPES, Format.YAML)) | ||
assert "https://raw.githubusercontent.com/linkml/linkml-model/SHA/model/schema/types.yaml" == current_loc | ||
assert 'https://raw.githubusercontent.com/linkml/linkml-model/missing_branch/linkml_model/owl/mappings.owl.ttl' == \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: change in path here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the thing more worthy of fixing to me here is /SHA/model
and missing_branch/linkml_model
having different directory structure, but if the URLs resolve and they give us the thing we were after then they are the correct URLs in my book!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basically i feel like i have already spent too much time for the three of us overengineering this thing, and am fine with whatever as long as the paths work ;) some minor suggestions on test semantics but take em or leave em
def test_fixed_meta_url(): | ||
""" | ||
One fixed canary value - the METAMODEL_URI as used in linkml main shouldn't change | ||
""" | ||
assert URL_FOR(Source.META, Format.YAML) == 'https://w3id.org/linkml/meta.yaml' | ||
assert URL_FOR(Source.META, Format.JSONLD) == 'https://w3id.org/linkml/meta.context.jsonld' | ||
|
||
|
||
root_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "linkml_model")) | ||
|
||
SKIP_GITHUB_API = False # True means don't do the github API tests | ||
|
||
|
||
class LinkMLFilesTestCase(unittest.TestCase): | ||
""" Test that linkml_model/linkml_files.py work """ | ||
def test_basic_rules(self): | ||
self.assertEqual("https://w3id.org/linkml/annotations.yaml", | ||
URL_FOR(Source.ANNOTATIONS, Format.YAML)) | ||
self.assertEqual("https://w3id.org/linkml/meta.model.context.jsonld", | ||
URL_FOR(Source.META, Format.NATIVE_JSONLD)) | ||
self.assertEqual(os.path.join(root_path, "model/schema/meta.yaml"), | ||
LOCAL_PATH_FOR(Source.META, Format.YAML)) | ||
print(LOCAL_PATH_FOR(Source.META, Format.YAML)) | ||
self.assertEqual(os.path.join(root_path, "jsonld/types.model.context.jsonld"), | ||
LOCAL_PATH_FOR(Source.TYPES, Format.NATIVE_JSONLD)) | ||
self.assertEqual("https://linkml.github.io/linkml-model/model/schema/meta.yaml", | ||
GITHUB_IO_PATH_FOR(Source.META, Format.YAML)) | ||
self.assertEqual("https://linkml.github.io/linkml-model/jsonld/types.model.context.jsonld", | ||
GITHUB_IO_PATH_FOR(Source.TYPES, Format.NATIVE_JSONLD)) | ||
self.assertEqual("https://raw.githubusercontent.com/linkml/linkml-model/main/jsonld/meta.model.context.jsonld", | ||
GITHUB_PATH_FOR(Source.META, Format.NATIVE_JSONLD, ReleaseTag.LATEST)) | ||
self.assertEqual("https://raw.githubusercontent.com/linkml/linkml-model/testing_branch/owl/mappings.owl.ttl", | ||
GITHUB_PATH_FOR(Source.MAPPINGS, Format.OWL, branch="testing_branch")) | ||
|
||
@unittest.skipIf(SKIP_GITHUB_API, "Github API tests skipped") | ||
def test_github_specific_rules(self): | ||
""" | ||
Test accesses that require github API to access | ||
|
||
def test_basic_rules(): | ||
assert "https://w3id.org/linkml/annotations.yaml" == URL_FOR(Source.ANNOTATIONS, Format.YAML) | ||
assert "https://w3id.org/linkml/meta.context.jsonld" == URL_FOR(Source.META, Format.NATIVE_JSONLD) | ||
assert os.path.join(root_path, "model/schema/meta.yaml") == LOCAL_PATH_FOR(Source.META, Format.YAML) | ||
print(LOCAL_PATH_FOR(Source.META, Format.YAML)) | ||
assert os.path.join(root_path, "jsonld/types.context.jsonld") == LOCAL_PATH_FOR(Source.TYPES, Format.NATIVE_JSONLD) | ||
assert "https://linkml.github.io/linkml-model/latest/linkml_model/model/schema/meta.yaml" == GITHUB_IO_PATH_FOR( | ||
Source.META, Format.YAML) | ||
assert "https://linkml.github.io/linkml-model/latest/linkml_model/jsonld/types.context.jsonld" == \ | ||
GITHUB_IO_PATH_FOR(Source.TYPES, Format.NATIVE_JSONLD) | ||
assert "https://raw.githubusercontent.com/linkml/linkml-model/main/linkml_model/jsonld/meta.context.jsonld" == \ | ||
GITHUB_PATH_FOR(Source.META, Format.NATIVE_JSONLD, ReleaseTag.LATEST) | ||
assert "https://raw.githubusercontent.com/linkml/linkml-model/testing_branch/linkml_model/owl/mappings.owl.ttl" == \ | ||
GITHUB_PATH_FOR(Source.MAPPINGS, Format.OWL, branch="testing_branch") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
test_basic_rules
is basically the same thing as test_fixed_meta_url
- in general i only test fixed values when there is a specific reason for it to be fixed forever, so that's why i only tested those two URLs, but i think if we're going to test the rest of these then they should be in a test structure that reflects the meaning of the test :)
test each of these functions (URL_FOR
, LOCAL_PATH_FOR
, GITHUB_IO_PATH_FOR
and GITHUB_PATH_FOR
) separately, and some of what we want to test for might already be covered by other tests.
URL_FOR
,LOCAL_PATH_FOR
, andGITHUB_IO_PATH_FOR
are trivial string formatting functions, so they can be covered by unit-style tests.LOCAL_PATH_FOR
has atest_local_paths
test for existence for all the files in the expected formats already.URL_FOR
hastest_url_for_format
that tests the resolution of the generated URLs for the formats that have rewrite rules in the w3id repo.GITHUB_PATH_FOR
is substantially different than the others and should be tested separately for sure,
ideally these would have docstrings about why they should be fixed forever (if they were just fixed forever, why wouldn't we just hardcode the strings in the module itself and call it a day?), but idrc about that ;)
GITHUB_PATH_FOR(Source.MAPPINGS, Format.OWL, branch="testing_branch") | ||
|
||
|
||
SKIP_GITHUB_API = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ideally this would be a pytest option or dependent on some flag, if we want to keep skipping them - i marked them as skip because they kept failing bc the ratelimits on github API were very low, but if we want to rescue them then yes a flag that can be accessed would be good :)
GITHUB_PATH_FOR(Source.META, Format.NATIVE_JSONLD, "v0.0.1") | ||
current_loc = re.sub(r'linkml-model/[0-9a-f]*/', 'linkml-model/SHA/', GITHUB_PATH_FOR(Source.TYPES, Format.YAML)) | ||
assert "https://raw.githubusercontent.com/linkml/linkml-model/SHA/model/schema/types.yaml" == current_loc | ||
assert 'https://raw.githubusercontent.com/linkml/linkml-model/missing_branch/linkml_model/owl/mappings.owl.ttl' == \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the thing more worthy of fixing to me here is /SHA/model
and missing_branch/linkml_model
having different directory structure, but if the URLs resolve and they give us the thing we were after then they are the correct URLs in my book!
@@ -33,11 +33,13 @@ packages = [ | |||
[tool.poetry.dependencies] | |||
python = "^3.9" | |||
linkml-runtime = "^1.6.3" | |||
rdflib = "^7.0.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oof if we are gaining rdflib as a runtime dependency just to use the Namespace
class to declare a module constant i woudl call that highly undesirable and worthy of thinking about whether we need that
please see: linkml/linkml-runtime#320
back-porting changes in the runtime PR to linkml-model (current SoT for linkml-model/linkml_files.py methods).
this PR was a result of my dive into converting schemaview (linkml-runtime) tests to pytest when I ran into an error in linkml_files.py and test_linkml_files.py in linkml-runtime/linkml-model. It might be that we want to reconsider vendoring.
my biggest concern here with this PR, is that the tests for linkml-model and linkml-runtime that exercise linkml_files.py were different. If I put back in (as I did with this PR), the tests currently in linkml-model, there are slight changes in resulting expected URLs for model files - the new URLs do resolve, so I've adjusted the tests. We likely need @cmungall's review here before merge.