Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[linter] Transform ConanFile class to match actual one #11162

Merged
merged 14 commits into from
Jun 17, 2022
18 changes: 15 additions & 3 deletions .github/workflows/linter-conan-v2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
env:
PYTHONPATH: ${{github.workspace}}
PYVER: "3.8"
SCORE_THRESHOLD: "9.0"
SCORE_THRESHOLD: "9.5"
REQUIREMENTS: "pylint==2.14"

jobs:
Expand All @@ -23,14 +23,20 @@ jobs:
with:
files: |
linter/**
- name: Get Conan v1 version
id: parse_conan_v1_version
if: steps.changed_files.outputs.any_changed == 'true'
uses: mikefarah/yq@master
with:
cmd: yq '.conan.version' '.c3i/c3i.yml'
- uses: actions/setup-python@v3
if: steps.changed_files.outputs.any_changed == 'true'
with:
python-version: ${{ env.PYVER }}
- name: Install requirements
if: steps.changed_files.outputs.any_changed == 'true'
run: |
pip install ${{ env.REQUIREMENTS }}
pip install ${{ env.REQUIREMENTS }} conan==${{ steps.parse_conan_v1_version.outputs.result }}
- name: Execute linter over all recipes in the repository
if: steps.changed_files.outputs.any_changed == 'true'
run: |
Expand Down Expand Up @@ -63,14 +69,20 @@ jobs:
with:
files: |
recipes/*/*/conanfile.py
- name: Get Conan v1 version
id: parse_conan_v1_version
if: steps.changed-files.outputs.any_changed == 'true'
uses: mikefarah/yq@master
with:
cmd: yq '.conan.version' '.c3i/c3i.yml'
- uses: actions/setup-python@v3
if: steps.changed-files.outputs.any_changed == 'true'
with:
python-version: ${{ env.PYVER }}
- name: Install dependencies
if: steps.changed-files.outputs.any_changed == 'true'
run: |
pip install ${{ env.REQUIREMENTS }}
pip install ${{ env.REQUIREMENTS }} conan==${{ steps.parse_conan_v1_version.outputs.result }}
- name: Run linter
if: steps.changed-files.outputs.any_changed == 'true'
run: |
Expand Down
13 changes: 11 additions & 2 deletions linter/.pylintrc
Original file line number Diff line number Diff line change
@@ -1,12 +1,21 @@
[MASTER]
load-plugins=linter.conanv2_transition
load-plugins=linter.conanv2_transition,
linter.conanfile_transform
py-version=3.6
recursive=no
suggestion-mode=yes
unsafe-load-any-extension=no

[MESSAGES CONTROL]
disable=all
disable=fixme,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My plan is to disable everything needed for all the recipes to work... then, following PRs can try to enable one by one.

line-too-long,
missing-module-docstring,
missing-function-docstring,
missing-class-docstring,
invalid-name,
wrong-import-order, # TODO: Remove
import-outside-toplevel # TODO: Remove

enable=conan-bad-name,
conan-missing-name

Expand Down
74 changes: 74 additions & 0 deletions linter/conanfile_transform.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@

# Class ConanFile doesn't declare all the valid members and functions,
# some are injected by Conan dynamically to the class.

import textwrap
import astroid
from astroid.builder import AstroidBuilder
from astroid.manager import AstroidManager


def _settings_transform():
module = AstroidBuilder(AstroidManager()).string_build(
textwrap.dedent("""
class Settings(object):
os = None
arch = None
compiler = None
build_type = None
""")
)
return module['Settings']

def _user_info_build_transform():
module = AstroidBuilder(AstroidManager()).string_build(
textwrap.dedent("""
class UserInfoBuild(defaultdict):
pass
""")
)
return module['UserInfoBuild']


def register(_):
pass

def transform_conanfile(node):
"""Transform definition of ConanFile class so dynamic fields are visible to pylint"""

str_class = astroid.builtin_lookup("str")
dict_class = astroid.builtin_lookup("dict")
info_class = astroid.MANAGER.ast_from_module_name("conans.model.info").lookup(
"ConanInfo")
build_requires_class = astroid.MANAGER.ast_from_module_name(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
build_requires_class = astroid.MANAGER.ast_from_module_name(
user_info_build_class = astroid.MANAGER.ast_from_module_name("conans.model.user_info").lookup(
"DepsUserInfo")
build_requires_class = astroid.MANAGER.ast_from_module_name(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why, but using this approach, it fails for odbc recipe. The error is weird and I didn't manage to find a fix on the internet. With the string_build approach, on the other hand, it works. I'm not very happy about how it is looking like, tbh.

"conans.client.graph.graph_manager").lookup("_RecipeBuildRequires")
file_copier_class = astroid.MANAGER.ast_from_module_name(
"conans.client.file_copier").lookup("FileCopier")
file_importer_class = astroid.MANAGER.ast_from_module_name(
"conans.client.importer").lookup("_FileImporter")
python_requires_class = astroid.MANAGER.ast_from_module_name(
"conans.client.graph.python_requires").lookup("PyRequires")

dynamic_fields = {
"conan_data": str_class,
"build_requires": build_requires_class,
"tool_requires": build_requires_class,
"info_build": info_class,
"user_info_build": [_user_info_build_transform()],
"info": info_class,
"copy": file_copier_class,
"copy_deps": file_importer_class,
"python_requires": [str_class, python_requires_class],
"recipe_folder": str_class,
"settings_build": [_settings_transform()],
"settings_target": [_settings_transform()],
"conf": dict_class,
}

for f, t in dynamic_fields.items():
node.locals[f] = [i for i in t]


astroid.MANAGER.register_transform(
astroid.ClassDef, transform_conanfile,
lambda node: node.qname() == "conans.model.conan_file.ConanFile")
13 changes: 13 additions & 0 deletions linter/recipe_linter.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
{
"problemMatcher": [
{
"owner": "recipe_linter_fatals",
"severity": "error",
"pattern": [
{
"regexp": "(\\S+):(\\d+): \\[(F\\d+\\(\\S+\\)),\\s(.+?)\\](.+)",
"file": 1,
"line": 2,
"message": 5,
"code": 3
}
]
},
{
"owner": "recipe_linter_errors",
"severity": "error",
Expand Down