Skip to content

Commit

Permalink
Merge branch 'master' into grubberr/oncall-115-csv_field_size_limit
Browse files Browse the repository at this point in the history
  • Loading branch information
@grubberr
grubberr committed Feb 4, 2022
2 parents 8f837ec + b5b0976 commit 88e88fa
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions buildSrc/src/main/groovy/airbyte-python.gradle
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,12 @@ class Helpers {
if (project.file(testFilesDirectory).exists()) {

project.projectDir.toPath().resolve(testFilesDirectory).traverse(type: FileType.FILES, nameFilter: ~/(^test_.*|.*_test)\.py$/) { file ->
project.task("_${taskName}Coverage", type: PythonTask) {
project.task("_${taskName}Coverage", type: PythonTask, dependsOn: taskDependencies) {
module = "coverage"
command = "run --data-file=${testFilesDirectory}/.coverage.${taskName} --rcfile=${project.rootProject.file('tools/python/.coveragerc').absolutePath} -m pytest -s ${testFilesDirectory}"
}
// generation of coverage report is optional and we should skip it if tests are empty
project.task(taskName, type: Exec, dependsOn: taskDependencies) {
project.task(taskName, type: Exec){
commandLine = ".venv/bin/python"
args "-m", "coverage", "report", "--data-file=${testFilesDirectory}/.coverage.${taskName}", "--rcfile=${project.rootProject.file('tools/python/.coveragerc').absolutePath}"
dependsOn project.tasks.findByName("_${taskName}Coverage")
Expand Down

1 comment on commit 88e88fa

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SonarQube Report

SonarQube report for Airbyte Connectors Source Salesforce(#10012)

Measures

Name Value Name Value Name Value
Coverage 14.3 Security Rating A Reliability Rating A
Lines to Cover 14 Duplicated Blocks 0 Duplicated Lines (%) 0.0
Quality Gate Status ERROR Lines of Code 1232 Code Smells 2
Bugs 0 Vulnerabilities 0 Blocker Issues 0
Critical Issues 4 Major Issues 9 Minor Issues 65

Detected Issues

Rule File Description Message
python:S108 (MAJOR) unit_tests/unit_test.py:584 Nested blocks of code should not be left empty Either remove or fill this block of code.
python:S108 (MAJOR) unit_tests/unit_test.py:589 Nested blocks of code should not be left empty Either remove or fill this block of code.
python:isort_need_format (MINOR) unit_tests/unit_test.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:isort_need_format (MINOR) source_salesforce/source.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:mypy_import (MINOR) source_salesforce/source.py:14 Require that imported module can be found or has stubs Library stubs not installed for "requests" (or incompatible with Python 3.7) . Code line: from requests import codes, exceptions
python:S1192 (CRITICAL) unit_tests/unit_test.py:347 String literals should not be duplicated Define a constant instead of duplicating this literal "TotalRequests Limit exceeded." 3 times.
python:S5886 (MAJOR) source_salesforce/streams.py:256 Function return types should be consistent with their type hint Remove this yield statement or annotate function "download_data" with "typing.Generator" or one of its supertypes.
python:S1192 (CRITICAL) unit_tests/unit_test.py:81 String literals should not be duplicated Define a constant instead of duplicating this literal "https://fase-account.salesforce.com" 3 times.
python:isort_need_format (MINOR) integration_tests/bulk_error_test.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:S1192 (CRITICAL) unit_tests/unit_test.py:138 String literals should not be duplicated Define a constant instead of duplicating this literal "2122-08-22T05:08:29.000Z" 3 times.
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:52 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def path(self, next_page_token: Mapping[str, Any] = None, **kwargs...
python:mypy_return_value (MINOR) source_salesforce/streams.py:57 Check that return value is compatible with signature Incompatible return value type (got "Mapping[str, Any]", expected "str") . Code line: return next_page_token
python:mypy_no_any_return (MINOR) source_salesforce/streams.py:62 Reject returning value with "Any" type if return type is not "Any" Returning Any from function declared to return "str" . Code line: return response_data.get("nextRecordsUrl")
python:mypy_valid_type (MINOR) source_salesforce/streams.py:271 Check that type (annotation) is valid Function "builtins.any" is not valid as a type . Code line: ...e: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_pag...
python:mypy_operator (MINOR) source_salesforce/streams.py:280 Check that operator is valid for operands Unsupported operand types for + ("str" and "Mapping[str, Any]") . Code line: query += next_page_token
python:mypy_valid_type (MINOR) source_salesforce/streams.py:334 Check that type (annotation) is valid Function "builtins.any" is not valid as a type . Code line: ...e: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_pag...
python:mypy_return (MINOR) source_salesforce/streams.py:370 Check that function always returns a value Missing return statement . Code line: def next_page_token(self, last_record: dict) -> str:
python:mypy_no_any_return (MINOR) source_salesforce/streams.py:372 Reject returning value with "Any" type if return type is not "Any" Returning Any from function declared to return "str" . Code line: return last_record[self.cursor_field]
python:mypy_no_untyped_def (MINOR) source_salesforce/api.py:209 Check that every function has an annotation Function is missing a return type annotation . Code line: def get_validated_streams(self, config: Mapping[str, Any], catalog...
python:mypy_no_untyped_def (MINOR) source_salesforce/utils.py:6 Check that every function has an annotation Function is missing a return type annotation . Code line: def filter_streams(streams_list: list, search_word: str, search_criter...
python:mypy_import (MINOR) source_salesforce/api.py:10 Require that imported module can be found or has stubs Library stubs not installed for "requests.exceptions" (or incompatible with Python 3.7) . Code line: from requests.exceptions import HTTPError
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:241 Check that every function has an annotation Function is missing a return type annotation . Code line: def filter_null_bytes(self, s: str):
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:322 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def init(self, replication_key: str, start_date: Optional[str]...
python:mypy_return (MINOR) source_salesforce/streams.py:328 Check that function always returns a value Missing return statement . Code line: def format_start_date(start_date: Optional[str]) -> Optional[str]:
python:mypy_no_any_return (MINOR) source_salesforce/streams.py:331 Reject returning value with "Any" type if return type is not "Any" Returning Any from function declared to return "Optional[str]" . Code line: return pendulum.parse(start_date).strftime("%Y-%m-%dT%H:%M...
python:mypy_attr_defined (MINOR) source_salesforce/streams.py:331 Check that attribute exists Module has no attribute "parse" . Code line: return pendulum.parse(start_date).strftime("%Y-%m-%dT%H:%M...
python:S5886 (MAJOR) source_salesforce/streams.py:227 Function return types should be consistent with their type hint Return a value of type "str" instead of "NoneType" or update function "execute_job" type hint.
python:mypy_attr_defined (MINOR) source_salesforce/streams.py:18 Check that attribute exists Module "pendulum" does not explicitly export attribute "DateTime"; implicit reexport disabled . Code line: from pendulum import DateTime
python:mypy_assignment (MINOR) source_salesforce/streams.py:89 Check that assigned value is compatible with target Incompatible types in assignment (expression has type "Mapping[str, Any]", variable has type "Optional[Dict[Any, Any]]") . Code line: self.schema = self.sf_api.generate_schema([self.name])
python:mypy_arg_type (MINOR) source_salesforce/streams.py:89 Check argument types in calls Argument 1 to "generate_schema" of "Salesforce" has incompatible type "List[str]"; expected "Optional[str]" . Code line: self.schema = self.sf_api.generate_schema([self.name])
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:119 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def init(self, wait_timeout: Optional[int], **kwargs):
python:mypy_assignment (MINOR) source_salesforce/streams.py:210 Check that assigned value is compatible with target Incompatible types in assignment (expression has type "float", variable has type "int") . Code line: delay_timeout = 0.5 + math.exp(delay_cnt) / 1000.0
python:S5799 (MAJOR) source_salesforce/streams.py:216 Implicit string and byte concatenations should not be confusing Merge these implicitly concatenated strings; or did you forget a comma?
python:S112 (MAJOR) source_salesforce/streams.py:238 "Exception" and "BaseException" should not be raised Replace this generic exception class with a more specific one.
flake8:E203 (MAJOR) unit_tests/unit_test.py:157 whitespace before ‘:’ whitespace before ':'
python:S125 (MAJOR) unit_tests/unit_test.py:226 Sections of code should not be commented out Remove this commented out code.
python:S5720 (CRITICAL) source_salesforce/streams.py:129 "self" should be the first argument to instance methods Rename "instance" to "self" or add the missing "self" parameter.
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:129 Check that every function has an annotation Function is missing a type annotation . Code line: def transform_empty_string_to_none(instance, schema):
python:mypy_has_type (MINOR) source_salesforce/streams.py:306 Check that type of reference can be determined Cannot determine type of "record" . Code line: yield record
python:isort_need_format (MINOR) main.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:mypy_no_untyped_def (MINOR) integration_tests/acceptance.py:12 Check that every function has an annotation Function is missing a return type annotation . Code line: def connector_setup():
python:mypy_import (MINOR) source_salesforce/api.py:7 Require that imported module can be found or has stubs Library stubs not installed for "requests" (or incompatible with Python 3.7) . Code line: import requests
python:mypy_no_untyped_def (MINOR) source_salesforce/api.py:177 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def init(
python:mypy_unreachable (MINOR) source_salesforce/api.py:194 Warn about unreachable statements or expressions Right operand of "and" is never evaluated . Code line: ...dbox is True or (isinstance(is_sandbox, str) and is_sandbox.lower() ==...
python:mypy_no_untyped_def (MINOR) source_salesforce/api.py:197 Check that every function has an annotation Function is missing a return type annotation . Code line: def _get_standard_headers(self):
python:mypy_no_untyped_def (MINOR) source_salesforce/api.py:249 Check that every function has an annotation Function is missing a return type annotation . Code line: def login(self):
python:mypy_no_any_return (MINOR) source_salesforce/api.py:274 Reject returning value with "Any" type if return type is not "Any" Returning Any from function declared to return "Mapping[str, Any]" . Code line: return resp.json()
python:mypy_index (MINOR) source_salesforce/api.py:280 Check indexing operations Unsupported target for indexed assignment ("object") . Code line: schema["properties"][field["name"]] = self.field_to_proper...
python:mypy_dict_item (MINOR) source_salesforce/api.py:308 Check dict items in a dict expression {key: value, ...} Dict entry 1 has incompatible type "str": "str"; expected "str": "List[str]" . Code line: ... property_schema = {"type": ["string", "null"], "format": "date-time"...
python:mypy_dict_item (MINOR) source_salesforce/api.py:314 Check dict items in a dict expression {key: value, ...} Dict entry 1 has incompatible type "str": "Dict[str, Dict[str, List[str]]]"; expected "str": "List[str]" . Code line: "properties": {
python:mypy_dict_item (MINOR) source_salesforce/api.py:326 Check dict items in a dict expression {key: value, ...} Dict entry 1 has incompatible type "str": "str"; expected "str": "List[str]" . Code line: ... property_schema = {"type": ["string", "null"], "format": "base64"}
python:mypy_dict_item (MINOR) source_salesforce/api.py:340 Check dict items in a dict expression {key: value, ...} Dict entry 1 has incompatible type "str": "Dict[str, Dict[str, List[str]]]"; expected "str": "List[str]" . Code line: "properties": {"longitude": {"type": ["null", "number"...
python:mypy_import (MINOR) source_salesforce/rate_limiting.py:11 Require that imported module can be found or has stubs Library stubs not installed for "requests" (or incompatible with Python 3.7) . Code line: from requests import codes, exceptions
python:mypy_no_untyped_def (MINOR) source_salesforce/rate_limiting.py:24 Check that every function has an annotation Function is missing a return type annotation . Code line: def default_backoff_handler(max_tries: int, factor: int, **kwargs):
python:mypy_no_untyped_def (MINOR) source_salesforce/rate_limiting.py:24 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def default_backoff_handler(max_tries: int, factor: int, **kwargs):
python:mypy_no_untyped_def (MINOR) source_salesforce/rate_limiting.py:25 Check that every function has an annotation Function is missing a type annotation . Code line: def log_retry_attempt(details):
python:mypy_no_untyped_def (MINOR) source_salesforce/rate_limiting.py:30 Check that every function has an annotation Function is missing a type annotation . Code line: def should_give_up(exc):
python:mypy_return (MINOR) source_salesforce/source.py:27 Check that function always returns a value Missing return statement . Code line: def check_connection(self, logger: AirbyteLogger, config: Mapping[...
python:mypy_valid_type (MINOR) source_salesforce/source.py:27 Check that type (annotation) is valid Function "builtins.any" is not valid as a type . Code line: ...logger: AirbyteLogger, config: Mapping[str, Any]) -> Tuple[bool, any]:
python:mypy_assignment (MINOR) source_salesforce/source.py:65 Check that assigned value is compatible with target Incompatible types in assignment (expression has type "Type[BulkSalesforceStream]", variable has type "Type[SalesforceStream]") . Code line: full_refresh, incremental = BulkSalesforceStream, Bulk...
python:mypy_assignment (MINOR) source_salesforce/source.py:65 Check that assigned value is compatible with target Incompatible types in assignment (expression has type "Type[BulkSalesforceStream]", variable has type "Type[IncrementalSalesforceStream]") . Code line: ...ull_refresh, incremental = BulkSalesforceStream, BulkIncrementalSalesf...
flake8:E501 (MAJOR) source_salesforce/source.py:101 line too long (82 > 79 characters) line too long (149 > 140 characters)
python:mypy_import (MINOR) source_salesforce/streams.py:14 Require that imported module can be found or has stubs Library stubs not installed for "requests" (or incompatible with Python 3.7) . Code line: import requests
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:33 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def init(self, sf_api: Salesforce, pk: str, stream_name: str, ...
python:mypy_valid_type (MINOR) source_salesforce/streams.py:65 Check that type (annotation) is valid Function "builtins.any" is not valid as a type . Code line: ...e: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_pag...
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:84 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def parse_response(self, response: requests.Response, **kwargs) ->...
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:92 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def read_records(self, **kwargs) -> Iterable[Mapping[str, Any]]:
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:123 Check that every function has an annotation Function is missing a type annotation for one or more arguments . Code line: def path(self, **kwargs) -> str:
python:mypy_override (MINOR) source_salesforce/streams.py:123 Check that method override is compatible with base class Signature of "path" incompatible with supertype "SalesforceStream" . Code line: def path(self, **kwargs) -> str:
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:139 Check that every function has an annotation Function is missing a return type annotation . Code line: def _send_http_request(self, method: str, url: str, json: dict = N...
python:mypy_no_any_return (MINOR) source_salesforce/streams.py:157 Reject returning value with "Any" type if return type is not "Any" Returning Any from function declared to return "Optional[str]" . Code line: return job_id
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:258 Check that every function has an annotation Function is missing a return type annotation . Code line: def abort_job(self, url: str):
python:mypy_no_untyped_def (MINOR) source_salesforce/streams.py:263 Check that every function has an annotation Function is missing a return type annotation . Code line: def delete_job(self, url: str):
python:mypy_return (MINOR) source_salesforce/streams.py:266 Check that function always returns a value Missing return statement . Code line: def next_page_token(self, last_record: dict) -> str:
python:mypy_override (MINOR) source_salesforce/streams.py:287 Check that method override is compatible with base class Signature of "read_records" incompatible with supertype "SalesforceStream" . Code line: def read_records(
python:mypy_misc (MINOR) source_salesforce/streams.py:305 Miscellaneous other checks "object" object is not iterable . Code line: for count, record in self.download_data(url=job_full_url):
python:mypy_has_type (MINOR) source_salesforce/streams.py:313 Check that type of reference can be determined Cannot determine type of "record" . Code line: next_page_token = self.next_page_token(record)
python:mypy_valid_type (MINOR) source_salesforce/streams.py:375 Check that type (annotation) is valid Function "builtins.any" is not valid as a type . Code line: ...e: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_pag...

Coverage (14.3%)

File Coverage File Coverage
integration_tests/acceptance.py 0.0 integration_tests/bulk_error_test.py 0.0
main.py 0.0 setup.py 0.0
source_salesforce/init.py 100.0 source_salesforce/api.py 73.8
source_salesforce/exceptions.py 100.0 source_salesforce/rate_limiting.py 86.4
source_salesforce/source.py 85.3 source_salesforce/streams.py 87.2
source_salesforce/utils.py 100.0 unit_tests/unit_test.py 0.0

Please sign in to comment.