[TST] Only skip tests for non-installed packages on local machine

[hectormz]

PR Description

Please describe the changes proposed in the pull request:

• Ensure that optional tests (for optionally installed packages) can be skipped on local machines (but not CI)
• Adds custom importorskip (borrowed from Skip tests for optional/extra dependencies when not installed arviz-devs/arviz#1113) to prevent skips on CI machine
• Removes pytest.mark.skipif wrapper on pyspark fixtures, which seems to be not needed (?)

This PR resolves #115.

PR Checklist

Please ensure that you have done the following:

1. PR in from a fork off your branch. Do not PR from <your_username>:dev, but rather from <your_username>:<feature-branch_name>.

2. If you're not on the contributors list, add yourself to AUTHORS.rst.

3. Add a line to CHANGELOG.rst under the latest version header (i.e. the one that is "on deck") describing the contribution.
   • Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

Quick Check

To do a very quick check that everything is correct, follow these steps below:

• Run the command make check from pyjanitor's top-level directory. This will automatically run:
  • black formatting
  • flake8 checking
  • running the test suite
  • docs build

Once done, please check off the check-box above.

If make check does not work for you, you can execute the commands listed in the Makefile individually.

Code Changes

If you are adding code changes, please ensure the following:

• Ensure that you have added tests.
• Run all tests ($ pytest .) locally on your machine.
  • Check to ensure that test coverage covers the lines of code that you have added.
  • Ensure that all tests pass.

Relevant Reviewers

Please tag maintainers to review.

• @ericmjl
• @zjpoh
• @szuckerman

[hectormz]

🤔 not sure why flake8 only passes on CI Windows and my Windows...

Disagree with the if/else complaint from DeepSource

[hectormz]

In hindsight, it actually might be worth scrapping the custom importorskip and use pytest's original importorskip:

if running_on_ci():
    import pyspark
else:
    pyspark = importorskip("pyspark")

More code in your face, but everything is more obvious.

[codecov]

Codecov Report

Merging #698 into dev will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              dev     #698   +/-   ##
=======================================
  Coverage   93.16%   93.16%           
=======================================
  Files          16       16           
  Lines         600      600           
=======================================
  Hits          559      559           
  Misses         41       41           

[ericmjl]

OK @hectormz! I fixed the issue about conditional imports, and submitted two corrective feedback pieces to deepsource that they a conditional import should be allowed.

Anybody else wanna do the honors of hittin' the button? 😸

[hectormz]

@ericmjl got it! I meant to check out if pyspark is actually used when imported. It might just be verifying that it is installed at that point

[hectormz]

Nevermind! I'm fine with current state, unless someone else reviews and finds a problem

[ericmjl]

OK, I'm gonna hit merge, two days have passed.

[ericmjl]

🎉