Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Fix code highlighting in third-party libraries integration guide #268

Merged
merged 6 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
default_language_version:
python: python3.11
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
Expand All @@ -13,27 +15,27 @@ repos:
rev: v1.8.0
hooks:
# See https://github.com/pre-commit/mirrors-mypy/blob/main/.pre-commit-hooks.yaml
- id: mypy
types_or: [ python, pyi ]
args: [--ignore-missing-imports, --scripts-are-modules]
- id: mypy
types_or: [python, pyi]
args: [--ignore-missing-imports, --scripts-are-modules]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.2.0
rev: v0.2.2
hooks:
- id: ruff
args: [ --fix, --exit-non-zero-on-fix ]
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
- repo: https://github.com/PyCQA/bandit
rev: 1.7.7
hooks:
- id: bandit
args: [-c, pyproject.toml]
additional_dependencies: ["bandit[toml]"]
- id: bandit
args: [-c, pyproject.toml]
additional_dependencies: ["bandit[toml]"]
Copy link
Collaborator

@maxmynter maxmynter Feb 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with these indents here (and above). But Nicho noted them on another PR from me. Do we have a projet standard on these idents/formatting things?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest of the file uses this style of indentation. From a quick look we also use it in the other files.

AFAIK, we do not have a standard defined.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't, but at least each file should be consistent. FWIW, I consider whatever defaults the YAML plugin in VS Code has my standard (which is why the file got reformatted when I saved it).

- repo: https://github.com/jsh9/pydoclint
rev: 0.3.9
rev: 0.4.1
hooks:
- id: pydoclint
- repo: https://github.com/jazzband/pip-tools
rev: 7.3.0
rev: 7.4.0
hooks:
- id: pip-compile
name: pip-compile requirements-dev.txt
Expand Down
2 changes: 1 addition & 1 deletion docs/_code/polars_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@
us_lakes = lakes.filter(pl.col("Country") == "United States of America")

with fs.open(f"lakefs://quickstart/{tx.branch.id}/us_lakes.csv", "wb") as f:
us_lakes.write_csv(f)
us_lakes.write_csv(f) # (1)!

tx.commit(message="Add US lakes")
9 changes: 4 additions & 5 deletions docs/guides/integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ See the Pandas documentation on [reading/writing remote files](https://pandas.py

The following code snippet illustrates how to read and write Pandas data frames in various formats from/to a lakeFS repository in the context of a [transaction](transactions.md):

```python hl_lines="10 12"
```python hl_lines="8 10"
--8<-- "docs/_code/pandas_example.py"
```

Expand All @@ -32,7 +32,7 @@ This allows DuckDB to transparently query and store data located in lakeFS repos

Similar to the example above, the following code snippet illustrates how to read and write data from/to a lakeFS repository in the context of a [transaction](transactions.md) through the [DuckDB Python API](https://duckdb.org/docs/api/python/overview.html){: target="_blank" rel="noopener"}:

```python hl_lines="6 11 13"
```python hl_lines="6 9 11"
--8<-- "docs/_code/duckdb_example.py"
```

Expand All @@ -48,8 +48,7 @@ The Python API wrapper for the Rust-based [Polars](https://pola-rs.github.io/pol

Again, the following code example demonstrates how to read a Parquet file and save a modified version back in CSV format to a lakeFS repository from Polars in the context of a [transaction](transactions.md):


```python hl_lines="10 13-14"
```python hl_lines="8 11-12"
--8<-- "docs/_code/polars_example.py"
```

Expand All @@ -63,6 +62,6 @@ PyArrow `read_*` and `write_*` functions take an explicit `filesystem` parameter

The following example code illustrates the use of lakeFS-spec with PyArrow, reading a Parquet file and writing it back to a lakeFS repository as a partitioned CSV dataset in the context of a [transaction](transactions.md):

```python hl_lines="12 17"
```python hl_lines="10 15"
--8<-- "docs/_code/pyarrow_example.py"
```
Loading