Skip to content

Commit

Permalink
Update list_files method signature (#8)
Browse files Browse the repository at this point in the history
* Added fileWatch for GCP

* added test for filewatch

* Updated README

* Updated CHANGELOG

* update version number

* retracting version

* modify version pattern

* bump version v24.28.1 -> v24.31.0

* update CHANGELOG to match the new version

* fix file regex

* Update CHANGELOG.md

* bump version v24.31.0 -> v24.32.0

* Removed google cloud import and handling regex matching locally after fetching all files from gcp

* limiting query results to 100 per page

* modified list_files to accept file_pattern arg

* bump version v24.32.0 -> v24.32.1

* Updated Changelog
  • Loading branch information
simal00 authored Aug 7, 2024
1 parent 78b4bdd commit 7adb0b8
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 4 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## v24.32.1

- Minor fix to accept file_pattern arg in list_files method.

## v24.32.0

- List all files on GCP bucket and perform regex pattern matching to return files.
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "otf-addons-gcp"
version = "v24.32.0"
version = "v24.32.1"
authors = [{ name = "Aldo Troiano", email = "a.troiano@reply.com" }]
license = { text = "GPLv3" }
classifiers = [
Expand Down Expand Up @@ -51,7 +51,7 @@ dev = [
profile = 'black'

[tool.bumpver]
current_version = "v24.32.0"
current_version = "v24.32.1"
version_pattern = "vYY.WW.PATCH[-TAG]"
commit_message = "bump version {old_version} -> {new_version}"
commit = true
Expand Down
6 changes: 4 additions & 2 deletions src/opentaskpy/addons/gcp/remotehandlers/bucket.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,18 +264,20 @@ def create_flag_files(self) -> int:
"""Not implemented for this transfer type."""
raise NotImplementedError

def list_files(self, directory: str | None = None) -> list:
def list_files(
self, directory: str | None = None, file_pattern: str | None = None
) -> list:
"""List Files in GCP with pagination and local regex matching.
Args:
directory (str): A directory to list on the bucket.
file_pattern (str): File pattern to match files.
Returns:
list: A list of filenames if successful, an empty list if not.
"""
self.logger.info("Listing Files in Bucket.")
try:
file_pattern = self.spec["fileRegex"]
directory = directory or self.spec.get("directory", "")

base_url = (
Expand Down

0 comments on commit 7adb0b8

Please sign in to comment.