Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: write processed file paths in tmp/file_list.json #50

Merged
merged 10 commits into from
Jul 13, 2022
Merged

feat: write processed file paths in tmp/file_list.json #50

Changes from 7 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
cde1d8c
feat: create polygon script return temporary file path
paulfouquet Jul 12, 2022
ee8f448
fix: main return value and formatting
paulfouquet Jul 12, 2022
ebd2bc3
feat: Allow Create Polygons script to run with multiple files
paulfouquet Jul 12, 2022
bebaf93
merge master
paulfouquet Jul 12, 2022
8c0aedd
fix: argument source was not correctly set
paulfouquet Jul 13, 2022
6213bdd
merge master
paulfouquet Jul 13, 2022
c3caa70
feat: write processed file paths in tmp/file_list.json
paulfouquet Jul 13, 2022
8130391
fix: remove unused imports
paulfouquet Jul 13, 2022
58a2d5c
Merge branch 'master' into feat/create-polygons-multiple-files
paulfouquet Jul 13, 2022
574372d
fix: pylint complaining about encoding
paulfouquet Jul 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions scripts/create_polygons.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import argparse
import json
import os
import subprocess
import tempfile
from collections import Counter
from typing import List
Expand Down Expand Up @@ -39,11 +41,11 @@ def get_pixel_count(file_path: str) -> int:
return data_pixels_count


def main() -> List[str]: # pylint: disable=too-many-locals
def main() -> None: # pylint: disable=too-many-locals
logger = get_log()

parser = argparse.ArgumentParser()
parser.add_argument("--source", dest="source", required=True)
parser.add_argument("--source", dest="source", nargs="+", required=True)
arguments = parser.parse_args()
source = arguments.source

Expand All @@ -53,6 +55,7 @@ def main() -> List[str]: # pylint: disable=too-many-locals
for file in source:
with tempfile.TemporaryDirectory() as tmp_dir:
source_file_name = os.path.basename(file)
uri_parse = file
# Download the file
if str(file).startswith("s3://"):
uri_parse = urlparse(file, allow_fragments=False)
Expand Down Expand Up @@ -86,7 +89,8 @@ def main() -> List[str]: # pylint: disable=too-many-locals

output_files.append(temp_file_path)

return output_files
with open("/tmp/file_list.json", "w") as jf:
json.dump(output_files, jf)


if __name__ == "__main__":
Expand Down