Skip to content

Conversation

Salakar
Copy link
Member

@Salakar Salakar commented May 4, 2023

Fixes #74
Fixes #75
Fixes #76
Fixes #81

Testing

Deployed the following sample:

from firebase_functions import storage_fn, params
from firebase_admin import initialize_app

initialize_app()

bucket = params.StringParam(
    "BUCKET",
    label="storage bucket",
    description="The bucket to resize images from.",
    input=params.ResourceInput(type=params.ResourceType.STORAGE_BUCKET),
    default=params.STORAGE_BUCKET,
)

output_path = params.StringParam(
    "OUTPUT_PATH",
    label="storage bucket output path",
    description=
    "The path of in the bucket where processed images will be stored.",
    input=params.TextInput(
        example="/images/processed",
        validation_regex=r"^\/.*$",
        validation_error_message=
        "Must be a valid path starting with a forward slash",
    ),
    default="/images/processed",
)

image_type = params.ListParam(
    "IMAGE_TYPE",
    label="convert image to preferred types",
    description="The image types you'd like your source image to convert to.",
    input=params.MultiSelectInput([
        params.SelectOption(value="jpeg", label="jpeg"),
        params.SelectOption(value="png", label="png"),
        params.SelectOption(value="webp", label="webp"),
    ]),
    default=["jpeg", "png"],
)

delete_original = params.BoolParam(
    "DELETE_ORIGINAL_FILE",
    label="delete the original file",
    description=
    "Do you want to automatically delete the original file from the Cloud Storage?",
    input=params.SelectInput([
        params.SelectOption(value=True, label="Delete on any resize attempt"),
        params.SelectOption(value=False, label="Don't delete"),
    ],),
    default=True,
)

image_resize_api_secret = params.SecretParam(
    "IMAGE_RESIZE_API_SECRET",
    label="image resize api secret",
    description="The fake secret key to use for the image resize API.",
)


@storage_fn.on_object_finalized(
    bucket=bucket,
    secrets=[image_resize_api_secret],
)
def resize_images(event: storage_fn.CloudEvent[storage_fn.StorageObjectData]):
    """
    This function will be triggered when a new object is created in the bucket.
    """
    print("Got a new image:", event)
    print("Selected image types:", image_type.value)
    print("Selected bucket resource:", bucket.value)
    print("Selected output location:", output_path.value)
    print("Testing a not so secret api key:", image_resize_api_secret.value)
    print("Should original images be deleted?:", delete_original.value)
    # TODO: Implement your image resize logic

Deployment successful:

? Would you like to proceed with deletion? Selecting no will continue the rest of the deployments. No
i  functions: creating Python 3.10 function resize_images(us-central1)...
✔  functions[resize_images(us-central1)] Successful create operation.
i  functions: cleaning up build files...

✔  Deploy complete!

Project Console: https://console.firebase.google.com/project/python-functions-testing/overview

image

Logs when a file is uploaded to storage;

2023-05-04 03:04:23.447 BST
Got a new image: CloudEvent(specversion='1.0', id='7594586441232904', source='//storage.googleapis.com/projects/_/buckets/python-functions-testing.appspot.com', type='google.cloud.storage.object.v1.finalized', time=datetime.datetime(2023, 5, 4, 2, 4, 22, 153261, tzinfo=datetime.timezone.utc), data=StorageObjectData(bucket='python-functions-testing.appspot.com', cache_control=None, component_count=None, content_disposition="inline; filename*=utf-8''IMG_5468.JPG", content_encoding=None, content_language=None, content_type='image/jpeg', crc32c='f9KoMQ==', customer_encryption=None, etag='CJeGubrJ2v4CEAE=', generation='1683165862052631', id='python-functions-testing.appspot.com/testing/IMG_5468.JPG/1683165862052631', kind='storage#object', md5_hash='MySB6kxWobchiW35zlXouA==', media_link='https://storage.googleapis.com/download/storage/v1/b/python-functions-testing.appspot.com/o/testing%2FIMG_5468.JPG?generation=1683165862052631&alt=media', metadata={'firebaseStorageDownloadTokens': '0a6b227c-cc28-4aff-be10-e47a735bebfe'}, metageneration='1', name='testing/IMG_5468.JPG', self_link='https://www.googleapis.com/storage/v1/b/python-functions-testing.appspot.com/o/testing%2FIMG_5468.JPG', size='298482', storage_class='STANDARD', time_created='2023-05-04T02:04:22.153Z', time_deleted=None, time_storage_class_updated='2023-05-04T02:04:22.153Z', updated='2023-05-04T02:04:22.153Z'), subject='objects/testing/IMG_5468.JPG')
2023-05-04 03:04:23.447 BST
Selected image types: ['jpeg', 'png']
2023-05-04 03:04:23.447 BST
Selected bucket resource: python-functions-testing.appspot.com
2023-05-04 03:04:23.447 BST
Selected output location: /a/good/path
2023-05-04 03:04:23.447 BST
Testing a not so secret api key: ffffff
2023-05-04 03:04:23.447 BST
Should original images be deleted?: True

A string param with ResourceInput to select a storage bucket, the default is set to the builtin default param params.STORAGE_BUCKET and you can see the input prompts with this value pre selected:
Pasted Graphic 7


A string param with text input to select a path:


Pasted Graphic 8

and tests regex validation:

Pasted Graphic 11


A list param with multi select + default selection correctly showing:

Pasted Graphic 13


A bool param with select input (also shows correct default value):

Pasted Graphic 14


A secret param:

image

(shows #76 is now fixed and secrets are being created correctly now)


The CLI generates a .env file:

BUCKET=python-functions-testing.appspot.com
OUTPUT_PATH=/a/good/path
IMAGE_TYPE=jpeg,png
DELETE_ORIGINAL_FILE=true

The generated manifest yaml:

endpoints:
  resize_images:
    availableMemoryMb: null
    concurrency: null
    entryPoint: resize_images
    eventTrigger:
      eventFilters:
        bucket: '{{ params.BUCKET }}'
      eventType: google.cloud.storage.object.v1.finalized
      retry: false
    ingressSettings: null
    labels: {}
    maxInstances: null
    minInstances: null
    platform: gcfv2
    secretEnvironmentVariables:
    - key: IMAGE_RESIZE_API_SECRET
    serviceAccountEmail: null
    timeoutSeconds: null
params:
- default: '{{ params.STORAGE_BUCKET }}'
  description: The bucket to resize images from.
  input:
    resource:
      type: storage.googleapis.com/Bucket
  label: storage bucket
  name: BUCKET
  type: string
- default: /images/processed
  description: The path of in the bucket where processed images will be stored.
  input:
    text:
      example: /images/processed
      validationErrorMessage: Must be a valid path starting with a forward slash
      validationRegex: ^\/.*$
  label: storage bucket output path
  name: OUTPUT_PATH
  type: string
- default:
  - jpeg
  - png
  description: The image types you'd like your source image to convert to.
  input:
    multiSelect:
      options:
      - label: jpeg
        value: jpeg
      - label: png
        value: png
      - label: webp
        value: webp
  label: convert image to preferred types
  name: IMAGE_TYPE
  type: list
- default: true
  description: Do you want to automatically delete the original file from the Cloud
    Storage?
  input:
    select:
      options:
      - label: Delete on any resize attempt
        value: true
      - label: Don't delete
        value: false
  label: delete the original file
  name: DELETE_ORIGINAL_FILE
  type: boolean
- description: The fake secret key to use for the image resize API.
  label: image resize api secret
  name: IMAGE_RESIZE_API_SECRET
  type: secret
requiredAPIs: []
specVersion: v1alpha1

@Salakar Salakar force-pushed the param-inputs branch 4 times, most recently from d8e6a5d to fbdaec7 Compare May 4, 2023 02:01
@Salakar Salakar marked this pull request as ready for review May 4, 2023 02:22
return []
# Otherwise, split the string by commas.
# (This is for emulator & the Firebase CLI generated .env file, the environment
# variable is a comma-separated list.)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@taeold FYI I had to handle both scenarios here; local .env is generated as comma delimited string:

image

cloud run env is json array string:

image

Copy link
Collaborator

@taeold taeold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Let's get rid of ListParam - we don't want people to use it :'(

if self.default is not None:
return self.default
return self.default.value if isinstance(
self.default, Expression) else self.default
return False


@_dataclasses.dataclass(frozen=True)
class ListParam(Param[list]):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - after all this work, I think the team is considering yanking ListParam support all together 🤦🏼 .

It isn't documented in https://firebase.google.com/docs/functions/config-env.

It is documented in JS SDK reference doc, and the team feels that it was a mistake to release it prematurely :'(

@taeold taeold merged commit 11f799f into firebase:main May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants