Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Lambda initial POC #139

Merged
merged 59 commits into from
Dec 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
28117fa
initial aws lambda poc
Nov 6, 2023
7c1d813
initial aws lambda poc
Nov 6, 2023
835a39e
initial aws lambda poc2
Nov 7, 2023
fc05ae1
initial aws lambda poc adding doc and code cleanup
Nov 8, 2023
4855ebc
lambda poc support bad usages cases + tests
Nov 8, 2023
be9a53f
pull main and code cleanup
Nov 9, 2023
bbc20d1
added region when creating boto3 la,bda client
Nov 9, 2023
acbb638
added region when creating boto3 lambda client
Nov 9, 2023
d24dc52
added region when creating boto3 lambda client
Nov 9, 2023
42b3965
removed unnecessary lambda global
Nov 9, 2023
7db9242
aws lambda tests cleanup
Nov 9, 2023
9bf3526
aws lambda raise exetion not enough args provided
Nov 12, 2023
6b5f95d
aws lambda del lambda_name, redundant
Nov 12, 2023
35c80cd
aws lambda add parameters auth in factory
Nov 12, 2023
ba77944
more lambda tests, get lambda role if exists
Nov 13, 2023
7722a6b
resolve a conflict
Nov 13, 2023
bb7dd87
Merge branch 'main' into rh/aws_lambda_poc
BelSasha Nov 13, 2023
54c1557
change region selection for lambda client
Nov 13, 2023
62ff7f0
merge remote changes
Nov 13, 2023
060a674
add option to create lambdas with function object (#141)
jlewitt1 Nov 13, 2023
3e828a4
added some TODOs
Nov 13, 2023
fb8f5f5
[WIP] send rh function instance to AWS Lambda (#143)
BelSasha Nov 29, 2023
f3eeea6
enable runhouse env, code cleanup
Dec 4, 2023
9a5e254
bugfix sending rh func to aws lambda
Dec 4, 2023
9dcad05
rh install lambda
Dec 4, 2023
4524031
create funciton folder
Dec 4, 2023
dff4e05
create funciton folder
Dec 4, 2023
ca9b68c
create funciton folder
Dec 4, 2023
216a930
change home dir to tmp and code clean up
Dec 5, 2023
7e9e838
loading lambda from den, enable delete
Dec 7, 2023
bdfca27
loading lambda from den, enable delete
Dec 7, 2023
b9d9df7
remove child config override
Dec 7, 2023
5cd3f51
factory method documantaiton changes
Dec 10, 2023
ebcd451
factory method documantaiton changes
Dec 10, 2023
1e4857c
Merge branch 'main' into rh/aws_lambda_poc
BelSasha Dec 10, 2023
5b452d4
merge main to rh/aws_lambda_poc
Dec 10, 2023
b0a4dc9
install requirements.txt during runtime
Dec 10, 2023
a448557
change constructor name
Dec 10, 2023
b33c3de
docs update
Dec 10, 2023
9970301
docs update
Dec 10, 2023
ee309d1
minor implementation changes
Dec 11, 2023
64c9771
minor test suite change
Dec 11, 2023
6622fbd
Merge branch 'main' into rh/aws_lambda_poc
Dec 12, 2023
c69dda3
update documents and add more examples
Dec 12, 2023
60e5675
update documents and add more examples
Dec 12, 2023
270e783
docs changes and some minor changes
Dec 12, 2023
33f91c9
Merge branch 'main' into rh/aws_lambda_poc
BelSasha Dec 12, 2023
ef17ad8
constructor name change
Dec 12, 2023
8f3273d
Merge branch 'rh/aws_lambda_poc' of github.com:run-house/runhouse int…
Dec 12, 2023
f0b6f32
constructor name change
Dec 12, 2023
a9f90a7
constructor name change
Dec 12, 2023
085aae0
lambda docs final touches
Dec 12, 2023
ded0c8b
merge main into rh/aws_lambda_poc
Dec 13, 2023
cda6ef4
docs minor changes
Dec 13, 2023
1733237
docs tests change
Dec 13, 2023
43f134a
docs tests change
Dec 13, 2023
5f0be39
Merge remote-tracking branch 'origin/rh/aws_lambda_poc' into rh/aws_l…
dongreenberg Dec 14, 2023
e371323
Flip back conftest
dongreenberg Dec 14, 2023
2f4f68b
Merge branch 'main' into rh/aws_lambda_poc
dongreenberg Dec 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/api/python.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ and rich debugging and accessibility interfaces built-in.

python/function

.. toctree::
:maxdepth: 1

python/lambda_function

.. toctree::
:maxdepth: 1

Expand Down
66 changes: 66 additions & 0 deletions docs/api/python/lambda_function.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
Lambda Function
BelSasha marked this conversation as resolved.
Show resolved Hide resolved
====================================

.. note::

Lambda Function support is an alpha and under active development.
Please report any bugs and let us know of any feature requests.

| A `Lambda Function <https://aws.amazon.com/lambda/>`_ is a serverless compute service provided by AWS. It allows you
to run applications and backend services without provisioning or managing servers. Runhouse will allow you to
maintain, invoke and share your Lambda functions and their code. It is comprised of the entry point, configuration, and
dependencies necessary to run the service.


| There are few core options to create an AWS Lambda using Runhouse:
#. Pass a callable Python function to the factory method.
#. Follow a typical Lambda creation flow (as if you are using AWS APIs). That includes passing to the constructor
path(s) to Python file(s) (containing the code) and a handler function name . Arguments such as runtime, Lambda name,
timeout and memory size are accepted by the factory method as well, but they are not mandatory and have default values.
#. Create rh.function instance, and than send it over to AWS Lambdas. For example:
:code:`rh.function(summer).to(system=aws_lambda)`



Lambda Function Factory Method
~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: runhouse.aws_lambda_fn

Lambda Function Class
~~~~~~~~~~~~~~

.. autoclass:: runhouse.LambdaFunction
:members:
:exclude-members: to

.. automethod:: __init__

Lambda Hardware Setup
~~~~~~~~~~~~~~

To create an AWS Lambda, you must grant the necessary permissions to do so. They are provided by an IAM
role, which should be attached to a certain AWS profile. This setup is made in the :code:`~/.aws/config` file.

For example, your local :code:`~/.aws/config` contains:

.. code-block:: ini

[profile lambda]
role_arn = arn:aws:iam::123456789:role/lambda-creation-role
region = us-east-1
source_profile = default

There are several ways to provide the necessary credentials which enables Lambda creation:

#. Specify the profile name in your code editor:

#. Install the `AWS Toolkit` extension.
#. Choose the relevant profile (e.g. `profile lambda`) and region (e.g `us-east-1`).

#. Environment Variable: setting :code:`AWS_PROFILE` to :code:`"lambda"`

.. note::
If no specific profile is provided, Runhouse will try using the default profile. Note if this default AWS identity
will not have the relevant IAM permissions for creating a Lambda, you will not be able to create a
Lambda.
3 changes: 2 additions & 1 deletion runhouse/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
from runhouse.resources.blobs import blob, Blob, file, File
from runhouse.resources.envs import conda_env, CondaEnv, env, Env
from runhouse.resources.folders import Folder, folder, GCSFolder, S3Folder
from runhouse.resources.function import function, Function
from runhouse.resources.functions.aws_lambda import aws_lambda_fn, LambdaFunction
from runhouse.resources.functions.function import function, Function
from runhouse.resources.hardware import (
_current_cluster,
cluster,
Expand Down
2 changes: 2 additions & 0 deletions runhouse/resources/functions/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
from .aws_lambda import aws_lambda_fn, LambdaFunction
from .function import function, Function
Loading
Loading