Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update RunInference documentation #22250

Merged
merged 52 commits into from
Jul 15, 2022
Merged
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
c7b0d99
starting RunInference documentation drafts
rszper Jun 29, 2022
eb89211
starting RunInference documentation drafts
rszper Jul 11, 2022
1a29d9e
RunInference documentation updates
rszper Jul 11, 2022
2be4739
RunInference documentation updates
rszper Jul 11, 2022
f5a0f0c
Skipping notebook because it doesn't exist
rszper Jul 11, 2022
1d404b6
Added troubleshooting section
rszper Jul 11, 2022
2d954de
Updated examples in snippets
rszper Jul 12, 2022
245dc49
Updated test
rszper Jul 12, 2022
99b1308
Updated troubleshooting section
rszper Jul 12, 2022
b0f430a
Updated troubleshooting section
rszper Jul 12, 2022
b02fd09
Moved batching elements content out of the RunInference transform page
rszper Jul 12, 2022
1234657
Added related links to the ML page
rszper Jul 12, 2022
0973e45
Updated docs based on comments
rszper Jul 12, 2022
5dd713e
Changed tensorflow to TensorFlow
rszper Jul 12, 2022
8a50ee8
Updated content based on Andy and Anand's comments
rszper Jul 13, 2022
e49ebec
Updated content based on Andy and Anand's comments
rszper Jul 13, 2022
5db0b67
Updated examples and made more changes based on comments.
rszper Jul 13, 2022
85d866b
Updated troubleshooting wording
rszper Jul 13, 2022
f766888
Updated model handler import intro
rszper Jul 13, 2022
40351c4
Trying to indicate that model_handler is a variable to be replaced
rszper Jul 13, 2022
e02160f
Fixed typos in model code
rszper Jul 13, 2022
2be34ea
Removed a line
rszper Jul 13, 2022
090c356
Fixing model_handler variables
rszper Jul 13, 2022
c2995c9
Removed unused examples from snippets
rszper Jul 13, 2022
dbe9b05
Update website/www/site/content/en/documentation/transforms/python/el…
rszper Jul 14, 2022
214ba7f
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
07e1f3e
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
7d8ce8e
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
c0c4548
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
580fa7f
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
651f52d
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
d3f80d5
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
380fcd3
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
033997b
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
335f1f1
Update website/www/site/content/en/documentation/sdks/python-machine-…
rszper Jul 14, 2022
489fce7
Updates based on comments
rszper Jul 14, 2022
4b962b4
Add import try and catch for unit tests that use torch
AnandInguva Jul 14, 2022
9e98188
fixup lint
AnandInguva Jul 14, 2022
46f5ebd
fixup: formatting
AnandInguva Jul 14, 2022
1f7ce97
fixup lint
AnandInguva Jul 14, 2022
07a99d7
Added error message to troubleshooting section
rszper Jul 14, 2022
49b0a7f
Add skip when GCP deps are not found
AnandInguva Jul 14, 2022
2d988c2
Added new pages to the TOC
rszper Jul 14, 2022
abde489
move LinearRegression out of the module
AnandInguva Jul 14, 2022
c340531
Add uses_pytorch marker to the torch tests
AnandInguva Jul 14, 2022
a3786c9
change importing order
AnandInguva Jul 15, 2022
cde3380
Modify torch tests
AnandInguva Jul 15, 2022
67b5d42
fixup torch test string
AnandInguva Jul 15, 2022
4160d1b
Merge pull request #18 from AnandInguva/inference-snippets
rszper Jul 15, 2022
4aa492c
Updates based on comments
rszper Jul 15, 2022
c1d5643
Fixed link location
rszper Jul 15, 2022
4e89126
Added a period
rszper Jul 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ For more information, see the [`BatchElements` transform documentation](https://

### Shared helper class

Instead of loading a model for each thread in the process, we use the `Shared` class, which allows us to load one model that is shared across all threads of each worker in a DoFn. For more information, see the
Using the `Shared` class within RunInference implementation allows us to load the model only once per process and share it with all DoFn instances created in that process. This feature reduces memory consumption and model loading time. For more information, see the
[`Shared` class documentation](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/utils/shared.py#L20).

### Multi-model pipelines
Expand All @@ -53,7 +53,7 @@ with pipeline as p:
```
Where `model_handler` is the model handler setup code.

To import models, you need to wrap them around a `ModelHandler` object. Which `ModelHandler` you import depends on the framework and type of data structure that contains the inputs. The following examples show some ModelHandlers that you might want to import.
To import models, you need to configure a `ModelHandler` object that wraps the underlying model. Which `ModelHandler` you import depends on the framework and type of data structure that contains the inputs. The following examples show some ModelHandlers that you might want to import.

```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These little chunks of code below here seem out of place.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yeandy How do you want to handle this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to reword it to something like this, and keep the (refactored) code block?

To import models, you need to wrap them around a ModelHandler object. The ModelHandler you import will depend on the framework and type of data structure that contains the inputs. See the following examples on which ones you may want to import.

from apache_beam.ml.inference.sklearn_inference import SklearnModelHandlerNumpy
from apache_beam.ml.inference.sklearn_inference import SklearnModelHandlerPandas
from apache_beam.ml.inference.pytorch_inference import PytorchModelHandlerTensor
from apache_beam.ml.inference.pytorch_inference import PytorchModelHandlerKeyedTensor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made some updates. Take a look and let me know if we need more changes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. By the way, the imports I originally wrote had some typos, so I fixed them

from apache_beam.ml.inference.sklearn_inference import SklearnModelHandlerNumpy
from apache_beam.ml.inference.sklearn_inference import SklearnModelHandlerPandas
from apache_beam.ml.inference.pytorch_inference import PytorchModelHandlerTensor
from apache_beam.ml.inference.pytorch_inference import PytorchModelHandlerKeyedTensor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to fix the typos

from apache_beam.ml.inference.sklearn_inference import SklearnModelHandlerNumpy
Expand Down Expand Up @@ -171,7 +171,7 @@ In some cases, the `PredictionResults` output might not include the correct pred

The RunInference API currently expects outputs to be an `Iterable[Any]`. Example return types are `Iterable[Tensor]` or `Iterable[Dict[str, Tensor]]`. When RunInference zips the inputs with the predictions, the predictions iterate over the dictionary keys instead of the batch elements. The result is that the key name is preserved but the prediction tensors are discarded. For more information, see the [Pytorch RunInference PredictionResult is a Dict](https://github.com/apache/beam/issues/22240) issue in the Apache Beam GitHub project.

To work with the current RunInference implementation, you can create a wrapper class that overrides the `model(input)` call. In PyTorch, for example, your wrapper would override the `forward()` function and return an output with the appropriate format of `List[Dict[str, torch.Tensor]]`. For more information, see our [HuggingFace language modeling example](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_language_modeling.py#L49).
To work with the current RunInference implementation, you can create a wrapper class that overrides the `model(input)` call. In PyTorch, for example, your wrapper would override the `forward()` function and return an output with the appropriate format of `List[Dict[str, torch.Tensor]]`. For more information, see our [HuggingFace language modeling example](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_language_modeling.py#L49) and our [Bert language modeling example](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_language_modeling.py).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are the same links, looks like not the change we intended to make?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my last comment referred to disable batching section

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. This should be fixed now.


### Unable to batch tensor elements

Expand Down