Skip to content

chamikaramj/multi-language-pipelines

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 

Repository files navigation

Example multi-language pipelines

  • Example 1 - JavaPrefix: A python pipeline that reads a text file and attaches a prefix in Java side to each input.
  • Example 2 - JavaCount: A Python WordCount example that utilizes a Count transform in Java side.

Instructions for running the pipelines

  • Start the expansion service by running script start_expansion_service.sh

  • To run the pipeline using DirectRunner, run corresponding run_<example>_direct.sh script.

  • To run the pipeline using DataflowRunner,

    • Follow the instructions here to setup a GCP project
    • Open the corresponding run_<example>_dataflow.sh script
    • Update the enironment variables at the top of the script.
    • Save and run the script.

About

Code related to Beam multi-language pipelines.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published