Add error reporting for BatchConverter match failure #24022

TheNeuralBit · 2022-11-07T21:59:36Z

This PR changes the contract for registered BatchConverter constructors (internal API). Previously a mismatched constructor should return None - now it should raise a TypeError with an explanation of why it is mismatched. When we fail to match any BatchConverters, an exception is raised that includes a summary of all the mismatch reasons.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

codecov · 2022-11-07T22:30:12Z

Codecov Report

Merging #24022 (4118956) into master (380d473) will increase coverage by 0.01%.
The diff coverage is 81.66%.

@@            Coverage Diff             @@
##           master   #24022      +/-   ##
==========================================
+ Coverage   73.39%   73.41%   +0.01%     
==========================================
  Files         710      710              
  Lines       96122    96174      +52     
==========================================
+ Hits        70553    70604      +51     
- Misses      24253    24254       +1     
  Partials     1316     1316

Flag	Coverage Δ
python	`83.21% <81.66%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...pache_beam/typehints/pytorch_type_compatibility.py	`0.00% <0.00%> (ø)`
sdks/python/apache_beam/typehints/batch.py	`92.16% <93.54%> (+1.08%)`	⬆️
.../apache_beam/typehints/arrow_type_compatibility.py	`86.82% <100.00%> (+0.57%)`	⬆️
...apache_beam/typehints/pandas_type_compatibility.py	`98.29% <100.00%> (+2.49%)`	⬆️
.../python/apache_beam/testing/test_stream_service.py	`88.09% <0.00%> (-4.77%)`	⬇️
...hon/apache_beam/runners/worker/worker_pool_main.py	`56.32% <0.00%> (-2.94%)`	⬇️
.../python/apache_beam/transforms/periodicsequence.py	`97.01% <0.00%> (-1.50%)`	⬇️
...che_beam/runners/interactive/interactive_runner.py	`90.50% <0.00%> (-1.27%)`	⬇️
...ks/python/apache_beam/runners/worker/sdk_worker.py	`89.08% <0.00%> (-0.17%)`	⬇️
sdks/python/apache_beam/ml/inference/__init__.py	`100.00% <0.00%> (ø)`
... and 8 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

TheNeuralBit · 2022-11-09T00:29:34Z

CC: @robertwb

github-actions · 2022-11-09T03:33:38Z

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @pabloem for label python.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

robertwb

Thanks, this looks good.

sdks/python/apache_beam/typehints/batch.py

TheNeuralBit · 2022-11-15T17:40:06Z

Run Python PreCommit

* add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else

fix test Add error reporting for BatchConverter match failure (apache#24022) * add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else Update automation to use Go 1.19 (apache#24175) Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com> Fix broken json for notebook (apache#24183) Using Teardown context instead of deprecated finalize (apache#24180) * Using Teardown context instead of deprecated finalize * making function public Co-authored-by: Scott Strong <scott.strong@wunderkind.co> [Python]Support pipe operator as Union (PEP -604) (apache#24106) Fixes apache#21972 Add custom inference function support to the PyTorch model handler (apache#24062) * Initial type def and function signature * [Draft] Add custom inference fn support to Pytorch Model Handler * Formatting * Split out default * Remove Keyed version for testing * Move device optimization * Make default available for import, add to test classes * Remove incorrect default from keyed test * Keyed impl * Fix device arg * custom inference test * formatting * Add helpers to define custom inference functions using model methods * Trailing whitespace * Unit tests * Fix incorrect getattr syntax * Type typo * Fix docstring * Fix keyed helper, add basic generate route * Modify generate() to be different than forward() * formatting * Remove extra generate() def Strip FGAC database role from changestreams metadata requests (apache#24177) Co-authored-by: Doug Judd <nuggetwheat@google.com> Updated README of Interactive Beam Removed deprecated cache_dir runner param in favor of the cache_root global option. Minor update Fix arguments to checkState in BatchViewOverrides Re-use serializable pipeline options when already available (apache#24192) Fix Python PostCommit Example CustomPTransformIT on portable (apache#24159) * Fix Python PostCommit Examples on portable * Fix custom_ptransform pipeline options gets modified * Specify flinkConfDir revert upgrade to go 1.19 for action unit tests (apache#24189) Use only ValueProviders in SpannerConfig (apache#24156) [Tour of Beam] [Frontend] Content tree URLs (apache#23776) * Content tree navigation (apache#23593) Unit content navigation (apache#23593) Update URL on node click (apache#23593) Active unit color (apache#23593) removeListener in unit (apache#23593) First unit is opened on group title click (apache#23593) WIP by Alexey Inkin (apache#23593) selectedUnitColor (apache#23593) Unit borderRadius (apache#23593) RegExp todo (apache#23593) added referenced collection package to remove warning (apache#23593) small refinement (apache#23593) expand on group tap, padding, openNode (apache#23593) group expansion bug fix (apache#23593) selected & unselected progress indicators (apache#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (apache#23593) * fixed _getNodeAncestors (apache#23593) * get sdkId (apache#23593) * addressing comments (apache#23593) * sdkId getter & StatelessExpansionTile (apache#23593) * expand & collapse group (apache#23593) * StatelessExpansionTile (apache#23593) * license (apache#23593) * ValueChanged and ValueKey in StatelessExpansionTile (apache#23593) Co-authored-by: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Co-authored-by: Alexey Inkin <alexey.inkin@akvelon.com> refs: issue-24196, fix broken hyperlink Add a reference to Java RunInference example Python TextIO Performance Test (apache#23951) * Python TextIO Performance Test * Add filebasedio_perf_test module for unified test framework for Python file-based IOs * Fix MetricsReader publishes metrics duplicately if more than one load test declared. This is because MetricsReader.publishers was static class variable * Fix pylint * Distribute Python performance tests random time at a day instead of all at 3PM * Add information about length conversion Fix PythonLint (apache#24219) Bump loader-utils from 1.4.1 to 1.4.2 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (apache#24191) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Temporary update Python RC validation job updates updates Uses _all to follow alias/datastreams when estimating index size Fixes apache#24117 Adds test for following aliases when estimating index size Bump github.com/aws/aws-sdk-go-v2/config from 1.18.0 to 1.18.1 in /sdks (apache#24222) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.0 to 1.18.1. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go-v2@config/v1.18.0...config/v1.18.1) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Add enableGzipHttpCompression option to SplunkIO (apache#24197) add enableBatchLogs as SplunkIO option spotless fix issue with setEnableBatchLogs

AuthNotifier draft (#23692) Comments (#23692) Comments (#23692)(1) sign in with google works (#23692) new configs (#23692) get user progress draft (#23692) comment fixes (#23692) sign in in IntroTextBody (#23692) reverted config (#23692) comment fixes (#23692) WIP before rebase (merge) (#23692) Squashed commit of the following: commit bff4919ff00ec3b5d7186efde41c884dfc4c8344 Merge: 79ba69483a ce8d618c77 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Thu Nov 17 10:34:02 2022 +0100 Merge pull request #24186: Uses _all to follow alias/datastreams when estimating index size commit 79ba69483a84ea0278d0b0ddb141200739607c77 Merge: 245fea9040 b7e860a762 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 20:47:40 2022 -0800 Merge pull request #24218: Update Python wheel format for RC validation commit 245fea904014cd58d4148807463dbaa40000774c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Nov 16 18:12:33 2022 -0800 Bump loader-utils from 1.4.1 to 1.4.2 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (#24191) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit e1de8e78deeb5d17617fda6591429eaaf8abb8a2 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 20:48:06 2022 -0500 Fix PythonLint (#24219) commit b7e860a7621771c300dcec625655f87e62591323 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:28:31 2022 -0800 updates commit c2feb09ea49dd815b69c65e531ce34128756d988 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:06:08 2022 -0800 updates commit ce8d618c77d23e20a1ddb128bb8183048597d096 Author: egalpin <egalpin@users.noreply.github.com> Date: Wed Nov 16 16:43:57 2022 -0800 Adds test for following aliases when estimating index size commit 959719d01c627328c0ca2849d2b7e2c9b322d4d1 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 15:16:06 2022 -0800 Temporary update Python RC validation job commit b952b41788acc20edbe5b75b2196f30dbf8fdeb0 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 14:18:12 2022 -0500 Python TextIO Performance Test (#23951) * Python TextIO Performance Test * Add filebasedio_perf_test module for unified test framework for Python file-based IOs * Fix MetricsReader publishes metrics duplicately if more than one load test declared. This is because MetricsReader.publishers was static class variable * Fix pylint * Distribute Python performance tests random time at a day instead of all at 3PM * Add information about length conversion commit 017f2cbde124af40a43be99ec88289fcf63c1c95 Merge: fef8acdbc0 88dba4f494 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 10:39:52 2022 -0800 Merge pull request #24187: Add a reference to Java RunInference example commit fef8acdbc0ecbcc85b49144adaf8830e3bc6b2de Merge: 6e9187e67e ead245539d Author: Ahmet Altay <aaltay@gmail.com> Date: Wed Nov 16 10:24:53 2022 -0800 Merge pull request #24199 from Laksh47/issue#24196 refs: issue-24196, fix broken hyperlink commit 6e9187e67e1bd8f73997f437f0ed4c29880ed73b Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Wed Nov 16 22:33:50 2022 +0600 [Tour of Beam] [Frontend] Content tree URLs (#23776) * Content tree navigation (#23593) Unit content navigation (#23593) Update URL on node click (#23593) Active unit color (#23593) removeListener in unit (#23593) First unit is opened on group title click (#23593) WIP by Alexey Inkin (#23593) selectedUnitColor (#23593) Unit borderRadius (#23593) RegExp todo (#23593) added referenced collection package to remove warning (#23593) small refinement (#23593) expand on group tap, padding, openNode (#23593) group expansion bug fix (#23593) selected & unselected progress indicators (#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (#23593) * fixed _getNodeAncestors (#23593) * get sdkId (#23593) * addressing comments (#23593) * sdkId getter & StatelessExpansionTile (#23593) * expand & collapse group (#23593) * StatelessExpansionTile (#23593) * license (#23593) * ValueChanged and ValueKey in StatelessExpansionTile (#23593) Co-authored-by: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Co-authored-by: Alexey Inkin <alexey.inkin@akvelon.com> commit b33fac2aa533d77cfa47f88466c8cd6bd3f3e864 Author: Bruno Volpato <bvolpato@google.com> Date: Wed Nov 16 10:51:11 2022 -0500 Use only ValueProviders in SpannerConfig (#24156) commit 5f013ab6567ec75b460b2081d7f89d332320caff Author: Robert Burke <lostluck@users.noreply.github.com> Date: Wed Nov 16 07:23:10 2022 -0800 revert upgrade to go 1.19 for action unit tests (#24189) commit 9337f4dbecc929886f8559949a082a649fd9d1bb Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 10:18:42 2022 -0500 Fix Python PostCommit Example CustomPTransformIT on portable (#24159) * Fix Python PostCommit Examples on portable * Fix custom_ptransform pipeline options gets modified * Specify flinkConfDir commit ead245539d01dec0f3e08699c1e1cc6777a5ef0e Author: Laksh <lakshmanansathya@gmail.com> Date: Wed Nov 16 09:32:46 2022 -0500 refs: issue-24196, fix broken hyperlink commit e83a996d4374d467d95bcfad7166905622ec615c Merge: 2fc56ec663 ffdee0b6ed Author: Jan Lukavský <je.ik@seznam.cz> Date: Wed Nov 16 15:15:31 2022 +0100 Merge pull request #24192: Re-use serializable pipeline options when already available. commit ffdee0b6edb8638c78a65ec85c727ea5dde1cb2f Author: Jozef Vilcek <jvilcek@zetaglobal.com> Date: Mon Nov 14 16:48:18 2022 +0100 Re-use serializable pipeline options when already available (#24192) commit 88dba4f494829b2b3530b767fb8c5252e0d2ba44 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Tue Nov 15 16:21:22 2022 -0800 Add a reference to Java RunInference example commit 2fc56ec663e335cfcf37dc57d471f79b601414f4 Merge: f763186987 83f1bc19b9 Author: Kenn Knowles <kenn@apache.org> Date: Tue Nov 15 16:16:47 2022 -0800 Merge pull request #24142: Fix arguments to checkState in BatchViewOverrides commit f763186987c00ba1d26efdc35406436a1fa69a9a Merge: c2bc2135e9 0d7ca04182 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 15:25:20 2022 -0800 Addresses #24161 Updated README of Interactive Beam commit c2bc2135e9bce715990a5d5551e2bc2dc0311da4 Author: Doug Judd <nuggetwheat@gmail.com> Date: Tue Nov 15 14:48:26 2022 -0800 Strip FGAC database role from changestreams metadata requests (#24177) Co-authored-by: Doug Judd <nuggetwheat@google.com> commit af637974f96ad1b5110d7dea3f9a26c68e19a51b Author: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Date: Tue Nov 15 17:16:43 2022 -0500 Add custom inference function support to the PyTorch model handler (#24062) * Initial type def and function signature * [Draft] Add custom inference fn support to Pytorch Model Handler * Formatting * Split out default * Remove Keyed version for testing * Move device optimization * Make default available for import, add to test classes * Remove incorrect default from keyed test * Keyed impl * Fix device arg * custom inference test * formatting * Add helpers to define custom inference functions using model methods * Trailing whitespace * Unit tests * Fix incorrect getattr syntax * Type typo * Fix docstring * Fix keyed helper, add basic generate route * Modify generate() to be different than forward() * formatting * Remove extra generate() def commit a014637106970a0a0e9eb7944aa5caf79fa5fd37 Author: egalpin <egalpin@users.noreply.github.com> Date: Tue Nov 15 13:57:54 2022 -0800 Uses _all to follow alias/datastreams when estimating index size Fixes #24117 commit 0d7ca041823bc2b09f76f86fdfd1d0b9508c9c88 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:57:27 2022 -0800 Minor update commit e8fc759d756f4a987e41d2b9da56b906a6cd7736 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:52:18 2022 -0800 Updated README of Interactive Beam Removed deprecated cache_dir runner param in favor of the cache_root global option. commit 08d5f72e5f35d41f3e9fa9fe799caea6bed1b7a7 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Tue Nov 15 16:34:21 2022 -0500 [Python]Support pipe operator as Union (PEP -604) (#24106) Fixes https://github.com/apache/beam/issues/21972 commit 526e7a58b62682582c27173ab21ed8667ddab766 Author: Scott Strong <scott.strong87@gmail.com> Date: Tue Nov 15 16:26:45 2022 -0500 Using Teardown context instead of deprecated finalize (#24180) * Using Teardown context instead of deprecated finalize * making function public Co-authored-by: Scott Strong <scott.strong@wunderkind.co> commit fb4d1d4dea7b26ed538a9f6aca0ed41e8c300e37 Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 16:25:22 2022 -0500 Fix broken json for notebook (#24183) commit f98db2008a97f4546d036ddf0dddfee8c87eb58a Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 15 12:49:23 2022 -0800 Update automation to use Go 1.19 (#24175) Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com> commit e5f58504eef1fdeebe0402cda8a2df259169c704 Author: Brian Hulette <bhulette@google.com> Date: Tue Nov 15 12:25:13 2022 -0800 Add error reporting for BatchConverter match failure (#24022) * add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else commit 3037747f66f0d71d65b6c65745b4f8942c22f05a Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 14:13:04 2022 -0500 Fix broken notebook (#24179) commit b2b1c739ce37690923891934ee317f799db937a2 Author: MakarkinSAkvelon <67736809+MakarkinSAkvelon@users.noreply.github.com> Date: Tue Nov 15 21:53:06 2022 +0500 [Playground] Move Playground in GKE and Infrastructure change (#23928) * changes to updated master branch * Change workflow * ingress changes * Certificate was added * Updates for cloud build backend * Update main.tf * Create main.tf * Create variables.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Create output.tf * Update output.tf * Update output.tf * Update main.tf * Update build.gradle.kts * Update output.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update output.tf * Update main.tf * Update main.tf * Update output.tf * Create variables.tf * Update main.tf * Update main.tf * Delete playground/terraform/infrastructure/cluddns directory * Update main.tf * Update output.tf * Update output.tf * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update README.md * Update README.md * helm folder name was changed * Update README.md * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Updates to readme * Fix DNS name * HelmChart was changed * Some workflows were changed * Remove unused file * playground-examples return * add license information * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * remove "stg" folder * Update README.md * Remove unused files * DNS Removed * var name changed * remove DNSName from var file * 1 * Clear terraform * remove unused records * gradle check * grade last change * issue fix * fix * 1 * run * test * Index creation for Gradle * Add IndexCreation in gradle * Update README.md * Update README.md * Fix names for Frontend * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Redis fix * services fix * Update variables.tf * change order in gradle * Fix Config.g.dart file issue * Update README.md * Playground workflow update Co-authored-by: Sergey Makarkin <sergey.makarkin@akvelon.com> Co-authored-by: Sergey Makarkin <sergey_makarkin@quicktest1.c.apache-beam-testing.internal> Co-authored-by: ruslan-ikhsan <ruslan.ikhsanov@akvelon.com> Co-authored-by: Alex Kosolapov <alex.kosolapov@gmail.com> commit 85df5f2eb2f299e28b36be0cce7b9c19d62124da Author: Yi Hu <yathu@google.com> Date: Tue Nov 15 11:38:13 2022 -0500 Eliminate CalciteUtil.CharType logical type (#24013) * Eliminate CalciteUtils.CharType logical type * Replace CalciteUtils.CharType to String Note that CalciteUtils still omits the precision of BINARY/VARBINARY/CHAR/VARCHAR as what it originally did. Support of the precision of these calcite types involves make use of making use of the overload method RelDataTypeFactory.createSqlType(var1, var2). * Replace every reference of CalciteUtil.CharType to generic PassThroughLogicalType check * Add TODO to Support sql types with arguments * Use VariableString in LogicalTypeTestCase commit f349f41010c5b238ff6020f7de718f938eef3c5e Author: alexeyinkin <alexey.inkin@akvelon.com> Date: Tue Nov 15 20:04:01 2022 +0400 Configure flutter_code_editor options with Hugo shortcode (#23926) (#24031) * Configure flutter_code_editor options with Hugo shortcode (#23926) * Minor fixes (#23926) * Refactor after review (#23926) commit 0f4ca6363b3ce0e5de3ad36517bb406aa6391a18 Author: Rebecca Szper <98840847+rszper@users.noreply.github.com> Date: Tue Nov 15 06:10:13 2022 -0800 Editorial review of the ML notebooks. (#24125) * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Update examples/notebooks/beam-ml/custom_remote_inference.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback * Update examples/notebooks/beam-ml/run_inference_sklearn.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 5bd34ede026253326ebff1a7e4f9edb5f71b4a2c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 15 07:17:28 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24131) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.3.2 to 1.11.39. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.3.2...feature/s3/manager/v1.11.39) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2ee809fa0ca7689dd0279e186ebc02d9569a8429 Merge: e3b9bdb2e6 563c66d6fd Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Tue Nov 15 11:01:14 2022 +0100 Merge pull request #23065: [Website] Update copy icon styles commit e3b9bdb2e607d85a4017ba7839000e92a0ad83c4 Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:40:50 2022 +0100 [Dockerized Jenkins] Fix build of dockerized jenkins (fixes #24053) (#24054) commit faaac2ab6e010374cb2be0e95a5dd345836a2a2c Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:38:59 2022 +0100 [Dockerized Jenkins] Update README how to use local repo (#24055) commit 689e70b5131620540faf52e2f1e2dca7a36f269d Author: Damon <damondouglas@users.noreply.github.com> Date: Mon Nov 14 17:34:29 2022 -0800 Implement embedded WebAssembly example (#24081) commit e1bf6c42950e8013f35e35fb9fee8017e01e5010 Merge: eddac84126 10337d2868 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 15:22:14 2022 -0800 Merge pull request #24160 Rename the test_splits flag to direct_test_splits. commit eddac841261228a2c63fa9b225c520ae0f853806 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 15:05:05 2022 -0800 More dataset templates to clean up (#24162) commit 2adb68bd12743566cc89b596bf204d7c807eb62d Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 13:28:13 2022 -0800 Adding a quickstart to README for the TS SDK (#23509) * More of a quickstart for the TS SDK * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 10337d28685ad5712e2ad8608977ec5c5e0e6b6b Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 12:46:32 2022 -0800 Rename the test_splits flag to direct_test_splits. This avoids possible flag conflicts. commit 48c70cc30742b45b17a1d18ece2f0d079bee3915 Author: arne-alex <108519096+arne-alex@users.noreply.github.com> Date: Mon Nov 14 21:33:02 2022 +0100 Merge pull request #23333: Track time on Cloud Dataflow streaming data reads and export via heartbeats commit 9c83de646ab52bd0b05e3346190dd55cd68b2a8b Author: Johanna Öjeling <51084516+johannaojeling@users.noreply.github.com> Date: Mon Nov 14 21:19:44 2022 +0100 Add more tests for S3 filesystem (#24138) commit 9e9c6d797ba52b460f83131431c8e53aebbbc9ac Merge: d5d76b9745 c600444e1d Author: Ning Kang <ningkang0957@gmail.com> Date: Mon Nov 14 12:06:15 2022 -0800 Merge pull request #24029 from apache/dependabot/npm_and_yarn/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/loader-utils-1.4.1 Bump loader-utils from 1.4.0 to 1.4.1 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel commit d5d76b974592d45de368ab641647ca5cc4ec12ec Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 15:03:28 2022 -0500 Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils (#23969) * Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils * Add TODO about java.time migration commit 330cc2010c9f4a2d4e30318bf50a4109ec1cd392 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 12:02:10 2022 -0800 Cleanup stale BQ datasets (#24158) * Cleanup stale BQ datasets * addressing comments commit 4a044999b8ed4bcd41f816f3a23ccb5da00c4c38 Merge: e563b9dd2f 5bd75c25de Author: Heejong Lee <heejong@gmail.com> Date: Mon Nov 14 11:16:00 2022 -0800 Merge pull request #24076 from chamikaramj/multilang_java_updates Updates Multi-lang Java quickstart commit e563b9dd2f3aa0484e6cdc08869991b5e438023e Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Mon Nov 14 20:56:35 2022 +0300 [Tour Of Beam] verify that unit exists when saving progress (#24118) * AIO * Update learning/tour-of-beam/backend/integration_tests/auth_test.go Co-authored-by: Danny McCormick <dannymccormick@google.com> * nit Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 774923e0dd089de870bfa5c77063ae2b28f79347 Merge: 71785de528 1ad0cbc445 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 09:52:26 2022 -0800 Merge pull request #24141: Fix checkArgument format in GcsPath commit 71785de52864313c2e3b14fe72a2a63281343617 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 11:54:37 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.17.10 to 1.18.0 in /sdks (#24151) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.17.10 to 1.18.0. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.17.10...config/v1.18.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 50d591d6cb3e799bee4e29dfc593c693a86e6276 Author: Bruno Volpato <bvolpato@google.com> Date: Mon Nov 14 11:50:01 2022 -0500 Change DataflowBatchWorkerHarness doWork error level to INFO (#24135) commit 5a72696bfda09fdb905ba8e58b636f8494ef955f Merge: ee0a5836d6 0633fe9634 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:12:12 2022 -0800 Merge pull request #24149: Remove extraneous jetbrains annotation commit ee0a5836d69b776834eb3bd9b2bd02eb5252c333 Merge: d001a69e1a 137799672e Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:11:00 2022 -0800 Merge pull request #24132: Fix checkArgument format string in AvroIO commit d001a69e1a58701d6ed4fcb5e3fb7a0921301dad Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 10:56:54 2022 -0500 Test Dataproc 2.1 with Flink load tests (#24129) * Test Dataproc 2.1 with Flink load tests * Minor fix flink_cluster script commit caabd9be52887ad70c8a4269395c893811ac6a84 Author: Israel Herraiz <ihr@google.com> Date: Mon Nov 14 16:03:39 2022 +0100 Make MonotonicWatermarkEstimator work like its Java SDK equivalent (#24146) * Make MonotonicWatermarkEstimator work like its Java SDK equivalent The current implementation of MonotonicWatermarkEstimator raises an exception with late messages, which makes the watermark estimator barely usable in real world scenarios. This PR fixes #20041 by making this watermark estimator work like its Java SDK equivalent (`WatermarkEstimators.MonotonicallyIncreasing`). * Update unit tests too * Make linter happy commit 451f6b3e7f58d0a3782ad942c6a1fd9f63932024 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 09:48:23 2022 -0500 Bump golang.org/x/net from 0.1.0 to 0.2.0 in /sdks (#24153) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.1.0 to 0.2.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/compare/v0.1.0...v0.2.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2bb03d62e2d7dc2d8e39040fc9adebccbde74fde Merge: 4e39ef2041 623083cd0a Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 15:01:13 2022 +0100 Merge pull request #24000: [Website] Change headers size from h4,h3 to h2 commit 563c66d6fd32165da14a07747f2764c17a5d24ea Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Wed Sep 7 18:28:42 2022 +0600 [Website] update pre tag copy link styles #23064 commit 4e39ef20410ee51c6040317bcd60171e64c5171f Merge: 223768f782 105ed6fedc Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 10:55:33 2022 +0100 Merge pull request #24115: [Website] update go-dependencies.md java-dependencies.md links commit 223768f782f771f0033b8d0686d86cf4c71fad75 Merge: aa0a35dabf a9da2abee6 Author: Kenn Knowles <kenn@apache.org> Date: Sun Nov 13 18:53:13 2022 -0800 Merge pull request #24136: Fix checkArgument format string in ExecutionStateTracker commit 0633fe9634fe61df7cbc0ecac205d81124fd504a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 15:15:16 2022 -0800 Remove extraneous jetbrains annotation commit 83f1bc19b95935e60ca1f4027d4b60c7e738a84a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 14:16:09 2022 -0800 Fix arguments to checkState in BatchViewOverrides commit 1ad0cbc44594d8405bf4b07a126265238013a02a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 13:41:02 2022 -0800 Fix checkArgument format in GcsPath commit aa0a35dabf9c2a0d9822faff06d939d9a77a3ab6 Author: Kenn Knowles <kenn@apache.org> Date: Fri Nov 11 20:26:30 2022 -0800 Fix checkArgument format string in TestStream (#24134) commit a9da2abee6455bc2cf0f18ba5f6cd7bbaeae669f Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 16:54:27 2022 -0800 Fix checkArgument format string in ExecutionStateTracker commit 369e2ba8622d3474c14c39b941b2c618842d1e47 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Fri Nov 11 19:46:07 2022 -0500 Add a ValidatesContainer integration test for use_sibling_sdk_workers (#24099) commit 137799672eb559a7586262e6a8a73d1ab3580e44 Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 15:30:01 2022 -0800 Fix checkArgument format string in AvroIO commit 5d2dbf957e4e82fb3980726940df02ac67e563cd Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Fri Nov 11 15:57:28 2022 -0500 Update staging of Python wheels (#24114) Fixes https://github.com/apache/beam/issues/24110 commit c2021bee1eba0322b43c90841397859048296b21 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 15:33:14 2022 -0500 Bump google.golang.org/api from 0.102.0 to 0.103.0 in /sdks (#24049) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.102.0 to 0.103.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.102.0...v0.103.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6557c91c79480b9d90573d52d257a11c2b160196 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 11:47:12 2022 -0800 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24112) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.29.1 to 1.29.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.29.1...service/s3/v1.29.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 96f9da1ab652156cd143d57e3aa3d94836338f2b Author: Yi Hu <yathu@google.com> Date: Fri Nov 11 14:34:22 2022 -0500 More cleanup containers (#24105) * More cleanup containers * prebuilt_beam_sdk/beam_python_prebuilt_sdk no longer exists in gcr.io Add correct paths to cleanup * Bump grace time to 15 days * Run job daily as in code comment * Set grace period to 30 days commit 836766ddb83d37463e4b036f156b8d7e15e3864b Author: Ritesh Ghorse <riteshghorse@gmail.com> Date: Fri Nov 11 14:30:30 2022 -0500 upgrade testcontainer dependency (#24123) commit 9fcd20c3712536f2d4580beead678cdbb6fd4746 Author: Damon <damondouglas@users.noreply.github.com> Date: Fri Nov 11 11:12:11 2022 -0800 Implement PubsubRowToMessage transform (#23897) * Begin PubsubRowToMessage Impl * Complete working draft * Unit tests validate user and non-user fields * Finish tests on supporting methods * Pass checks before finalizing tests * WIP * fix timestamp * finalize tests * Finalize code comments * Clean up check findings * Add InputSchemaFactory * Patch code comment typo commit 3a6fcc1ca4d07a467464ed1214a94b5c9c147295 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Fri Nov 11 19:13:45 2022 +0300 disable (#24121) commit 027fb142038b45c443d87af96cac082264c43188 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 10:26:23 2022 -0500 Bump cloud.google.com/go/bigtable from 1.17.0 to 1.18.0 in /sdks (#24113) Bumps [cloud.google.com/go/bigtable](https://github.com/googleapis/google-cloud-go) from 1.17.0 to 1.18.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.17.0...pubsub/v1.18.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/bigtable dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit b2d28a64bb0837dd97e8be7f97f31f88d380f110 Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 11 08:45:14 2022 -0500 Add TFX support in pydoc (#23960) * Add TFX support in pydoc * Wording commit 105ed6fedcb6ff33d4a43940c342a653e67cb25e Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Thu Nov 10 17:47:54 2022 +0600 [Website] update go-dependencies.md java-dependencies.md programming-guide.md links #24084 commit 4b96193250a63b27721a8c5e1a3bd6ecff983093 Author: Brian Hulette <bhulette@google.com> Date: Thu Nov 10 16:51:06 2022 -0800 Remove TheNeuralBit from the pool of Python reviewers (#24108) commit 8f8f089a5f565f179905984cef3522a0577d9219 Author: Sanil Jain <snjain@linkedin.com> Date: Thu Nov 10 16:47:31 2022 -0800 Wire SamzaPipelineOptions to Exeption listener interface (#24109) commit b3186ba91f9bb22764d47b78eb2b7ff017080f75 Author: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> Date: Thu Nov 10 18:04:50 2022 -0500 Support using BigQueryIO Storage Read API with SchemaTransforms (#23827) * support schema transform for bq direct read method * use vendor Strings import * add BigQueryServices argument to config object * suppress nullability errors * add package-info.java file to providers subdirectory * removing accidentally created file * added documentation and moved configuration filclass into provider class * validate config params before expansion. config has a validaate me method * updated URN identifier to follow standards commit a4a94da1936d8cd2f8d1145fd2e8329fe06a2990 Author: Oleh Borysevych <oleg.borisevich@akvelon.com> Date: Fri Nov 11 00:28:43 2022 +0200 fixing linter error (#24104) commit 156a6099d71a155c3379340a0b98256306e1755e Author: Danny McCormick <dannymccormick@google.com> Date: Thu Nov 10 17:26:15 2022 -0500 Add blog post on new ML resources (#24071) * Create ml-resources.md * Add ensemble notebook image * Add image link * Fix image link * Add ml-landing page image * Add image + move around * Add paragraph on upcoming changes * Remove bad whitespace commit 38742d40c895f5d38ee6ee5eb328d11b3262307e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Nov 10 16:12:40 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.5.0 to 1.17.10 in /sdks (#24080) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.5.0 to 1.17.10. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.5.0...config/v1.17.10) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 92cef32915c40d353b51529a87d8f0131b0a3538 Author: BjornPrime <32173247+BjornPrime@users.noreply.github.com> Date: Thu Nov 10 15:30:37 2022 -0500 Num failed inferences (#23830) * created test_increment_num_failed_inferences and test_num_failed_inferences_no_failures * added assertRaises to test_increment_num_failed_inferences * added num_failed_inferences to _MetricsCollector * changed error handling and update() implementation * updated metric name in tests * removed unnecessary else blocking * removed unnecessary inference_args from test_increment_failed_batches_counter() * changed final test_increment_failed_batches_counter assertion * clarified error handling and updated failed_batches_counter initialization * decreased examples array length to 1 to ensure repeatability * troubleshooting tests * trying to get test_increment_failed_batches_counter to fail as expected * corrected assertion details * simplified assertRaises and added reminder comment to assertEqual counter * lint test * lint test passed, resetting pre-commit-config.yaml * fixed lingering linting issues * shortened comment line to comply with linting * formatter worked its magic commit 2341f61d48d838ddaf2e4fb990a49987244c3513 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Thu Nov 10 21:24:55 2022 +0300 [Tour Of Beam] handle CORS pre-flight requests (#24083) * cors * README * nit * nit * -headers, -cache * allow-headers * +1h cache,comment commit 36b0c8f1a76865e4dc3b1eaec1566c3ef9fd4345 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Thu Nov 10 21:21:21 2022 +0300 [Playground] update snippet by persistence_key (#24056) * proto * AIO * -frontend * testify * +license * -build * index * skipKey * Update playground/backend/internal/db/datastore/datastore_db.go Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit df553d1a4c6c6cc9b37ada2a6c84c88d8aef38b9 Author: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> Date: Thu Nov 10 13:17:36 2022 -0500 Add random string at the end of BigQuery query job name to make it resilient to retries (#24041) * add random string at the end of query job name * use deterministic temp table name commit e439f4120ef4c25aa36e5b03756dc7391bdbd211 Author: Pablo <pabloem@users.noreply.github.com> Date: Thu Nov 10 10:06:26 2022 -0800 Improving stale container cleanup script (#24040) * Improving stale container cleanup script * Avoid also latest image * Update .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh Co-authored-by: Yi Hu <huuyyi@gmail.com> * Update .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh Co-authored-by: Yi Hu <huuyyi@gmail.com> Co-authored-by: Yi Hu <huuyyi@gmail.com> commit 8d585242e158babd8dd2ca5a6d5a49d353be2935 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Thu Nov 10 12:58:41 2022 -0500 [Python]Set pickle library at the Pipeline creation stage (#24069) Fixes https://github.com/apache/beam/issues/21615 commit 08b6a524fecc333a12ce42971733ef64ae7d02f1 Author: Janek Bevendorff <janek.bevendorff@uni-weimar.de> Date: Thu Nov 10 18:37:09 2022 +0100 [BEAM-12792] Install pipline dependencies to temporary venv (#16658) commit 74f87b0282a23aa01ecb63288013311d207960aa Merge: 38a85b288a 383ea77bc9 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 10 09:10:37 2022 -0800 Merge pull request #23985 Support dynamic sharding in the worker. commit 38a85b288ad10171c444d4092eb969ca5de58af3 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Nov 10 10:39:15 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24077) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.11.1 to 1.29.1. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.11.1...service/s3/v1.29.1) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit c5110bae06fb9db6cace1f7657745fa97e3f58af Author: Yi Hu <yathu@google.com> Date: Thu Nov 10 10:33:55 2022 -0500 Fix FhirIO javadoc format broken (#24072) commit 623083cd0a72d86c7ecba7653e9a8564861abc4d Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Mon Nov 7 16:08:39 2022 +0600 [Website] change headers size from h4,h3 to h2 #24082 commit 5bd75c25de291e517cc5c5799ae4adaaaaceacb7 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 9 18:04:28 2022 -0800 Updates Multi-lang Java quickstart commit 4522f4ce916223afb3c4edd02a7be6018d0ce5a4 Author: Trevor Gevers <tlgeverssa@gmail.com> Date: Wed Nov 9 18:56:18 2022 -0600 Update datastore_wordcount.py (#23724) commit 96cc252348ce7229731be3c74bf5848d1b004c08 Author: tvalentyn <tvalentyn@users.noreply.github.com> Date: Wed Nov 9 16:08:00 2022 -0800 Remove a duplicate label (#24043) commit 62a6bd1e9317a7658810e9a0eebfe925460e8982 Author: Thiago Nunes <thiagotnunes@google.com> Date: Thu Nov 10 08:45:27 2022 +1100 test: add more tests to throughput estimator (#23915) Verifies that getting the throughput when no updates have occurred for the size of the window should return 0. commit e6647c34bb367b3058c42d0201150395c534e7ce Author: Ritesh Ghorse <riteshghorse@gmail.com> Date: Wed Nov 9 15:12:59 2022 -0500 skip output coder field in exp request (#24066) commit 383ea77bc982416afcbbbe11a4dc1f1424670589 Author: Robert Bradshaw <robertwb@gmail.com> Date: Wed Nov 9 10:49:37 2022 -0800 Reduce flakiness of time-based split manager test. Increase wait time from a tenth to half a second. commit 539fa9159ffc116b2e79e6de2804dfdd1c1e4722 Author: Danny McCormick <dannymccormick@google.com> Date: Wed Nov 9 11:45:20 2022 -0500 Convert initialisms to all caps (#24061) * Convert initialisms to all caps * Fix test refs commit 4941b9633c6ccf9eb8aad554186aac24ca7b7492 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Wed Nov 9 11:25:48 2022 -0500 added comments for tensorflow notebook (#23726) * added comments for tensorflow notebook * added note to string type commit 858f5048f578548a63e5e4319a7ab5363311c269 Merge: 95c121b254 1929968891 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Wed Nov 9 16:35:19 2022 +0100 Merge pull request #24057: [Spark Dataset runner] Enable projection pushdown for Spark dataset runner commit 95c121b2549d3f8fda7668e3759b8991b55d9865 Author: Oleh Borysevych <oleg.borisevich@akvelon.com> Date: Wed Nov 9 16:03:33 2022 +0200 Fix dependency mismatch in Playground Java runner (#24059) * fixing existing and potential dependency mismatch * extract grpc version commit 1929968891d629c83249c574c28db6c103ba9271 Author: Moritz Mack <mmack@talend.com> Date: Wed Nov 9 13:35:12 2022 +0100 [Spark Dataset runner] Enable projection pushdown for Spark dataset runner. commit 63362f5ba60a22e77e54c2df47a844f031036309 Author: Johanna Öjeling <51084516+johannaojeling@users.noreply.github.com> Date: Wed Nov 9 07:49:54 2022 +0100 [Go SDK] S3 implementation of the Beam filesystem (#23992) * Implement filesystem for S3 * Update CHANGES.md commit 73142bad2472e4516831c24a8dfb05c9acebf791 Author: Kanishk Karanawat <kkdoon@gmail.com> Date: Tue Nov 8 20:46:04 2022 -0500 Handle Avro schema generation for logical data types in BigQueryAvroUtils (#23620): handle avro logical types for TableFieldSchema to Avro schema conversion Co-authored-by: Kanishk Karanawat <kkaranawat@twitter.com> commit 8db8fae173056c0d89a36a8a5ce0443b6b9e54fa Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 8 15:56:53 2022 -0800 [Go] Add pipeline resource hints to CHANGES.md (#24036) commit a84b7dc179fed5dfc1d48dddfb04266de3716e31 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 16:35:25 2022 -0800 Make mypy happy. commit b9655e7de1a682d8ec4efcafb4d610f794e1b40e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 8 12:27:38 2022 -0800 Bump cloud.google.com/go/storage from 1.27.0 to 1.28.0 in /sdks (#24028) Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.27.0 to 1.28.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.27.0...spanner/v1.28.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/storage dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 708e05f0dc6c0a2e62674cd0ecb8a6744ae57d3a Author: Brian Hulette <bhulette@google.com> Date: Tue Nov 8 12:18:23 2022 -0800 Retroactively announce Batched DoFn support in 2.42.0 Blog (#24011) * Retroactively announce Batched DoFn support in 2.42.0 * Add to blog as well commit b9ff75a18b40141a8d2340e397fd70382b3d4e8f Author: Israel Herraiz <ihr@google.com> Date: Tue Nov 8 20:03:43 2022 +0100 Update my Twitter handle (#23653) commit bf621ccb88023e0bc09752978c099ac4629086e2 Author: Rebecca Szper <98840847+rszper@users.noreply.github.com> Date: Tue Nov 8 10:50:02 2022 -0800 Editorial review of the ML base API descriptions (#24026) commit f0a6ff46619ee1fd0639fa3f06e04708f69dee74 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 8 13:04:03 2022 -0500 Bump cloud.google.com/go/bigtable from 1.16.0 to 1.17.0 in /sdks (#24027) Bumps [cloud.google.com/go/bigtable](https://github.com/googleapis/google-cloud-go) from 1.16.0 to 1.17.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.16.0...pubsub/v1.17.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/bigtable dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 7dac3f5ef40b5d24b24d9ce5bec4717284260b85 Author: Lukasz Cwik <lcwik@google.com> Date: Tue Nov 8 09:42:29 2022 -0800 [#21250] Trivial removal of loop over something that always has one element (#24014) Multiplexing was put into the PCollectionConsumerRegistry a long time ago and this seems to have been missed during that migration. commit a6a9b23fd5b74142c35948f10d1840b882817246 Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 8 09:19:49 2022 -0800 [Go] Pipeline Resource Hints (#23990) commit 8cf2a63cc25399109f64b84ddfcd7c74d21e61ce Author: tvalentyn <tvalentyn@users.noreply.github.com> Date: Tue Nov 8 09:19:16 2022 -0800 Update release notes. (#23986) commit c600444e1dfa2f4afaad056bc65fa009a60a32bb Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 8 06:22:14 2022 +0000 Bump loader-utils Bumps [loader-utils](https://github.com/webpack/loader-utils) from 1.4.0 to 1.4.1. - [Release notes](https://github.com/webpack/loader-utils/releases) - [Changelog](https://github.com/webpack/loader-utils/blob/v1.4.1/CHANGELOG.md) - [Commits](https://github.com/webpack/loader-utils/compare/v1.4.0...v1.4.1) --- updated-dependencies: - dependency-name: loader-utils dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> commit 62d8bdc604dfeccae08d00225bce8c9b9c3ffb28 Merge: d4142f7eab 930c4d2d87 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:31:37 2022 -0800 Merge pull request #23789 Better surfacing of Scala support via Scio. commit 7f7713de09b3b0e400ed02ffdfcb34a6a8b6e1e4 Merge: 5f587b9339 d4142f7eab Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:29:27 2022 -0800 Merge branch 'master' into javascript-liquid-sharding commit 5f587b9339b3b19ae5c5b1cf15e3ce218f75d21a Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:25:33 2022 -0800 Clarifying comments. commit 60fe3cdddf9d3c27d9493b345d1f5f2a9284a76f Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:06:26 2022 -0800 Update style sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py Co-authored-by: Danny McCormick <dannymccormick@google.com> commit d4142f7eab7d5a3d21a3d3930c448fe403880f93 Author: Robert Burke <lostluck@users.noreply.github.com> Date: Mon Nov 7 12:11:21 2022 -0800 Switch && for || to fix bug in #23889 resolution (#24017) commit 45100d708d954b25b5a884dc3c7a965f008228c6 Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 14:53:24 2022 -0500 Add files then check cached diff to get untracked files commit 380d4730d3c52b5c2165bdb84aac11d0feccfe4f Author: scwhittle <scwhittle@users.noreply.github.com> Date: Mon Nov 7 19:33:37 2022 +0100 Enforce splitting invariants by ensuring split state is reset in the same synchronized block as window index increments. (#23882) * Enforce splitting invariants by ensuring split state is reset in the same synchronized block as window index increments. Fixes #23881. * Add missing currentElement = null; Co-authored-by: Lukasz Cwik <lcwik@google.com> commit 8b3fd2e0d652045bf94b83638129bb12562ce738 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 7 10:33:26 2022 -0800 Bump google.golang.org/api from 0.101.0 to 0.102.0 in /sdks (#23957) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.101.0 to 0.102.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.101.0...v0.102.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 9533fc3757d3041135cbe2f5078dff0855adc1f1 Author: capthiron <24925821+capthiron@users.noreply.github.com> Date: Mon Nov 7 19:32:18 2022 +0100 feat: implement bigtable io connector with write capabilities (#23411) commit c4218e572aa8a8f46b433b60073a59fdd7fdf148 Merge: 23676a9ec1 a5ee669a43 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 7 18:24:45 2022 +0100 Merge pull request #23934: [Website] change case studies page mobile layout commit 23676a9ec17069b309e8803d1fbfeb8009035828 Author: Oleh Borysevych <oleg.borisevich@akvelon.com> Date: Mon Nov 7 17:24:59 2022 +0200 [Tour of Beam] Learning content for "Introduction" module (#23085) * learning content for introduction module * removing white spaces from md files * delete whitespaces * delete whitespaces in python * delete whitespace #2 * divide pipeline concepts * add pipeline example concepts * adding category tag to python examples * adding category to java examples * adding category to go examples * fixed go example * fixed go example compilation * fixing python duplicate example names * add runner concepts * fixing java examples * add licence for runner unit * some minor fixes for unit names * fixed unit name * resolving CR comments * adding complexity to examples * adding tags * fixed go example compilation * fixed python example with duplicate transform * change indent python * fixing missing pipeline options * change arrow symbol * delete example prefix * minor formatting and readability fixes * add example description * minor fix * minor code review comment Co-authored-by: Abzal Tuganbay <abzal.tugan@gmail.com> commit 98d2ffd69ab9d5c49196a78f7c25daacb57b6478 Author: Brian Hulette <bhulette@google.com> Date: Mon Nov 7 07:03:12 2022 -0800 Make `documentation/io/connectors/` canonical (#23877) * Drop documentation/io/built-in/ * Update links in Custom I/O pattern commit c0b41fa045418beb782c0a59e6f5c511da898c53 Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:29:08 2022 -0500 Use git diff instead of git diff-index to avoid file timestamp changes being picked up commit 0476d6498217b7b0e833643c47fef2c4bdaec529 Merge: 1d5fc14031 6ae37b6f82 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 7 15:28:59 2022 +0100 Merge pull request #23962: [CdapIO] Add sparkreceiver:2 module. commit 1d5fc14031d583e92018ebfbeaff9b4e3b978fdb Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:14:24 2022 -0500 Remove quiet flag on debug commit c08273b9355566181d2d5c0792130f8d5cc1428c Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:12:26 2022 -0500 Correctly print diff and swallow empty commits for the moment commit 66fb431deb77db5f7f0fac31e6bf3b2c094c8ac5 Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:08:53 2022 -0500 Print diff and scope to state path commit e0e10b9e5432643b884c381d145e5924cc4ef193 Author: Danny McCormick <dannymccormick@google.com> Date: Sun Nov 6 12:12:22 2022 -0500 PR Bot - Dont throw error on return code 1 commit bbcb790461c47d22399c681cd42c434b206eb550 Author: Danny McCormick <dannymccormick@google.com> Date: Sun Nov 6 11:59:32 2022 -0500 Fix pr bot - exec doesn't allow command chaining commit 88f9a66a9f4cc2fe3259d5ebd83ebb472a29dd5e Author: Danny McCormick <dannymccormick@google.com> Date: Sun Nov 6 11:46:34 2022 -0500 Fix diff to stop repeated bot runs commit ddb4f2d53538fea68383fe43221d4074d05d11b9 Author: bullet03 <bulat.safiullin@akvelon.com> Date: Fri Nov 4 16:11:10 2022 -0700 Website add and update logos (#23899) * [Website] add and update logos * Delete talend.svg:Zone.Identifier commit 6d42219ff547720996489aa29de0d64246db2541 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 14:30:18 2022 -0700 Enable more portable-runner requiring tests. (#23970) * Enable more portable-runner requiring tests. * Run all basic tests on portable runner as well. * Mark ULR tests. commit 1a643d16112a5f3f1418f1be393de51af65d752b Author: David Cavazos <dcavazos@google.com> Date: Fri Nov 4 13:45:12 2022 -0700 Beam starter projects blog post (#23964) Co-authored-by: Ahmet Altay <aaltay@gmail.com> commit d10b4a28fff667289c146152535275035f2dedf7 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Fri Nov 4 16:06:49 2022 -0400 removed trailing whitespace (#23987) commit 3bd697561aab4a5d2aa49d29143695d6d57a834a Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:43:47 2022 -0700 Add dynamic splitting support to the worker. commit ed520243f472c7224a1f8c74a0c2e8f81965cbfc Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:07:40 2022 -0700 Add the a Reshuffle operation and use it in Create. This allows created sources to be staticly and dynamically split. commit 9283512544b8de169be9cd7795b42284fdf45b87 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 10:28:51 2022 -0700 Add the ability to schedule splits on the ULR via a pipeline option. This can be used to test arbitrary SDKs for behavioral correctness for dynamic splitting. These splits are wall-time based, which isn't ideal, but easier than setting up and managing a cross-process synchronization channel. commit ed7b560d7ce7dda0b55a067b9d9a1a9d692a7d7a Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 09:46:19 2022 -0700 Compute element counts for all PCollections. commit 1cfdb127983a55c77b3259c4fdd248497b28e250 Author: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Date: Fri Nov 4 15:43:16 2022 -0400 Add custom inference fn suport to the sklearn model handlers (#23642) * Add custom inference fn suport to the sklearn model handlers * Clean up import order * Update typing, add numpy unit test * Add Pandas unit test * Formatting, linting * yapf run * Remove trailing whitespace * import order * Change inference_fn to keyword-only arg commit 8617b86fd792ac555233dd79c8ede34c2ce67ecf Merge: aa178f8252 2496a0d1e0 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:42:21 2022 -0700 Merge pull request #23978 Add basic counter support to the typescript SDK. commit 2496a0d1e0fda74e8fc0f24eee011362426369a2 Merge: 64abcbe4a3 aa178f8252 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:15:47 2022 -0700 Merge branch 'master' into javascript-metrics commit aa178f825223cbb0a6208fad0e02f98292eeded1 Merge: 97628802fe 1807c307aa Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 09:34:11 2022 -0700 Merge pull request #23976 Correctly capture log levels and attach stage information. commit 64abcbe4a3a15160aba3f3163ff8cbf5186bbfdb Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 09:27:22 2022 -0700 Only report counters that were actually used. commit 1807c307aa1e493acb2d8ba542baffefcb96e7e2 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 08:52:21 2022 -0700 Remove obsolete TODO. commit 97628802fe4cf6640a3b6854e27b410dd5731379 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 4 11:47:07 2022 -0400 Bump cloud.google.com/go/datastore from 1.8.0 to 1.9.0 in /sdks (#23916) Bumps [cloud.google.com/go/datastore](https://github.com/googleapis/google-cloud-go) from 1.8.0 to 1.9.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/asset/v1.8.0...asset/v1.9.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/datastore dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 058c56af735e10184d1501cc08cd16af4d21e64a Author: Ayush Sharma <114604338+ayushthe1@users.noreply.github.com> Date: Fri Nov 4 20:58:11 2022 +0530 [Task]: PR Bot will push commits only if they are non-empty (#23937) * fix pr-bot * fix the git pr bot commit 7dba78d611089a1826c5655ed7ffcdf3f6007eae Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 4 09:35:16 2022 -0400 Immediately truncate full restriction on drain of periodic impulse (#23765) * Immediately truncate full restriction on drain of periodic impulse * Whitespace format commit 16852518861514a56ad1b57113e3815cfcc8533a Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Fri Nov 4 08:39:34 2022 -0400 TFX image classification example (#23456) * TFX image classification example * TFX image classification with serving model * Add TF Model Wrapper * Clean up code * Refactoring * Add a test for tfx_bsl RunInference * fixup lint * Refactoring * Fixup lint * Add brief summary for the example * Apply suggestions from code review Co-authored-by: Andy Ye <andyye333@gmail.com> * Refactoring code and add comments * Update help description * reorder imports * Reorder imports again * Add docstring * Refactoring * Add pillow to tfx pipeline requirements * Move inferencePostCommitIT to Python 3.9 suite * Uncomment other postcommit suites Co-authored-by: Andy Ye <andyye333@gmail.com> commit 30b26171b4038c1c976adbc3be386e118b35153b Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 4 08:22:17 2022 -0400 Update watermark during periodic sequence/impulse (#23507) * Update watermark during periodic sequence/impulse * Remove extraneous import * Formatting * Linting * Only run on dataflow for guaranteed watermark support * More permissive test to avoid timing issues * Test pipeline options * Fix test * Formatting * Formatting * Apply feedback - cleanup/naming/flink * Format * Unused import commit 1563b9721b4bee5a22d384b282762726247e0703 Author: Brian Hulette <bhulette@google.com> Date: Fri Nov 4 05:09:39 2022 -0700 Improve Iterator error message (#23972) commit e3987b5435d39c5fb058d4085eb5e210f8756224 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 17:39:49 2022 -0700 old prettier change commit 89c8dd5ca23591be6c2778045df32c7d08d9f99e Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 17:12:26 2022 -0700 Add distribution metric type. commit e4e606a8f2652c2fa761a95462f230385b6bcb47 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 17:09:03 2022 -0700 Support metrics over the portability API. commit e04f3d6fdb39528264cb2a14e60e565991121f1c Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 16:52:51 2022 -0700 Add basic counter setting and getting to the typescript SDK. commit 1daa9c373ac7fbdb696120eeaa2ca9e85a679a1f Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 16:01:59 2022 -0700 Cleanup worker logging. Attempt to correctly capture and propagate log levels and attaches stage information to them. Also remove some excessively verbose logs. commit d3a173cf16355bfe23a9d918afa12cd61d290b59 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 15:11:40 2022 -0700 Move logging to its own module. commit 6ae37b6f82ae3cd77cc82aff4566d8218a8acc57 Author: vitaly.terentyev <vitaly.terentyev@akvelon.com> Date: Thu Nov 3 19:44:54 2022 +0400 Fix sparkreceiver dependencies commit 8223375d2f9606a29d6498a73c1914a61f3277c4 Author: vitaly.terentyev <vitaly.terentyev@akvelon.com> Date: Thu Nov 3 19:34:04 2022 +0400 Add sparkreceiver:2 module. commit a5ee669a439dc3d13dfc4926c1c5ae2351bb9135 Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Wed Nov 2 13:53:08 2022 +0600 [Website] change case-study-card width on mobile commit 930c4d2d87d462b2457369fd06066203a6628a46 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Oct 21 12:22:13 2022 -0700 Better surfacing of Scala support via Scio. draft working code test commit Revert "Squashed commit of the following:" This reverts commit 8fc833a5e7abf4cbfb63e56d450631bbfc1985ff. auth files

…lection (#24457) * Content tree navigation (#23593) Unit content navigation (#23593) Update URL on node click (#23593) Active unit color (#23593) removeListener in unit (#23593) First unit is opened on group title click (#23593) WIP by Alexey Inkin (#23593) selectedUnitColor (#23593) Unit borderRadius (#23593) RegExp todo (#23593) added referenced collection package to remove warning (#23593) small refinement (#23593) expand on group tap, padding, openNode (#23593) group expansion bug fix (#23593) selected & unselected progress indicators (#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (#23593) * fixed _getNodeAncestors (#23593) * get sdkId (#23593) * addressing comments (#23593) * sdkId getter & StatelessExpansionTile (#23593) * expand & collapse group (#23593) * StatelessExpansionTile (#23593) * license (#23593) * ValueChanged and ValueKey in StatelessExpansionTile (#23593) * optional SDK selector in tour scaffold * AppNotifier * moved AppNotifier into _initializeState * StorageKeys * remove listener * auth, complete unit, user progress AuthNotifier draft (#23692) Comments (#23692) Comments (#23692)(1) sign in with google works (#23692) new configs (#23692) get user progress draft (#23692) comment fixes (#23692) sign in in IntroTextBody (#23692) reverted config (#23692) comment fixes (#23692) WIP before rebase (merge) (#23692) Squashed commit of the following: commit bff4919ff00ec3b5d7186efde41c884dfc4c8344 Merge: 79ba69483a ce8d618c77 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Thu Nov 17 10:34:02 2022 +0100 Merge pull request #24186: Uses _all to follow alias/datastreams when estimating index size commit 79ba69483a84ea0278d0b0ddb141200739607c77 Merge: 245fea9040 b7e860a762 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 20:47:40 2022 -0800 Merge pull request #24218: Update Python wheel format for RC validation commit 245fea904014cd58d4148807463dbaa40000774c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Nov 16 18:12:33 2022 -0800 Bump loader-utils from 1.4.1 to 1.4.2 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (#24191) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit e1de8e78deeb5d17617fda6591429eaaf8abb8a2 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 20:48:06 2022 -0500 Fix PythonLint (#24219) commit b7e860a7621771c300dcec625655f87e62591323 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:28:31 2022 -0800 updates commit c2feb09ea49dd815b69c65e531ce34128756d988 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:06:08 2022 -0800 updates commit ce8d618c77d23e20a1ddb128bb8183048597d096 Author: egalpin <egalpin@users.noreply.github.com> Date: Wed Nov 16 16:43:57 2022 -0800 Adds test for following aliases when estimating index size commit 959719d01c627328c0ca2849d2b7e2c9b322d4d1 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 15:16:06 2022 -0800 Temporary update Python RC validation job commit b952b41788acc20edbe5b75b2196f30dbf8fdeb0 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 14:18:12 2022 -0500 Python TextIO Performance Test (#23951) * Python TextIO Performance Test * Add filebasedio_perf_test module for unified test framework for Python file-based IOs * Fix MetricsReader publishes metrics duplicately if more than one load test declared. This is because MetricsReader.publishers was static class variable * Fix pylint * Distribute Python performance tests random time at a day instead of all at 3PM * Add information about length conversion commit 017f2cbde124af40a43be99ec88289fcf63c1c95 Merge: fef8acdbc0 88dba4f494 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 10:39:52 2022 -0800 Merge pull request #24187: Add a reference to Java RunInference example commit fef8acdbc0ecbcc85b49144adaf8830e3bc6b2de Merge: 6e9187e67e ead245539d Author: Ahmet Altay <aaltay@gmail.com> Date: Wed Nov 16 10:24:53 2022 -0800 Merge pull request #24199 from Laksh47/issue#24196 refs: issue-24196, fix broken hyperlink commit 6e9187e67e1bd8f73997f437f0ed4c29880ed73b Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Wed Nov 16 22:33:50 2022 +0600 [Tour of Beam] [Frontend] Content tree URLs (#23776) * Content tree navigation (#23593) Unit content navigation (#23593) Update URL on node click (#23593) Active unit color (#23593) removeListener in unit (#23593) First unit is opened on group title click (#23593) WIP by Alexey Inkin (#23593) selectedUnitColor (#23593) Unit borderRadius (#23593) RegExp todo (#23593) added referenced collection package to remove warning (#23593) small refinement (#23593) expand on group tap, padding, openNode (#23593) group expansion bug fix (#23593) selected & unselected progress indicators (#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (#23593) * fixed _getNodeAncestors (#23593) * get sdkId (#23593) * addressing comments (#23593) * sdkId getter & StatelessExpansionTile (#23593) * expand & collapse group (#23593) * StatelessExpansionTile (#23593) * license (#23593) * ValueChanged and ValueKey in StatelessExpansionTile (#23593) Co-authored-by: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Co-authored-by: Alexey Inkin <alexey.inkin@akvelon.com> commit b33fac2aa533d77cfa47f88466c8cd6bd3f3e864 Author: Bruno Volpato <bvolpato@google.com> Date: Wed Nov 16 10:51:11 2022 -0500 Use only ValueProviders in SpannerConfig (#24156) commit 5f013ab6567ec75b460b2081d7f89d332320caff Author: Robert Burke <lostluck@users.noreply.github.com> Date: Wed Nov 16 07:23:10 2022 -0800 revert upgrade to go 1.19 for action unit tests (#24189) commit 9337f4dbecc929886f8559949a082a649fd9d1bb Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 10:18:42 2022 -0500 Fix Python PostCommit Example CustomPTransformIT on portable (#24159) * Fix Python PostCommit Examples on portable * Fix custom_ptransform pipeline options gets modified * Specify flinkConfDir commit ead245539d01dec0f3e08699c1e1cc6777a5ef0e Author: Laksh <lakshmanansathya@gmail.com> Date: Wed Nov 16 09:32:46 2022 -0500 refs: issue-24196, fix broken hyperlink commit e83a996d4374d467d95bcfad7166905622ec615c Merge: 2fc56ec663 ffdee0b6ed Author: Jan Lukavský <je.ik@seznam.cz> Date: Wed Nov 16 15:15:31 2022 +0100 Merge pull request #24192: Re-use serializable pipeline options when already available. commit ffdee0b6edb8638c78a65ec85c727ea5dde1cb2f Author: Jozef Vilcek <jvilcek@zetaglobal.com> Date: Mon Nov 14 16:48:18 2022 +0100 Re-use serializable pipeline options when already available (#24192) commit 88dba4f494829b2b3530b767fb8c5252e0d2ba44 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Tue Nov 15 16:21:22 2022 -0800 Add a reference to Java RunInference example commit 2fc56ec663e335cfcf37dc57d471f79b601414f4 Merge: f763186987 83f1bc19b9 Author: Kenn Knowles <kenn@apache.org> Date: Tue Nov 15 16:16:47 2022 -0800 Merge pull request #24142: Fix arguments to checkState in BatchViewOverrides commit f763186987c00ba1d26efdc35406436a1fa69a9a Merge: c2bc2135e9 0d7ca04182 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 15:25:20 2022 -0800 Addresses #24161 Updated README of Interactive Beam commit c2bc2135e9bce715990a5d5551e2bc2dc0311da4 Author: Doug Judd <nuggetwheat@gmail.com> Date: Tue Nov 15 14:48:26 2022 -0800 Strip FGAC database role from changestreams metadata requests (#24177) Co-authored-by: Doug Judd <nuggetwheat@google.com> commit af637974f96ad1b5110d7dea3f9a26c68e19a51b Author: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Date: Tue Nov 15 17:16:43 2022 -0500 Add custom inference function support to the PyTorch model handler (#24062) * Initial type def and function signature * [Draft] Add custom inference fn support to Pytorch Model Handler * Formatting * Split out default * Remove Keyed version for testing * Move device optimization * Make default available for import, add to test classes * Remove incorrect default from keyed test * Keyed impl * Fix device arg * custom inference test * formatting * Add helpers to define custom inference functions using model methods * Trailing whitespace * Unit tests * Fix incorrect getattr syntax * Type typo * Fix docstring * Fix keyed helper, add basic generate route * Modify generate() to be different than forward() * formatting * Remove extra generate() def commit a014637106970a0a0e9eb7944aa5caf79fa5fd37 Author: egalpin <egalpin@users.noreply.github.com> Date: Tue Nov 15 13:57:54 2022 -0800 Uses _all to follow alias/datastreams when estimating index size Fixes #24117 commit 0d7ca041823bc2b09f76f86fdfd1d0b9508c9c88 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:57:27 2022 -0800 Minor update commit e8fc759d756f4a987e41d2b9da56b906a6cd7736 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:52:18 2022 -0800 Updated README of Interactive Beam Removed deprecated cache_dir runner param in favor of the cache_root global option. commit 08d5f72e5f35d41f3e9fa9fe799caea6bed1b7a7 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Tue Nov 15 16:34:21 2022 -0500 [Python]Support pipe operator as Union (PEP -604) (#24106) Fixes https://github.com/apache/beam/issues/21972 commit 526e7a58b62682582c27173ab21ed8667ddab766 Author: Scott Strong <scott.strong87@gmail.com> Date: Tue Nov 15 16:26:45 2022 -0500 Using Teardown context instead of deprecated finalize (#24180) * Using Teardown context instead of deprecated finalize * making function public Co-authored-by: Scott Strong <scott.strong@wunderkind.co> commit fb4d1d4dea7b26ed538a9f6aca0ed41e8c300e37 Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 16:25:22 2022 -0500 Fix broken json for notebook (#24183) commit f98db2008a97f4546d036ddf0dddfee8c87eb58a Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 15 12:49:23 2022 -0800 Update automation to use Go 1.19 (#24175) Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com> commit e5f58504eef1fdeebe0402cda8a2df259169c704 Author: Brian Hulette <bhulette@google.com> Date: Tue Nov 15 12:25:13 2022 -0800 Add error reporting for BatchConverter match failure (#24022) * add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else commit 3037747f66f0d71d65b6c65745b4f8942c22f05a Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 14:13:04 2022 -0500 Fix broken notebook (#24179) commit b2b1c739ce37690923891934ee317f799db937a2 Author: MakarkinSAkvelon <67736809+MakarkinSAkvelon@users.noreply.github.com> Date: Tue Nov 15 21:53:06 2022 +0500 [Playground] Move Playground in GKE and Infrastructure change (#23928) * changes to updated master branch * Change workflow * ingress changes * Certificate was added * Updates for cloud build backend * Update main.tf * Create main.tf * Create variables.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Create output.tf * Update output.tf * Update output.tf * Update main.tf * Update build.gradle.kts * Update output.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update output.tf * Update main.tf * Update main.tf * Update output.tf * Create variables.tf * Update main.tf * Update main.tf * Delete playground/terraform/infrastructure/cluddns directory * Update main.tf * Update output.tf * Update output.tf * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update README.md * Update README.md * helm folder name was changed * Update README.md * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Updates to readme * Fix DNS name * HelmChart was changed * Some workflows were changed * Remove unused file * playground-examples return * add license information * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * remove "stg" folder * Update README.md * Remove unused files * DNS Removed * var name changed * remove DNSName from var file * 1 * Clear terraform * remove unused records * gradle check * grade last change * issue fix * fix * 1 * run * test * Index creation for Gradle * Add IndexCreation in gradle * Update README.md * Update README.md * Fix names for Frontend * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Redis fix * services fix * Update variables.tf * change order in gradle * Fix Config.g.dart file issue * Update README.md * Playground workflow update Co-authored-by: Sergey Makarkin <sergey.makarkin@akvelon.com> Co-authored-by: Sergey Makarkin <sergey_makarkin@quicktest1.c.apache-beam-testing.internal> Co-authored-by: ruslan-ikhsan <ruslan.ikhsanov@akvelon.com> Co-authored-by: Alex Kosolapov <alex.kosolapov@gmail.com> commit 85df5f2eb2f299e28b36be0cce7b9c19d62124da Author: Yi Hu <yathu@google.com> Date: Tue Nov 15 11:38:13 2022 -0500 Eliminate CalciteUtil.CharType logical type (#24013) * Eliminate CalciteUtils.CharType logical type * Replace CalciteUtils.CharType to String Note that CalciteUtils still omits the precision of BINARY/VARBINARY/CHAR/VARCHAR as what it originally did. Support of the precision of these calcite types involves make use of making use of the overload method RelDataTypeFactory.createSqlType(var1, var2). * Replace every reference of CalciteUtil.CharType to generic PassThroughLogicalType check * Add TODO to Support sql types with arguments * Use VariableString in LogicalTypeTestCase commit f349f41010c5b238ff6020f7de718f938eef3c5e Author: alexeyinkin <alexey.inkin@akvelon.com> Date: Tue Nov 15 20:04:01 2022 +0400 Configure flutter_code_editor options with Hugo shortcode (#23926) (#24031) * Configure flutter_code_editor options with Hugo shortcode (#23926) * Minor fixes (#23926) * Refactor after review (#23926) commit 0f4ca6363b3ce0e5de3ad36517bb406aa6391a18 Author: Rebecca Szper <98840847+rszper@users.noreply.github.com> Date: Tue Nov 15 06:10:13 2022 -0800 Editorial review of the ML notebooks. (#24125) * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Update examples/notebooks/beam-ml/custom_remote_inference.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback * Update examples/notebooks/beam-ml/run_inference_sklearn.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 5bd34ede026253326ebff1a7e4f9edb5f71b4a2c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 15 07:17:28 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24131) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.3.2 to 1.11.39. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.3.2...feature/s3/manager/v1.11.39) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2ee809fa0ca7689dd0279e186ebc02d9569a8429 Merge: e3b9bdb2e6 563c66d6fd Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Tue Nov 15 11:01:14 2022 +0100 Merge pull request #23065: [Website] Update copy icon styles commit e3b9bdb2e607d85a4017ba7839000e92a0ad83c4 Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:40:50 2022 +0100 [Dockerized Jenkins] Fix build of dockerized jenkins (fixes #24053) (#24054) commit faaac2ab6e010374cb2be0e95a5dd345836a2a2c Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:38:59 2022 +0100 [Dockerized Jenkins] Update README how to use local repo (#24055) commit 689e70b5131620540faf52e2f1e2dca7a36f269d Author: Damon <damondouglas@users.noreply.github.com> Date: Mon Nov 14 17:34:29 2022 -0800 Implement embedded WebAssembly example (#24081) commit e1bf6c42950e8013f35e35fb9fee8017e01e5010 Merge: eddac84126 10337d2868 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 15:22:14 2022 -0800 Merge pull request #24160 Rename the test_splits flag to direct_test_splits. commit eddac841261228a2c63fa9b225c520ae0f853806 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 15:05:05 2022 -0800 More dataset templates to clean up (#24162) commit 2adb68bd12743566cc89b596bf204d7c807eb62d Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 13:28:13 2022 -0800 Adding a quickstart to README for the TS SDK (#23509) * More of a quickstart for the TS SDK * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 10337d28685ad5712e2ad8608977ec5c5e0e6b6b Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 12:46:32 2022 -0800 Rename the test_splits flag to direct_test_splits. This avoids possible flag conflicts. commit 48c70cc30742b45b17a1d18ece2f0d079bee3915 Author: arne-alex <108519096+arne-alex@users.noreply.github.com> Date: Mon Nov 14 21:33:02 2022 +0100 Merge pull request #23333: Track time on Cloud Dataflow streaming data reads and export via heartbeats commit 9c83de646ab52bd0b05e3346190dd55cd68b2a8b Author: Johanna Öjeling <51084516+johannaojeling@users.noreply.github.com> Date: Mon Nov 14 21:19:44 2022 +0100 Add more tests for S3 filesystem (#24138) commit 9e9c6d797ba52b460f83131431c8e53aebbbc9ac Merge: d5d76b9745 c600444e1d Author: Ning Kang <ningkang0957@gmail.com> Date: Mon Nov 14 12:06:15 2022 -0800 Merge pull request #24029 from apache/dependabot/npm_and_yarn/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/loader-utils-1.4.1 Bump loader-utils from 1.4.0 to 1.4.1 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel commit d5d76b974592d45de368ab641647ca5cc4ec12ec Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 15:03:28 2022 -0500 Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils (#23969) * Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils * Add TODO about java.time migration commit 330cc2010c9f4a2d4e30318bf50a4109ec1cd392 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 12:02:10 2022 -0800 Cleanup stale BQ datasets (#24158) * Cleanup stale BQ datasets * addressing comments commit 4a044999b8ed4bcd41f816f3a23ccb5da00c4c38 Merge: e563b9dd2f 5bd75c25de Author: Heejong Lee <heejong@gmail.com> Date: Mon Nov 14 11:16:00 2022 -0800 Merge pull request #24076 from chamikaramj/multilang_java_updates Updates Multi-lang Java quickstart commit e563b9dd2f3aa0484e6cdc08869991b5e438023e Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Mon Nov 14 20:56:35 2022 +0300 [Tour Of Beam] verify that unit exists when saving progress (#24118) * AIO * Update learning/tour-of-beam/backend/integration_tests/auth_test.go Co-authored-by: Danny McCormick <dannymccormick@google.com> * nit Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 774923e0dd089de870bfa5c77063ae2b28f79347 Merge: 71785de528 1ad0cbc445 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 09:52:26 2022 -0800 Merge pull request #24141: Fix checkArgument format in GcsPath commit 71785de52864313c2e3b14fe72a2a63281343617 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 11:54:37 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.17.10 to 1.18.0 in /sdks (#24151) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.17.10 to 1.18.0. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.17.10...config/v1.18.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 50d591d6cb3e799bee4e29dfc593c693a86e6276 Author: Bruno Volpato <bvolpato@google.com> Date: Mon Nov 14 11:50:01 2022 -0500 Change DataflowBatchWorkerHarness doWork error level to INFO (#24135) commit 5a72696bfda09fdb905ba8e58b636f8494ef955f Merge: ee0a5836d6 0633fe9634 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:12:12 2022 -0800 Merge pull request #24149: Remove extraneous jetbrains annotation commit ee0a5836d69b776834eb3bd9b2bd02eb5252c333 Merge: d001a69e1a 137799672e Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:11:00 2022 -0800 Merge pull request #24132: Fix checkArgument format string in AvroIO commit d001a69e1a58701d6ed4fcb5e3fb7a0921301dad Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 10:56:54 2022 -0500 Test Dataproc 2.1 with Flink load tests (#24129) * Test Dataproc 2.1 with Flink load tests * Minor fix flink_cluster script commit caabd9be52887ad70c8a4269395c893811ac6a84 Author: Israel Herraiz <ihr@google.com> Date: Mon Nov 14 16:03:39 2022 +0100 Make MonotonicWatermarkEstimator work like its Java SDK equivalent (#24146) * Make MonotonicWatermarkEstimator work like its Java SDK equivalent The current implementation of MonotonicWatermarkEstimator raises an exception with late messages, which makes the watermark estimator barely usable in real world scenarios. This PR fixes #20041 by making this watermark estimator work like its Java SDK equivalent (`WatermarkEstimators.MonotonicallyIncreasing`). * Update unit tests too * Make linter happy commit 451f6b3e7f58d0a3782ad942c6a1fd9f63932024 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 09:48:23 2022 -0500 Bump golang.org/x/net from 0.1.0 to 0.2.0 in /sdks (#24153) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.1.0 to 0.2.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/compare/v0.1.0...v0.2.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2bb03d62e2d7dc2d8e39040fc9adebccbde74fde Merge: 4e39ef2041 623083cd0a Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 15:01:13 2022 +0100 Merge pull request #24000: [Website] Change headers size from h4,h3 to h2 commit 563c66d6fd32165da14a07747f2764c17a5d24ea Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Wed Sep 7 18:28:42 2022 +0600 [Website] update pre tag copy link styles #23064 commit 4e39ef20410ee51c6040317bcd60171e64c5171f Merge: 223768f782 105ed6fedc Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 10:55:33 2022 +0100 Merge pull request #24115: [Website] update go-dependencies.md java-dependencies.md links commit 223768f782f771f0033b8d0686d86cf4c71fad75 Merge: aa0a35dabf a9da2abee6 Author: Kenn Knowles <kenn@apache.org> Date: Sun Nov 13 18:53:13 2022 -0800 Merge pull request #24136: Fix checkArgument format string in ExecutionStateTracker commit 0633fe9634fe61df7cbc0ecac205d81124fd504a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 15:15:16 2022 -0800 Remove extraneous jetbrains annotation commit 83f1bc19b95935e60ca1f4027d4b60c7e738a84a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 14:16:09 2022 -0800 Fix arguments to checkState in BatchViewOverrides commit 1ad0cbc44594d8405bf4b07a126265238013a02a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 13:41:02 2022 -0800 Fix checkArgument format in GcsPath commit aa0a35dabf9c2a0d9822faff06d939d9a77a3ab6 Author: Kenn Knowles <kenn@apache.org> Date: Fri Nov 11 20:26:30 2022 -0800 Fix checkArgument format string in TestStream (#24134) commit a9da2abee6455bc2cf0f18ba5f6cd7bbaeae669f Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 16:54:27 2022 -0800 Fix checkArgument format string in ExecutionStateTracker commit 369e2ba8622d3474c14c39b941b2c618842d1e47 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Fri Nov 11 19:46:07 2022 -0500 Add a ValidatesContainer integration test for use_sibling_sdk_workers (#24099) commit 137799672eb559a7586262e6a8a73d1ab3580e44 Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 15:30:01 2022 -0800 Fix checkArgument format string in AvroIO commit 5d2dbf957e4e82fb3980726940df02ac67e563cd Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Fri Nov 11 15:57:28 2022 -0500 Update staging of Python wheels (#24114) Fixes https://github.com/apache/beam/issues/24110 commit c2021bee1eba0322b43c90841397859048296b21 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 15:33:14 2022 -0500 Bump google.golang.org/api from 0.102.0 to 0.103.0 in /sdks (#24049) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.102.0 to 0.103.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.102.0...v0.103.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6557c91c79480b9d90573d52d257a11c2b160196 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 11:47:12 2022 -0800 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24112) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.29.1 to 1.29.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.29.1...service/s3/v1.29.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 96f9da1ab652156cd143d57e3aa3d94836338f2b Author: Yi Hu <yathu@google.com> Date: Fri Nov 11 14:34:22 2022 -0500 More cleanup containers (#24105) * More cleanup containers * prebuilt_beam_sdk/beam_python_prebuilt_sdk no longer exists in gcr.io Add correct paths to cleanup * Bump grace time to 15 days * Run job daily as in code comment * Set grace period to 30 days commit 836766ddb83d37463e4b036f156b8d7e15e3864b Author: Ritesh Ghorse <riteshghorse@gmail.com> Date: Fri Nov 11 14:30:30 2022 -0500 upgrade testcontainer dependency (#24123) commit 9fcd20c3712536f2d4580beead678cdbb6fd4746 Author: Damon <damondouglas@users.noreply.github.com> Date: Fri Nov 11 11:12:11 2022 -0800 Implement PubsubRowToMessage transform (#23897) * Begin PubsubRowToMessage Impl * Complete working draft * Unit tests validate user and non-user fields * Finish tests on supporting methods * Pass checks before finalizing tests * WIP * fix timestamp * finalize tests * Finalize code comments * Clean up check findings * Add InputSchemaFactory * Patch code comment typo commit 3a6fcc1ca4d07a467464ed1214a94b5c9c147295 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Fri Nov 11 19:13:45 2022 +0300 disable (#24121) commit 027fb142038b45c443d87af96cac082264c43188 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 10:26:23 2022 -0500 Bump cloud.google.com/go/bigtable from 1.17.0 to 1.18.0 in /sdks (#24113) Bumps [cloud.google.com/go/bigtable](https://github.com/googleapis/google-cloud-go) from 1.17.0 to 1.18.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.17.0...pubsub/v1.18.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/bigtable dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit b2d28a64bb0837dd97e8be7f97f31f88d380f110 Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 11 08:45:14 2022 -0500 Add TFX support in pydoc (#23960) * Add TFX support in pydoc * Wording commit 105ed6fedcb6ff33d4a43940c342a653e67cb25e Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Thu Nov 10 17:47:54 2022 +0600 [Website] update go-dependencies.md java-dependencies.md programming-guide.md links #24084 commit 4b96193250a63b27721a8c5e1a3bd6ecff983093 Author: Brian Hulette <bhulette@google.com> Date: Thu Nov 10 16:51:06 2022 -0800 Remove TheNeuralBit from the pool of Python reviewers (#24108) commit 8f8f089a5f565f179905984cef3522a0577d9219 Author: Sanil Jain <snjain@linkedin.com> Date: Thu Nov 10 16:47:31 2022 -0800 Wire SamzaPipelineOptions to Exeption listener interface (#24109) commit b3186ba91f9bb22764d47b78eb2b7ff017080f75 Author: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> Date: Thu Nov 10 18:04:50 2022 -0500 Support using BigQueryIO Storage Read API with SchemaTransforms (#23827) * support schema transform for bq direct read method * use vendor Strings import * add BigQueryServices argument to config object * suppress nullability errors * add package-info.java file to providers subdirectory * removing accidentally created file * added documentation and moved configuration filclass into provider class * validate config params before expansion. config has a validaate me method * updated URN identifier to follow standards commit a4a94da1936d8cd2f8d1145fd2e8329fe06a2990 Author: Oleh Borysevych <oleg.borisevich@akvelon.com> Date: Fri Nov 11 00:28:43 2022 +0200 fixing linter error (#24104) commit 156a6099d71a155c3379340a0b98256306e1755e Author: Danny McCormick <dannymccormick@google.com> Date: Thu Nov 10 17:26:15 2022 -0500 Add blog post on new ML resources (#24071) * Create ml-resources.md * Add ensemble notebook image * Add image link * Fix image link * Add ml-landing page image * Add image + move around * Add paragraph on upcoming changes * Remove bad whitespace commit 38742d40c895f5d38ee6ee5eb328d11b3262307e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Nov 10 16:12:40 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.5.0 to 1.17.10 in /sdks (#24080) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.5.0 to 1.17.10. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.5.0...config/v1.17.10) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 92cef32915c40d353b51529a87d8f0131b0a3538 Author: BjornPrime <32173247+BjornPrime@users.noreply.github.com> Date: Thu Nov 10 15:30:37 2022 -0500 Num failed inferences (#23830) * created test_increment_num_failed_inferences and test_num_failed_inferences_no_failures * added assertRaises to test_increment_num_failed_inferences * added num_failed_inferences to _MetricsCollector * changed error handling and update() implementation * updated metric name in tests * removed unnecessary else blocking * removed unnecessary inference_args from test_increment_failed_batches_counter() * changed final test_increment_failed_batches_counter assertion * clarified error handling and updated failed_batches_counter initialization * decreased examples array length to 1 to ensure repeatability * troubleshooting tests * trying to get test_increment_failed_batches_counter to fail as expected * corrected assertion details * simplified assertRaises and added reminder comment to assertEqual counter * lint test * lint test passed, resetting pre-commit-config.yaml * fixed lingering linting issues * shortened comment line to comply with linting * formatter worked its magic commit 2341f61d48d838ddaf2e4fb990a49987244c3513 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Thu Nov 10 21:24:55 2022 +0300 [Tour Of Beam] handle CORS pre-flight requests (#24083) * cors * README * nit * nit * -headers, -cache * allow-headers * +1h cache,comment commit 36b0c8f1a76865e4dc3b1eaec1566c3ef9fd4345 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Thu Nov 10 21:21:21 2022 +0300 [Playground] update snippet by persistence_key (#24056) * proto * AIO * -frontend * testify * +license * -build * index * skipKey * Update playground/backend/internal/db/datastore/datastore_db.go Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit df553d1a4c6c6cc9b37ada2a6c84c88d8aef38b9 Author: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> Date: Thu Nov 10 13:17:36 2022 -0500 Add random string at the end of BigQuery query job name to make it resilient to retries (#24041) * add random string at the end of query job name * use deterministic temp table name commit e439f4120ef4c25aa36e5b03756dc7391bdbd211 Author: Pablo <pabloem@users.noreply.github.com> Date: Thu Nov 10 10:06:26 2022 -0800 Improving stale container cleanup script (#24040) * Improving stale container cleanup script * Avoid also latest image * Update .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh Co-authored-by: Yi Hu <huuyyi@gmail.com> * Update .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh Co-authored-by: Yi Hu <huuyyi@gmail.com> Co-authored-by: Yi Hu <huuyyi@gmail.com> commit 8d585242e158babd8dd2ca5a6d5a49d353be2935 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Thu Nov 10 12:58:41 2022 -0500 [Python]Set pickle library at the Pipeline creation stage (#24069) Fixes https://github.com/apache/beam/issues/21615 commit 08b6a524fecc333a12ce42971733ef64ae7d02f1 Author: Janek Bevendorff <janek.bevendorff@uni-weimar.de> Date: Thu Nov 10 18:37:09 2022 +0100 [BEAM-12792] Install pipline dependencies to temporary venv (#16658) commit 74f87b0282a23aa01ecb63288013311d207960aa Merge: 38a85b288a 383ea77bc9 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 10 09:10:37 2022 -0800 Merge pull request #23985 Support dynamic sharding in the worker. commit 38a85b288ad10171c444d4092eb969ca5de58af3 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Nov 10 10:39:15 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24077) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.11.1 to 1.29.1. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.11.1...service/s3/v1.29.1) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit c5110bae06fb9db6cace1f7657745fa97e3f58af Author: Yi Hu <yathu@google.com> Date: Thu Nov 10 10:33:55 2022 -0500 Fix FhirIO javadoc format broken (#24072) commit 623083cd0a72d86c7ecba7653e9a8564861abc4d Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Mon Nov 7 16:08:39 2022 +0600 [Website] change headers size from h4,h3 to h2 #24082 commit 5bd75c25de291e517cc5c5799ae4adaaaaceacb7 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 9 18:04:28 2022 -0800 Updates Multi-lang Java quickstart commit 4522f4ce916223afb3c4edd02a7be6018d0ce5a4 Author: Trevor Gevers <tlgeverssa@gmail.com> Date: Wed Nov 9 18:56:18 2022 -0600 Update datastore_wordcount.py (#23724) commit 96cc252348ce7229731be3c74bf5848d1b004c08 Author: tvalentyn <tvalentyn@users.noreply.github.com> Date: Wed Nov 9 16:08:00 2022 -0800 Remove a duplicate label (#24043) commit 62a6bd1e9317a7658810e9a0eebfe925460e8982 Author: Thiago Nunes <thiagotnunes@google.com> Date: Thu Nov 10 08:45:27 2022 +1100 test: add more tests to throughput estimator (#23915) Verifies that getting the throughput when no updates have occurred for the size of the window should return 0. commit e6647c34bb367b3058c42d0201150395c534e7ce Author: Ritesh Ghorse <riteshghorse@gmail.com> Date: Wed Nov 9 15:12:59 2022 -0500 skip output coder field in exp request (#24066) commit 383ea77bc982416afcbbbe11a4dc1f1424670589 Author: Robert Bradshaw <robertwb@gmail.com> Date: Wed Nov 9 10:49:37 2022 -0800 Reduce flakiness of time-based split manager test. Increase wait time from a tenth to half a second. commit 539fa9159ffc116b2e79e6de2804dfdd1c1e4722 Author: Danny McCormick <dannymccormick@google.com> Date: Wed Nov 9 11:45:20 2022 -0500 Convert initialisms to all caps (#24061) * Convert initialisms to all caps * Fix test refs commit 4941b9633c6ccf9eb8aad554186aac24ca7b7492 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Wed Nov 9 11:25:48 2022 -0500 added comments for tensorflow notebook (#23726) * added comments for tensorflow notebook * added note to string type commit 858f5048f578548a63e5e4319a7ab5363311c269 Merge: 95c121b254 1929968891 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Wed Nov 9 16:35:19 2022 +0100 Merge pull request #24057: [Spark Dataset runner] Enable projection pushdown for Spark dataset runner commit 95c121b2549d3f8fda7668e3759b8991b55d9865 Author: Oleh Borysevych <oleg.borisevich@akvelon.com> Date: Wed Nov 9 16:03:33 2022 +0200 Fix dependency mismatch in Playground Java runner (#24059) * fixing existing and potential dependency mismatch * extract grpc version commit 1929968891d629c83249c574c28db6c103ba9271 Author: Moritz Mack <mmack@talend.com> Date: Wed Nov 9 13:35:12 2022 +0100 [Spark Dataset runner] Enable projection pushdown for Spark dataset runner. commit 63362f5ba60a22e77e54c2df47a844f031036309 Author: Johanna Öjeling <51084516+johannaojeling@users.noreply.github.com> Date: Wed Nov 9 07:49:54 2022 +0100 [Go SDK] S3 implementation of the Beam filesystem (#23992) * Implement filesystem for S3 * Update CHANGES.md commit 73142bad2472e4516831c24a8dfb05c9acebf791 Author: Kanishk Karanawat <kkdoon@gmail.com> Date: Tue Nov 8 20:46:04 2022 -0500 Handle Avro schema generation for logical data types in BigQueryAvroUtils (#23620): handle avro logical types for TableFieldSchema to Avro schema conversion Co-authored-by: Kanishk Karanawat <kkaranawat@twitter.com> commit 8db8fae173056c0d89a36a8a5ce0443b6b9e54fa Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 8 15:56:53 2022 -0800 [Go] Add pipeline resource hints to CHANGES.md (#24036) commit a84b7dc179fed5dfc1d48dddfb04266de3716e31 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 16:35:25 2022 -0800 Make mypy happy. commit b9655e7de1a682d8ec4efcafb4d610f794e1b40e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 8 12:27:38 2022 -0800 Bump cloud.google.com/go/storage from 1.27.0 to 1.28.0 in /sdks (#24028) Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.27.0 to 1.28.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.27.0...spanner/v1.28.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/storage dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 708e05f0dc6c0a2e62674cd0ecb8a6744ae57d3a Author: Brian Hulette <bhulette@google.com> Date: Tue Nov 8 12:18:23 2022 -0800 Retroactively announce Batched DoFn support in 2.42.0 Blog (#24011) * Retroactively announce Batched DoFn support in 2.42.0 * Add to blog as well commit b9ff75a18b40141a8d2340e397fd70382b3d4e8f Author: Israel Herraiz <ihr@google.com> Date: Tue Nov 8 20:03:43 2022 +0100 Update my Twitter handle (#23653) commit bf621ccb88023e0bc09752978c099ac4629086e2 Author: Rebecca Szper <98840847+rszper@users.noreply.github.com> Date: Tue Nov 8 10:50:02 2022 -0800 Editorial review of the ML base API descriptions (#24026) commit f0a6ff46619ee1fd0639fa3f06e04708f69dee74 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 8 13:04:03 2022 -0500 Bump cloud.google.com/go/bigtable from 1.16.0 to 1.17.0 in /sdks (#24027) Bumps [cloud.google.com/go/bigtable](https://github.com/googleapis/google-cloud-go) from 1.16.0 to 1.17.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.16.0...pubsub/v1.17.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/bigtable dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 7dac3f5ef40b5d24b24d9ce5bec4717284260b85 Author: Lukasz Cwik <lcwik@google.com> Date: Tue Nov 8 09:42:29 2022 -0800 [#21250] Trivial removal of loop over something that always has one element (#24014) Multiplexing was put into the PCollectionConsumerRegistry a long time ago and this seems to have been missed during that migration. commit a6a9b23fd5b74142c35948f10d1840b882817246 Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 8 09:19:49 2022 -0800 [Go] Pipeline Resource Hints (#23990) commit 8cf2a63cc25399109f64b84ddfcd7c74d21e61ce Author: tvalentyn <tvalentyn@users.noreply.github.com> Date: Tue Nov 8 09:19:16 2022 -0800 Update release notes. (#23986) commit c600444e1dfa2f4afaad056bc65fa009a60a32bb Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 8 06:22:14 2022 +0000 Bump loader-utils Bumps [loader-utils](https://github.com/webpack/loader-utils) from 1.4.0 to 1.4.1. - [Release notes](https://github.com/webpack/loader-utils/releases) - [Changelog](https://github.com/webpack/loader-utils/blob/v1.4.1/CHANGELOG.md) - [Commits](https://github.com/webpack/loader-utils/compare/v1.4.0...v1.4.1) --- updated-dependencies: - dependency-name: loader-utils dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> commit 62d8bdc604dfeccae08d00225bce8c9b9c3ffb28 Merge: d4142f7eab 930c4d2d87 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:31:37 2022 -0800 Merge pull request #23789 Better surfacing of Scala support via Scio. commit 7f7713de09b3b0e400ed02ffdfcb34a6a8b6e1e4 Merge: 5f587b9339 d4142f7eab Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:29:27 2022 -0800 Merge branch 'master' into javascript-liquid-sharding commit 5f587b9339b3b19ae5c5b1cf15e3ce218f75d21a Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:25:33 2022 -0800 Clarifying comments. commit 60fe3cdddf9d3c27d9493b345d1f5f2a9284a76f Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 7 15:06:26 2022 -0800 Update style sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py Co-authored-by: Danny McCormick <dannymccormick@google.com> commit d4142f7eab7d5a3d21a3d3930c448fe403880f93 Author: Robert Burke <lostluck@users.noreply.github.com> Date: Mon Nov 7 12:11:21 2022 -0800 Switch && for || to fix bug in #23889 resolution (#24017) commit 45100d708d954b25b5a884dc3c7a965f008228c6 Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 14:53:24 2022 -0500 Add files then check cached diff to get untracked files commit 380d4730d3c52b5c2165bdb84aac11d0feccfe4f Author: scwhittle <scwhittle@users.noreply.github.com> Date: Mon Nov 7 19:33:37 2022 +0100 Enforce splitting invariants by ensuring split state is reset in the same synchronized block as window index increments. (#23882) * Enforce splitting invariants by ensuring split state is reset in the same synchronized block as window index increments. Fixes #23881. * Add missing currentElement = null; Co-authored-by: Lukasz Cwik <lcwik@google.com> commit 8b3fd2e0d652045bf94b83638129bb12562ce738 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 7 10:33:26 2022 -0800 Bump google.golang.org/api from 0.101.0 to 0.102.0 in /sdks (#23957) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.101.0 to 0.102.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.101.0...v0.102.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 9533fc3757d3041135cbe2f5078dff0855adc1f1 Author: capthiron <24925821+capthiron@users.noreply.github.com> Date: Mon Nov 7 19:32:18 2022 +0100 feat: implement bigtable io connector with write capabilities (#23411) commit c4218e572aa8a8f46b433b60073a59fdd7fdf148 Merge: 23676a9ec1 a5ee669a43 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 7 18:24:45 2022 +0100 Merge pull request #23934: [Website] change case studies page mobile layout commit 23676a9ec17069b309e8803d1fbfeb8009035828 Author: Oleh Borysevych <oleg.borisevich@akvelon.com> Date: Mon Nov 7 17:24:59 2022 +0200 [Tour of Beam] Learning content for "Introduction" module (#23085) * learning content for introduction module * removing white spaces from md files * delete whitespaces * delete whitespaces in python * delete whitespace #2 * divide pipeline concepts * add pipeline example concepts * adding category tag to python examples * adding category to java examples * adding category to go examples * fixed go example * fixed go example compilation * fixing python duplicate example names * add runner concepts * fixing java examples * add licence for runner unit * some minor fixes for unit names * fixed unit name * resolving CR comments * adding complexity to examples * adding tags * fixed go example compilation * fixed python example with duplicate transform * change indent python * fixing missing pipeline options * change arrow symbol * delete example prefix * minor formatting and readability fixes * add example description * minor fix * minor code review comment Co-authored-by: Abzal Tuganbay <abzal.tugan@gmail.com> commit 98d2ffd69ab9d5c49196a78f7c25daacb57b6478 Author: Brian Hulette <bhulette@google.com> Date: Mon Nov 7 07:03:12 2022 -0800 Make `documentation/io/connectors/` canonical (#23877) * Drop documentation/io/built-in/ * Update links in Custom I/O pattern commit c0b41fa045418beb782c0a59e6f5c511da898c53 Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:29:08 2022 -0500 Use git diff instead of git diff-index to avoid file timestamp changes being picked up commit 0476d6498217b7b0e833643c47fef2c4bdaec529 Merge: 1d5fc14031 6ae37b6f82 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 7 15:28:59 2022 +0100 Merge pull request #23962: [CdapIO] Add sparkreceiver:2 module. commit 1d5fc14031d583e92018ebfbeaff9b4e3b978fdb Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:14:24 2022 -0500 Remove quiet flag on debug commit c08273b9355566181d2d5c0792130f8d5cc1428c Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:12:26 2022 -0500 Correctly print diff and swallow empty commits for the moment commit 66fb431deb77db5f7f0fac31e6bf3b2c094c8ac5 Author: Danny McCormick <dannymccormick@google.com> Date: Mon Nov 7 09:08:53 2022 -0500 Print diff and scope to state path commit e0e10b9e5432643b884c381d145e5924cc4ef193 Author: Danny McCormick <dannymccormick@google.com> Date: Sun Nov 6 12:12:22 2022 -0500 PR Bot - Dont throw error on return code 1 commit bbcb790461c47d22399c681cd42c434b206eb550 Author: Danny McCormick <dannymccormick@google.com> Date: Sun Nov 6 11:59:32 2022 -0500 Fix pr bot - exec doesn't allow command chaining commit 88f9a66a9f4cc2fe3259d5ebd83ebb472a29dd5e Author: Danny McCormick <dannymccormick@google.com> Date: Sun Nov 6 11:46:34 2022 -0500 Fix diff to stop repeated bot runs commit ddb4f2d53538fea68383fe43221d4074d05d11b9 Author: bullet03 <bulat.safiullin@akvelon.com> Date: Fri Nov 4 16:11:10 2022 -0700 Website add and update logos (#23899) * [Website] add and update logos * Delete talend.svg:Zone.Identifier commit 6d42219ff547720996489aa29de0d64246db2541 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 14:30:18 2022 -0700 Enable more portable-runner requiring tests. (#23970) * Enable more portable-runner requiring tests. * Run all basic tests on portable runner as well. * Mark ULR tests. commit 1a643d16112a5f3f1418f1be393de51af65d752b Author: David Cavazos <dcavazos@google.com> Date: Fri Nov 4 13:45:12 2022 -0700 Beam starter projects blog post (#23964) Co-authored-by: Ahmet Altay <aaltay@gmail.com> commit d10b4a28fff667289c146152535275035f2dedf7 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Fri Nov 4 16:06:49 2022 -0400 removed trailing whitespace (#23987) commit 3bd697561aab4a5d2aa49d29143695d6d57a834a Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:43:47 2022 -0700 Add dynamic splitting support to the worker. commit ed520243f472c7224a1f8c74a0c2e8f81965cbfc Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:07:40 2022 -0700 Add the a Reshuffle operation and use it in Create. This allows created sources to be staticly and dynamically split. commit 9283512544b8de169be9cd7795b42284fdf45b87 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 10:28:51 2022 -0700 Add the ability to schedule splits on the ULR via a pipeline option. This can be used to test arbitrary SDKs for behavioral correctness for dynamic splitting. These splits are wall-time based, which isn't ideal, but easier than setting up and managing a cross-process synchronization channel. commit ed7b560d7ce7dda0b55a067b9d9a1a9d692a7d7a Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 09:46:19 2022 -0700 Compute element counts for all PCollections. commit 1cfdb127983a55c77b3259c4fdd248497b28e250 Author: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Date: Fri Nov 4 15:43:16 2022 -0400 Add custom inference fn suport to the sklearn model handlers (#23642) * Add custom inference fn suport to the sklearn model handlers * Clean up import order * Update typing, add numpy unit test * Add Pandas unit test * Formatting, linting * yapf run * Remove trailing whitespace * import order * Change inference_fn to keyword-only arg commit 8617b86fd792ac555233dd79c8ede34c2ce67ecf Merge: aa178f8252 2496a0d1e0 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:42:21 2022 -0700 Merge pull request #23978 Add basic counter support to the typescript SDK. commit 2496a0d1e0fda74e8fc0f24eee011362426369a2 Merge: 64abcbe4a3 aa178f8252 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 11:15:47 2022 -0700 Merge branch 'master' into javascript-metrics commit aa178f825223cbb0a6208fad0e02f98292eeded1 Merge: 97628802fe 1807c307aa Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 09:34:11 2022 -0700 Merge pull request #23976 Correctly capture log levels and attach stage information. commit 64abcbe4a3a15160aba3f3163ff8cbf5186bbfdb Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 09:27:22 2022 -0700 Only report counters that were actually used. commit 1807c307aa1e493acb2d8ba542baffefcb96e7e2 Author: Robert Bradshaw <robertwb@gmail.com> Date: Fri Nov 4 08:52:21 2022 -0700 Remove obsolete TODO. commit 97628802fe4cf6640a3b6854e27b410dd5731379 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 4 11:47:07 2022 -0400 Bump cloud.google.com/go/datastore from 1.8.0 to 1.9.0 in /sdks (#23916) Bumps [cloud.google.com/go/datastore](https://github.com/googleapis/google-cloud-go) from 1.8.0 to 1.9.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/asset/v1.8.0...asset/v1.9.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/datastore dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 058c56af735e10184d1501cc08cd16af4d21e64a Author: Ayush Sharma <114604338+ayushthe1@users.noreply.github.com> Date: Fri Nov 4 20:58:11 2022 +0530 [Task]: PR Bot will push commits only if they are non-empty (#23937) * fix pr-bot * fix the git pr bot commit 7dba78d611089a1826c5655ed7ffcdf3f6007eae Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 4 09:35:16 2022 -0400 Immediately truncate full restriction on drain of periodic impulse (#23765) * Immediately truncate full restriction on drain of periodic impulse * Whitespace format commit 16852518861514a56ad1b57113e3815cfcc8533a Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Fri Nov 4 08:39:34 2022 -0400 TFX image classification example (#23456) * TFX image classification example * TFX image classification with serving model * Add TF Model Wrapper * Clean up code * Refactoring * Add a test for tfx_bsl RunInference * fixup lint * Refactoring * Fixup lint * Add brief summary for the example * Apply suggestions from code review Co-authored-by: Andy Ye <andyye333@gmail.com> * Refactoring code and add comments * Update help description * reorder imports * Reorder imports again * Add docstring * Refactoring * Add pillow to tfx pipeline requirements * Move inferencePostCommitIT to Python 3.9 suite * Uncomment other postcommit suites Co-authored-by: Andy Ye <andyye333@gmail.com> commit 30b26171b4038c1c976adbc3be386e118b35153b Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 4 08:22:17 2022 -0400 Update watermark during periodic sequence/impulse (#23507) * Update watermark during periodic sequence/impulse * Remove extraneous import * Formatting * Linting * Only run on dataflow for guaranteed watermark support * More permissive test to avoid timing issues * Test pipeline options * Fix test * Formatting * Formatting * Apply feedback - cleanup/naming/flink * Format * Unused import commit 1563b9721b4bee5a22d384b282762726247e0703 Author: Brian Hulette <bhulette@google.com> Date: Fri Nov 4 05:09:39 2022 -0700 Improve Iterator error message (#23972) commit e3987b5435d39c5fb058d4085eb5e210f8756224 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 17:39:49 2022 -0700 old prettier change commit 89c8dd5ca23591be6c2778045df32c7d08d9f99e Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 17:12:26 2022 -0700 Add distribution metric type. commit e4e606a8f2652c2fa761a95462f230385b6bcb47 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 17:09:03 2022 -0700 Support metrics over the portability API. commit e04f3d6fdb39528264cb2a14e60e565991121f1c Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 3 16:52:51 2022 -0700 Add basic counter setting and getting to the typescript SDK. commit 1daa9c373ac7fbdb696120eeaa2ca9e85a679a1f Author: Robert Bradshaw <robertwb@gmail.…

* DebeziumIO schema transform (#24629) * prototyping a debezium-based schema transform for continuous ingestion of change streams from databases * Writing a significant integration test for DebeziumIO SchemaTransformProvider * Improvements to debuggability, performance on expansion * fix tests. mysql integration test remaining * fixing and improving mysql test * Fix JdbcIO issue * add integration tests and reduce size for now * fixup * passing tests * Address comments * add comment * jamm on -javaagent instead of classpath * [Website] update links with 404 status code * [Website] update links from absolute to relative in md files * Bump actions/setup-java from 3.6.0 to 3.8.0 (#24559) Bumps [actions/setup-java](https://github.com/actions/setup-java) from 3.6.0 to 3.8.0. - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](https://github.com/actions/setup-java/compare/v3.6.0...v3.8.0) --- updated-dependencies: - dependency-name: actions/setup-java dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix some typos (#24736) * Fix some typos * Fix write config * Adding Setup python to Dataflow job in Java Tests (#24712) * Adding Setup python to Dataflow job * Adding Setup python to Dataflow job * Using standard setup-self-hosted composite action for installations Co-authored-by: Elias Segundo <elias.segundo@luisrazo.local> Co-authored-by: Danny McCormick <dannymccormick@google.com> * Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24740) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.11.44 to 1.11.46. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.11.44...feature/s3/manager/v1.11.46) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Refactor focusing to contextLine (#24674) * Refactor focusing to contextLine (#24613) * Rename a widget (#24613) * Minor reordering (#24613) * Fix after review (#24613) * Support SqlTypes Date in AvroUtils (#24756) * Fix SingleStoreIO performance test job (#24753) * Fix a small wrong url link in notebook (#24746) * Fix POM of beam-sdks-java-core (closes #24675) * Exclude MultimapState related tests in Dataflow legacy worker validates runner tests (#24758) * [Website] delete 404 links #24745 (#24744) * [Release Tasks Migration] - Git Tag Workflow (#24418) * worflow for git tag final release * modify workflow and descriptions * modify git tag command * add git config * update CI doc * Change Point Analysis (#23931) Co-authored-by: tvalentyn <tvalentyn@users.noreply.github.com> Fixes https://github.com/apache/beam/issues/23930 * change examples to licensed ones with mentions of autors and licenses (#24762) Co-authored-by: Philippe Moussalli <philippe@moussalli@ml6.eu> * Implement PubsubWriteSchemaTransformProvider (#24443) * Implement PubsubWriteSchemaTransformProvider * Patch error message for format * Patch method comment * Fix identifiers * spotlessApply * [CdapIO] Complete examples for CDAP Hubspot plugins (#24568) * Add examples for Cdap Hubspot plugins * Fix dependencies * Move Cdap Hubspot examples to the separate gradle module * Move common classes to Examples Cdap module * Fix readme * Refactoring * Fix typo * Fix Permission denied error build beam locally (#24766) * fix typo: UnmarshallWindowFn should be UnmarshalWindowFn (#24771) * Remove slow review label after processing commands to avoid race (#24765) * [#24292] Create Avro extension for Java SDK * Address the review comments * Apply #24454 changes * [BigQueryIO] Don't update schema of destination table when no schema is provided (#24700) * skip schema update when no schema is provided * test for using create_if_needed disposition * spotless * [Playground] Python multifile examples (#24751) * proto * gen * models * with multifile * models * examples * buf.gen.yaml+gen * buf.gen.yaml * gen * files_content * backend_run_code * infra_datastore * fix * GetPrecompiledObjectCode * ignore_incomplete * build * user_score.py * rename * no java * user_score * fix Co-authored-by: Alexey Inkin <leha@inkin.ru> * [CdapIO] Complete examples for CDAP Salesforce plugins (#24567) * Add examples for Cdap Salesforce plugins * Move Cdap Salesforce examples to the separate gradle module * Move common classes to Examples Cdap module * Fix readme * Rename package-info file and rename gradle tasks * Fix checkstyle * Make go vet and go staticcheck output visible in CI (#24786) * Make go vet and go staticcheck output visible in CI * Intentionally fail staticcheck * Actually violate staticcheck * Actually violate staticcheck * Fix staticcheck errors * [Go SDK] Fix multimap support for the direct runner (#24775) Updates the `NewKeyedIterable()` implementation to correctly parse a multimap side input from an in memory ReStream. * [#24515] Ensure that Go pipelines on Dataflow are not allowed to opt out of runner v2. (#24767) * [#24515] Ensure that Go pipelines on Dataflow are not allowed to opt out of runner v2. This is towards #24515 * Drop the unused worker jar and update breaking changes. * Update Java portable container for Dataflow (#24791) * [BigQuery] Extend timestamp precision to microseconds when writing with Beam Rows (Storage API) (#24784) * [Website] update check links, catch prod & staging links * [Spark Dataset runner] Trigger evaluation using write noop rather than using foreach (closes #24797) (#24565) * Install sklearn < 1.20.0 for sklearn postcommit tests (#24788) * Install sklearn < 1.20.0 * fixup syntax * [Spark Dataset runner] Skip unconsumed additional outputs of ParDo.MultiOutput to avoid caching if not necessary (resolves #24710) (#24711) * [Go SDK]: MongoDB IO connector (#24663) * Better naming and inference for PythonCallable. (#24735) This is particularly valuable for cross-language calls. * [Spark Dataset runner] Reuse SparkSession when testing using class rule. (#24812) * [#24515] Ensure that portable Java pipelines on Dataflow are not able to opt out of runner v2. (#24805) Towards #24515 * 24802 fix missing test resources (#24803) * Remove use of test resource files * Patch method comment * Add Python xlang KafkaIO performance test (#24633) * Add Python xlang KafkaIO performance test * Use SDF read for performance test in both Java and Python * Fix kafka run out of resource upon increase number of records for streaming test * Fix assert expected and actual misplace * Fix Java test fail whenever a write bundle fail * Fix test workaound (see PR comment) * Shard python precommit (#24204) * WIP: Shard python precommit * Naming * Formatting * Allow empty test suites * Fix no posargs case * Fix paths * Split out integration tests * Do common tasks in IT suite * Fix name base * More splitting * Fix quotes * try ignore-glob * Add groovy files per subfolder * name typo * Filter dataframes tests * Logging * Switch to correct directory syntax * Merge master + fix paths * Consolidate back to reasonable set * Add jobs to README * Try full quotes * Add in missing tests * Move arg lines down * Add gradle files for typescript containers and update the version info. Container are now built with ./gradlew ":sdks:typescript:container:docker" just like all other SDKs and they will be pushed as part of the release process. * Refactor integration tests to use internal containers package (#24823) * Various cleanups to the typescript SDK: * Better errors for bad serialization imports. * Semver-correct version for fake worker package. * Package source into temporary directory. * Allow "latest" as a xlang beam jar version for non-released SDKs. * Less verbose logging. * Fix issue with typescript sibling sdks. * [#24789][Go SDK] Fix Minor race conditions (#24808) * Add timeout to cross-language tests. * npm install before build * Format go file. * [#24789] Spot fix fullvalue wrapping for SDF. (#24826) * [#24789] Spot fix fullvalue wrapping for SDF. * Fix indenting on diagrams * Reinstate comment * Add thread prefix for StreamingDataflowWorker work executor * Fix Cassandra read bug when user query has no where clause (fixes #24829) (#24830) * Fix Cassandra read bug when user query has no where clause * update changelog * fix format violation * fix remaining format violation * add unity test for read with unfiltered query * fix unfiltered query unit test count validation * use constant for num rows assertion on unfiltered query unit test Co-authored-by: Yi Hu <huuyyi@gmail.com> * fix cassandra query bugfix description Co-authored-by: Yi Hu <huuyyi@gmail.com> Co-authored-by: Lucas Marques <lucas.pmarques@b2wdigital.com> Co-authored-by: Yi Hu <huuyyi@gmail.com> * Retry create database in InfluxDbIOIT (#24800) * Retry create database in InfluxDbIOIT * Rename InfluxDb performance test to post commit * This test is actually an integration test on direct runner * Revert #23680 relax job node * Update .test-infra/jenkins/README * BEAM-13261 added max connections setting * Support SqlTypes Date in AvroUtils (sync) * Revert "Shard python precommit (#24204)" (#24855) This reverts commit 598324a8847e8bd4ea00c9526777e9e6e51229af. * [Spark Dataset runner] Fix initialization of metrics accumulator on driver (fixes #24809) (#24810) * Improve error message (#24843) * improve error message * improve error message * Playground Example CI fix for 24807 (#24819) * remove deleted files from git diff output * fixing ci example checker * remove diff filtering * Bump github.com/tetratelabs/wazero from 1.0.0-pre.4 to 1.0.0-pre.7 in /sdks (#24858) * Bump github.com/tetratelabs/wazero in /sdks Bumps [github.com/tetratelabs/wazero](https://github.com/tetratelabs/wazero) from 1.0.0-pre.4 to 1.0.0-pre.7. - [Release notes](https://github.com/tetratelabs/wazero/releases) - [Commits](https://github.com/tetratelabs/wazero/compare/v1.0.0-pre.4...v1.0.0-pre.7) --- updated-dependencies: - dependency-name: github.com/tetratelabs/wazero dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Remove ctx param from wazero fns Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jack McCluskey <thejackmccluskey@gmail.com> * Fix beam_CleanUpGCPResources job (#24770) * Fail the job at the end * Update README.md * Set DELETE_BEFORE_DAY in accordance with release period (#24867) * adding parameters flag to support schema based transforms (#24864) * Fix SQLIO Performance Test (#24824) * Fix SQLIO Performance Test * Update jenkins README * Reduce redundant Get API call when creating bq temp dataset (#24831) * Reduce redundant Get API call when creating bq temp dataset * Move logging to get_or_create_dataset and set appropriate level * Disable sonarqube cron job (#24769) * Saving empty output and log precompiled objects for consistency (#24877) * save empty ppc objects * adding missing argument * Playground runners scala update (#24873) * adding maven repo for sbt * externalize grpc timeout and document variables * fix rat check * fixing typo * minor change to re-trigger whitespace check * typo fixed * [#24801] ValueState.read() should specify returning null since it will if unset (#24869) * [#24801] ValueState.read() should allow returning null Also clarify documentation. Fixes #24801 * Address spotbugs and @Nullable on ValueState in sdks/java/io/google-cloud-platform * Add links to colab/github repo in notebook (#24886) * Add links to colab/github repo in notebook * Align links * Fix 2 missed notebooks * Remove trailing space breaking website blocks (#24875) * Remove trailing space breaking website blocks * Convert remaining references * Bump google.golang.org/api from 0.105.0 to 0.106.0 in /sdks (#24891) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.105.0 to 0.106.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.105.0...v0.106.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [#21391]Increase unit testing coverage in the exec package (#24772) * add more test in sdks/go/pkg/beam/core/runtime/exec/translate_test.go * add test for translate.go unmarshalPort() * add test on function TestNewUserStateAdapter * add test for translate.go newBuilder(desc *fnpb.ProcessBundleDescriptor) * add test for translate.go UnmarshalPlan(desc *fnpb.ProcessBundleDescriptor) * better naming and error message * Update sdks/go/pkg/beam/core/runtime/exec/translate_test.go Co-authored-by: Ritesh Ghorse <riteshghorse@gmail.com> * use errors.New() when format string is not used Co-authored-by: v-zhenglinli <v-zhenglinli@microsoft.com> Co-authored-by: Ritesh Ghorse <riteshghorse@gmail.com> * Load categories from BEAM_EXAMPLE_CATEGORIES in checker.py (#24900) * Reduce lock contention for metrics hashName hasher (#24881) * Add metrics parallel bench Utilize parallel benchmark for metrics to measure scalability on multiple cores. * Reduce locking for hashName Use a pool of hashers rather than a singleton. For workloads that create Counters on the fly this results in a 4x or so overall speedup in adding incrementing Counters. * Bump golang.org/x/text from 0.5.0 to 0.6.0 in /sdks (#24889) Bumps [golang.org/x/text](https://github.com/golang/text) from 0.5.0 to 0.6.0. - [Release notes](https://github.com/golang/text/releases) - [Commits](https://github.com/golang/text/compare/v0.5.0...v0.6.0) --- updated-dependencies: - dependency-name: golang.org/x/text dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump golang.org/x/sys from 0.3.0 to 0.4.0 in /sdks (#24890) Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.3.0 to 0.4.0. - [Release notes](https://github.com/golang/sys/releases) - [Commits](https://github.com/golang/sys/compare/v0.3.0...v0.4.0) --- updated-dependencies: - dependency-name: golang.org/x/sys dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go from 1.30.19 to 1.33.0 in /sdks (#24896) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.19 to 1.33.0. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.33.0/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.30.19...v1.33.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump golang.org/x/net from 0.4.0 to 0.5.0 in /sdks (#24892) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.5.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/compare/v0.4.0...v0.5.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Print logical identifier in FieldType.toString (#24888) * Fix PR number in 2.43.0 changelog (#24904) * Update CHANGES.md * Update beam-2.43.0.md * Fix poorly formatted metadata (#24905) This was caused by a bad edit in https://github.com/apache/beam/commit/12f107249b7617d84c0552df4977de4f3801e557# * Add static override for JDK TLS disabled/legacy algorithms in Java container (#24619) * Adding jdk.tls security property options to java containers * Update CHANGES.md remove whitespace line endings * Fix invalid copy task configuration * Adding license to TLS properties files * Added bugfix description and link to CHANGES.md for #24623 * Renaming to match global implications for security properties override as TLS configs are not the only properties that can be changed * Add license * Add license * Adding TLS-enabled check to SdkHarnessEnvironment tests * Update sdks/java/container/java11/option-java11-security.json Co-authored-by: Kiley Sok <kileysok@gmail.com> * Adjusting version fallback specification for Java * Making suggested fixes * Fixing indentation * Adding SSLContext check to TLS availability test * Making suggested improvements to test * Removing exception imports no longer needed * Remove whitespace and erroneous context null initialization * Fix typo * Update sdks/java/core/src/test/java/org/apache/beam/sdk/SdkHarnessEnvironmentTest.java Co-authored-by: Lukasz Cwik <lcwik@google.com> * Fix spotless java precommit formatting error Co-authored-by: Kiley Sok <kileysok@gmail.com> Co-authored-by: Lukasz Cwik <lcwik@google.com> * Remove shuffle mode=appliance in tests (#24879) * Modify windmill DirectStreamObserver to call isReady only every 10 messages by default. This provides more output buffering which ensures that output is not throttled on synchronization when message sizes exceed 32KB grpc isready limit. * Sync script for grafana dashboard - github actions postcommit workflows (#24073) * Create job-nexmark-dataflow.yml * Update test-properties.json * Update job-nexmark-dataflow.yml * Update job-nexmark-dataflow.yml * Update test-properties.json * Update job-nexmark-dataflow.yml * Update test-properties.json * Update job-nexmark-dataflow.yml * Create job-nexmark-flink.yml * Create job-nexmark-spark.yml * Update job-nexmark-spark.yml * Update test-properties.json * Update job-nexmark-flink.yml * Update job-nexmark-spark.yml * Update job-nexmark-spark.yml * Create job-python-examples.yml * Update job-python-examples.yml * Create script to sync GA workflows with Grafana Dashboard * Sync using a cloud function * Tested with cloud function * Sync files * Delete new line * add comments * Add comments * Create Workflow Object, fix typos, modify database table, added exceptions, added multiple pages support, , fix style * Fix filename in database * Add retries, create a retrie function, delete comment, fix comment typo * Create table string now is using a for * Add comment related with the executions number * Update the number of workflows runs to 30 * Update number of executions to 100 * Integration test to load the default example of the default SDK and change the example (#24731) * Integration test to load the default example of the default SDK and change the example (#24730) (#24729) * Fix formatting and README (#24730) * Support collection v1.17.0 (#24730) * LoadingIndicator on chaning examples, remove duplicating licenses (#24730) * Add a missing license header (#24730) * Integration test for changing SDK and running code (#24779) (#382) * Integration test for changing SDK and running code (#24779) * Rename an integration test (#24779) * Use enum to switch SDK in integration test (#24779) * Find SDK in a dropdown by key (#24779) * Add a TODO (#24779) * Fix exports (#24779) * Issue24779 integration changing sdk from 24370 (#387) * Integration test for changing SDK and running code (#24779) * Rename an integration test (#24779) * Use enum to switch SDK in integration test (#24779) * Find SDK in a dropdown by key (#24779) * Add a TODO (#24779) * Fix exports (#24779) * Integration tests miscellaneous UI (#383) * miscellaneous ui integration tests * reverted pubspec.lock * gradle tasks ordered alhpabetically * integration tests refactoring * clean code * integration tests miscellaneous ui fix pr * rename method * added layout adaptivity * A minor cleanup (#24779) Co-authored-by: Dmitry Repin <mr.malarg@gmail.com> Co-authored-by: Dmitry Repin <mr.malarg@gmail.com> * issue23923 emulated data indicator (#24708) * issue23939 emulated data indicator * issue23923 removed assets constants, other small fixes * minor changes * removed redundant args * removed more redundant datasets * reverted pubspec * reverted lock * Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24917) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.29.6 to 1.30.0. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.29.6...service/s3/v1.30.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * issue23918 extract java symbols (#24588) * issue23918 extract java symbols * issue23918 java symbols mapper uses jackson for writing a file * issue23918 java symbols mapper uses jackson for writing a file * issue23918 jackson replaces on yamlbeans * issue23918 add java symbols to playground * rat * rat * rat * minor fix * newlines added * issue23918 extract symbols for scala + minor fixes * issue23918 add build task when build java * issue 23918 java.g.yaml output alphabetically * empty line * issue23918 refactoring * issue23918 fix comparator * issue23918 tests for java sdk * issue23918 deleted settings.gradle * fix * issue 23918 fix pr * new lines at the end * load symbols optimization * issue23918 reverted mocks * fix tests * issue_23918 removed wget and added classes compilation to test * issue23918 support java 8 Co-authored-by: alexeyinkin <leha@inkin.ru> * Attempt deserialize all non-standard portable logical types from proto (#24910) * Attempt deserialize all non-standard logical types from proto * Fixes portable and not-yet-standard logical type get deserialized to UnknownLogicalType * TFT Criteo benchmarks (#24382) * Add criteo Cloud ML Beam pipeline * Test for Criteo small dataset * Add tft criteo code * Add pytest marker and requirements file * Update GCS path for test data * Add shuffle param * add gradle task * Fix uuid * Fixup flag * Add requirements file * Update tests * Revert "add gradle task" This reverts commit 63236ca410d0d2ae075d17af24b3f7d963aa1a86. * Add gradle task for TFT tests * Fix up lint * Add skip to unittest * Exclude tests from pydoc generation * Update timeout * Addressing comments * Add phrase trigger * Fixup job * Fix up groovy file * Update .test-infra/jenkins/job_CloudMLBenchmarkTests_Python.groovy * Fixup groovy file * update Java version from 11 to 17 (#24839) * [#21384] Add unit tests to the sql package (#24811) * [#24927] Ensure the SDK harness handles JCL/log4j/log4j2 messages. (#24928) * [#24927] Ensure the SDK harness handles JCL/log4j/log4j2 messages. This routes JCL/log4j/log4j2 messages to SLF4 which is then routed to JUL and finally over the Fn Logging API to the runner. Fixes #24927 * Fix up overrides * Improve Error Prone configuration Pass the required `--add-exports=` flags when running on JDK 17, even if `java17Home` is unset. * Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24915) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.11.46 to 1.11.47. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.11.46...feature/s3/manager/v1.11.47) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update comment * update Beam GCP BOM (#24929) * update BOM * remove hard-coded pubsub lite version * [#24931][Go SDK] Make element checkpoints independant (#24932) * Update Kafka watermark based on policy when records poll empty (#24205) * Update watermark based on policy when records poll empty, to ensure watermark can advance even on empty partitions * add integration test to validate watermark advancement * filter out non-sdf kafka versions for testint * Clean up kafka build, rework kafka test case * apply spotles * shrink wait duration * update wait duration * Grafana dashboard - Github actions postcommit workflows mapping (#24176) * grafana dashboard postcommit GA and workflow to sync jobs * grafana dashboard postcommit to map GA workflows * update GA state in dashboard * apply changes into grafana dashboard required in script * Shard Python PreCommit (#24866) * WIP: Support passing in special pytest args * Shard precommit * Syntax + easier testing * Fix syntax * Skip -> ignore * Fix loop * Fix loop * Remove local testing mechanism and setup to interact with Jenkins * Strip off any quotes * Strip off any quotes * Strip off any quotes * Strip off any quotes * Strip off any quotes * Simplify arg parsing * Improve comment * Fail on no tests run * reapply single executor to storage write API writes (#24950) * [#21368] Clean-up and use the FixedExecutorProvider (#24952) This is a minor clean-up for https://github.com/apache/beam/pull/24950 to re-use existing implementation from gax-java * Add doLast on Playground integration tests (#24954) (#24955) * JDBC SchemaTransform implementation. Need to break out into a separat… (#24918) * JDBC SchemaTransform implementation. Need to break out into a separate PR to submit. * Update configuration to accommodate SDKs that might send empty strings or zeros in place of nulls. * Move write options validation test to correct class. Update classname and driver check to use Strings.isNullOrEmpty * Add a little documentation blurb to the providers * Fix periods that Github complains about but locally doesn't throw an error. * fix pinning bug in storage-api writes * [WebSite] Add new Go quickstart (#24885) Co-authored-by: Veronica Wasson <VeronicaWasson@users.noreply.github.com> * Add missing dependencies to maven archetype * Bump google.golang.org/grpc from 1.51.0 to 1.52.0 in /sdks (#24969) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.51.0 to 1.52.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.51.0...v1.52.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/containerd/containerd from 1.6.8 to 1.6.12 in /sdks (#24945) Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.6.8 to 1.6.12. - [Release notes](https://github.com/containerd/containerd/releases) - [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md) - [Commits](https://github.com/containerd/containerd/compare/v1.6.8...v1.6.12) --- updated-dependencies: - dependency-name: github.com/containerd/containerd dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [Playground] Add an option for Datastore namespace value for ci_cd.py script (#24818) * Add option to pass an arbitrary Datastore namespace value into ci_cd.py script * Pass Google Cloud Datastore project value into ci_cd.py script via an argument instead of using an environment variable directly * Update README.md to reflect changes to ci_cd.py script arguments * Multifile examples on frontend (#24859) (#24865) * Multifile examples on frontend (#24859) * Make a wrapper local (#24859) * Revert sdk.g.dart deletion, minor fixes (#24859) * Fix after review (#24859) * Return only Playground examples in GetCatalog() (#24816) * [Website] add credit karma case-study card (#24970) * [Website] add credit karma case-study card * Case study updates * Case study updates * add link Co-authored-by: Alex Kosolapov <alex.kosolapov@gmail.com> * [Playground] Fix for failing CI/CD in playground (#24984) * Fix test running for CI/CD scripts * Require --datastore-project in ci_Cd.py only for CD step * Improve error message for unencodeable types (#24844) * Add single job to run python coverage (#24988) * Add single job to run python coverage * Fix nameBase * Bump cloud.google.com/go/bigquery from 1.44.0 to 1.45.0 in /sdks (#24913) Bumps [cloud.google.com/go/bigquery](https://github.com/googleapis/google-cloud-go) from 1.44.0 to 1.45.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/bigquery/v1.44.0...bigquery/v1.45.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/bigquery dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update beam-master version * Bump google.golang.org/api from 0.106.0 to 0.107.0 in /sdks (#24989) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.106.0 to 0.107.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.106.0...v0.107.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [ToB] [Frontend] Authentication, unit progress, complete unit, SDK selection (#24457) * Content tree navigation (#23593) Unit content navigation (#23593) Update URL on node click (#23593) Active unit color (#23593) removeListener in unit (#23593) First unit is opened on group title click (#23593) WIP by Alexey Inkin (#23593) selectedUnitColor (#23593) Unit borderRadius (#23593) RegExp todo (#23593) added referenced collection package to remove warning (#23593) small refinement (#23593) expand on group tap, padding, openNode (#23593) group expansion bug fix (#23593) selected & unselected progress indicators (#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (#23593) * fixed _getNodeAncestors (#23593) * get sdkId (#23593) * addressing comments (#23593) * sdkId getter & StatelessExpansionTile (#23593) * expand & collapse group (#23593) * StatelessExpansionTile (#23593) * license (#23593) * ValueChanged and ValueKey in StatelessExpansionTile (#23593) * optional SDK selector in tour scaffold * AppNotifier * moved AppNotifier into _initializeState * StorageKeys * remove listener * auth, complete unit, user progress AuthNotifier draft (#23692) Comments (#23692) Comments (#23692)(1) sign in with google works (#23692) new configs (#23692) get user progress draft (#23692) comment fixes (#23692) sign in in IntroTextBody (#23692) reverted config (#23692) comment fixes (#23692) WIP before rebase (merge) (#23692) Squashed commit of the following: commit bff4919ff00ec3b5d7186efde41c884dfc4c8344 Merge: 79ba69483a ce8d618c77 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Thu Nov 17 10:34:02 2022 +0100 Merge pull request #24186: Uses _all to follow alias/datastreams when estimating index size commit 79ba69483a84ea0278d0b0ddb141200739607c77 Merge: 245fea9040 b7e860a762 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 20:47:40 2022 -0800 Merge pull request #24218: Update Python wheel format for RC validation commit 245fea904014cd58d4148807463dbaa40000774c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Nov 16 18:12:33 2022 -0800 Bump loader-utils from 1.4.1 to 1.4.2 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (#24191) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit e1de8e78deeb5d17617fda6591429eaaf8abb8a2 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 20:48:06 2022 -0500 Fix PythonLint (#24219) commit b7e860a7621771c300dcec625655f87e62591323 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:28:31 2022 -0800 updates commit c2feb09ea49dd815b69c65e531ce34128756d988 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:06:08 2022 -0800 updates commit ce8d618c77d23e20a1ddb128bb8183048597d096 Author: egalpin <egalpin@users.noreply.github.com> Date: Wed Nov 16 16:43:57 2022 -0800 Adds test for following aliases when estimating index size commit 959719d01c627328c0ca2849d2b7e2c9b322d4d1 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 15:16:06 2022 -0800 Temporary update Python RC validation job commit b952b41788acc20edbe5b75b2196f30dbf8fdeb0 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 14:18:12 2022 -0500 Python TextIO Performance Test (#23951) * Python TextIO Performance Test * Add filebasedio_perf_test module for unified test framework for Python file-based IOs * Fix MetricsReader publishes metrics duplicately if more than one load test declared. This is because MetricsReader.publishers was static class variable * Fix pylint * Distribute Python performance tests random time at a day instead of all at 3PM * Add information about length conversion commit 017f2cbde124af40a43be99ec88289fcf63c1c95 Merge: fef8acdbc0 88dba4f494 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 10:39:52 2022 -0800 Merge pull request #24187: Add a reference to Java RunInference example commit fef8acdbc0ecbcc85b49144adaf8830e3bc6b2de Merge: 6e9187e67e ead245539d Author: Ahmet Altay <aaltay@gmail.com> Date: Wed Nov 16 10:24:53 2022 -0800 Merge pull request #24199 from Laksh47/issue#24196 refs: issue-24196, fix broken hyperlink commit 6e9187e67e1bd8f73997f437f0ed4c29880ed73b Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Wed Nov 16 22:33:50 2022 +0600 [Tour of Beam] [Frontend] Content tree URLs (#23776) * Content tree navigation (#23593) Unit content navigation (#23593) Update URL on node click (#23593) Active unit color (#23593) removeListener in unit (#23593) First unit is opened on group title click (#23593) WIP by Alexey Inkin (#23593) selectedUnitColor (#23593) Unit borderRadius (#23593) RegExp todo (#23593) added referenced collection package to remove warning (#23593) small refinement (#23593) expand on group tap, padding, openNode (#23593) group expansion bug fix (#23593) selected & unselected progress indicators (#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (#23593) * fixed _getNodeAncestors (#23593) * get sdkId (#23593) * addressing comments (#23593) * sdkId getter & StatelessExpansionTile (#23593) * expand & collapse group (#23593) * StatelessExpansionTile (#23593) * license (#23593) * ValueChanged and ValueKey in StatelessExpansionTile (#23593) Co-authored-by: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Co-authored-by: Alexey Inkin <alexey.inkin@akvelon.com> commit b33fac2aa533d77cfa47f88466c8cd6bd3f3e864 Author: Bruno Volpato <bvolpato@google.com> Date: Wed Nov 16 10:51:11 2022 -0500 Use only ValueProviders in SpannerConfig (#24156) commit 5f013ab6567ec75b460b2081d7f89d332320caff Author: Robert Burke <lostluck@users.noreply.github.com> Date: Wed Nov 16 07:23:10 2022 -0800 revert upgrade to go 1.19 for action unit tests (#24189) commit 9337f4dbecc929886f8559949a082a649fd9d1bb Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 10:18:42 2022 -0500 Fix Python PostCommit Example CustomPTransformIT on portable (#24159) * Fix Python PostCommit Examples on portable * Fix custom_ptransform pipeline options gets modified * Specify flinkConfDir commit ead245539d01dec0f3e08699c1e1cc6777a5ef0e Author: Laksh <lakshmanansathya@gmail.com> Date: Wed Nov 16 09:32:46 2022 -0500 refs: issue-24196, fix broken hyperlink commit e83a996d4374d467d95bcfad7166905622ec615c Merge: 2fc56ec663 ffdee0b6ed Author: Jan Lukavský <je.ik@seznam.cz> Date: Wed Nov 16 15:15:31 2022 +0100 Merge pull request #24192: Re-use serializable pipeline options when already available. commit ffdee0b6edb8638c78a65ec85c727ea5dde1cb2f Author: Jozef Vilcek <jvilcek@zetaglobal.com> Date: Mon Nov 14 16:48:18 2022 +0100 Re-use serializable pipeline options when already available (#24192) commit 88dba4f494829b2b3530b767fb8c5252e0d2ba44 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Tue Nov 15 16:21:22 2022 -0800 Add a reference to Java RunInference example commit 2fc56ec663e335cfcf37dc57d471f79b601414f4 Merge: f763186987 83f1bc19b9 Author: Kenn Knowles <kenn@apache.org> Date: Tue Nov 15 16:16:47 2022 -0800 Merge pull request #24142: Fix arguments to checkState in BatchViewOverrides commit f763186987c00ba1d26efdc35406436a1fa69a9a Merge: c2bc2135e9 0d7ca04182 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 15:25:20 2022 -0800 Addresses #24161 Updated README of Interactive Beam commit c2bc2135e9bce715990a5d5551e2bc2dc0311da4 Author: Doug Judd <nuggetwheat@gmail.com> Date: Tue Nov 15 14:48:26 2022 -0800 Strip FGAC database role from changestreams metadata requests (#24177) Co-authored-by: Doug Judd <nuggetwheat@google.com> commit af637974f96ad1b5110d7dea3f9a26c68e19a51b Author: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Date: Tue Nov 15 17:16:43 2022 -0500 Add custom inference function support to the PyTorch model handler (#24062) * Initial type def and function signature * [Draft] Add custom inference fn support to Pytorch Model Handler * Formatting * Split out default * Remove Keyed version for testing * Move device optimization * Make default available for import, add to test classes * Remove incorrect default from keyed test * Keyed impl * Fix device arg * custom inference test * formatting * Add helpers to define custom inference functions using model methods * Trailing whitespace * Unit tests * Fix incorrect getattr syntax * Type typo * Fix docstring * Fix keyed helper, add basic generate route * Modify generate() to be different than forward() * formatting * Remove extra generate() def commit a014637106970a0a0e9eb7944aa5caf79fa5fd37 Author: egalpin <egalpin@users.noreply.github.com> Date: Tue Nov 15 13:57:54 2022 -0800 Uses _all to follow alias/datastreams when estimating index size Fixes #24117 commit 0d7ca041823bc2b09f76f86fdfd1d0b9508c9c88 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:57:27 2022 -0800 Minor update commit e8fc759d756f4a987e41d2b9da56b906a6cd7736 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:52:18 2022 -0800 Updated README of Interactive Beam Removed deprecated cache_dir runner param in favor of the cache_root global option. commit 08d5f72e5f35d41f3e9fa9fe799caea6bed1b7a7 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Tue Nov 15 16:34:21 2022 -0500 [Python]Support pipe operator as Union (PEP -604) (#24106) Fixes https://github.com/apache/beam/issues/21972 commit 526e7a58b62682582c27173ab21ed8667ddab766 Author: Scott Strong <scott.strong87@gmail.com> Date: Tue Nov 15 16:26:45 2022 -0500 Using Teardown context instead of deprecated finalize (#24180) * Using Teardown context instead of deprecated finalize * making function public Co-authored-by: Scott Strong <scott.strong@wunderkind.co> commit fb4d1d4dea7b26ed538a9f6aca0ed41e8c300e37 Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 16:25:22 2022 -0500 Fix broken json for notebook (#24183) commit f98db2008a97f4546d036ddf0dddfee8c87eb58a Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 15 12:49:23 2022 -0800 Update automation to use Go 1.19 (#24175) Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com> commit e5f58504eef1fdeebe0402cda8a2df259169c704 Author: Brian Hulette <bhulette@google.com> Date: Tue Nov 15 12:25:13 2022 -0800 Add error reporting for BatchConverter match failure (#24022) * add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else commit 3037747f66f0d71d65b6c65745b4f8942c22f05a Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 14:13:04 2022 -0500 Fix broken notebook (#24179) commit b2b1c739ce37690923891934ee317f799db937a2 Author: MakarkinSAkvelon <67736809+MakarkinSAkvelon@users.noreply.github.com> Date: Tue Nov 15 21:53:06 2022 +0500 [Playground] Move Playground in GKE and Infrastructure change (#23928) * changes to updated master branch * Change workflow * ingress changes * Certificate was added * Updates for cloud build backend * Update main.tf * Create main.tf * Create variables.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Create output.tf * Update output.tf * Update output.tf * Update main.tf * Update build.gradle.kts * Update output.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update output.tf * Update main.tf * Update main.tf * Update output.tf * Create variables.tf * Update main.tf * Update main.tf * Delete playground/terraform/infrastructure/cluddns directory * Update main.tf * Update output.tf * Update output.tf * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update README.md * Update README.md * helm folder name was changed * Update README.md * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Updates to readme * Fix DNS name * HelmChart was changed * Some workflows were changed * Remove unused file * playground-examples return * add license information * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * remove "stg" folder * Update README.md * Remove unused files * DNS Removed * var name changed * remove DNSName from var file * 1 * Clear terraform * remove unused records * gradle check * grade last change * issue fix * fix * 1 * run * test * Index creation for Gradle * Add IndexCreation in gradle * Update README.md * Update README.md * Fix names for Frontend * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Redis fix * services fix * Update variables.tf * change order in gradle * Fix Config.g.dart file issue * Update README.md * Playground workflow update Co-authored-by: Sergey Makarkin <sergey.makarkin@akvelon.com> Co-authored-by: Sergey Makarkin <sergey_makarkin@quicktest1.c.apache-beam-testing.internal> Co-authored-by: ruslan-ikhsan <ruslan.ikhsanov@akvelon.com> Co-authored-by: Alex Kosolapov <alex.kosolapov@gmail.com> commit 85df5f2eb2f299e28b36be0cce7b9c19d62124da Author: Yi Hu <yathu@google.com> Date: Tue Nov 15 11:38:13 2022 -0500 Eliminate CalciteUtil.CharType logical type (#24013) * Eliminate CalciteUtils.CharType logical type * Replace CalciteUtils.CharType to String Note that CalciteUtils still omits the precision of BINARY/VARBINARY/CHAR/VARCHAR as what it originally did. Support of the precision of these calcite types involves make use of making use of the overload method RelDataTypeFactory.createSqlType(var1, var2). * Replace every reference of CalciteUtil.CharType to generic PassThroughLogicalType check * Add TODO to Support sql types with arguments * Use VariableString in LogicalTypeTestCase commit f349f41010c5b238ff6020f7de718f938eef3c5e Author: alexeyinkin <alexey.inkin@akvelon.com> Date: Tue Nov 15 20:04:01 2022 +0400 Configure flutter_code_editor options with Hugo shortcode (#23926) (#24031) * Configure flutter_code_editor options with Hugo shortcode (#23926) * Minor fixes (#23926) * Refactor after review (#23926) commit 0f4ca6363b3ce0e5de3ad36517bb406aa6391a18 Author: Rebecca Szper <98840847+rszper@users.noreply.github.com> Date: Tue Nov 15 06:10:13 2022 -0800 Editorial review of the ML notebooks. (#24125) * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Update examples/notebooks/beam-ml/custom_remote_inference.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback * Update examples/notebooks/beam-ml/run_inference_sklearn.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 5bd34ede026253326ebff1a7e4f9edb5f71b4a2c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 15 07:17:28 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24131) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.3.2 to 1.11.39. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.3.2...feature/s3/manager/v1.11.39) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2ee809fa0ca7689dd0279e186ebc02d9569a8429 Merge: e3b9bdb2e6 563c66d6fd Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Tue Nov 15 11:01:14 2022 +0100 Merge pull request #23065: [Website] Update copy icon styles commit e3b9bdb2e607d85a4017ba7839000e92a0ad83c4 Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:40:50 2022 +0100 [Dockerized Jenkins] Fix build of dockerized jenkins (fixes #24053) (#24054) commit faaac2ab6e010374cb2be0e95a5dd345836a2a2c Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:38:59 2022 +0100 [Dockerized Jenkins] Update README how to use local repo (#24055) commit 689e70b5131620540faf52e2f1e2dca7a36f269d Author: Damon <damondouglas@users.noreply.github.com> Date: Mon Nov 14 17:34:29 2022 -0800 Implement embedded WebAssembly example (#24081) commit e1bf6c42950e8013f35e35fb9fee8017e01e5010 Merge: eddac84126 10337d2868 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 15:22:14 2022 -0800 Merge pull request #24160 Rename the test_splits flag to direct_test_splits. commit eddac841261228a2c63fa9b225c520ae0f853806 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 15:05:05 2022 -0800 More dataset templates to clean up (#24162) commit 2adb68bd12743566cc89b596bf204d7c807eb62d Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 13:28:13 2022 -0800 Adding a quickstart to README for the TS SDK (#23509) * More of a quickstart for the TS SDK * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 10337d28685ad5712e2ad8608977ec5c5e0e6b6b Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 12:46:32 2022 -0800 Rename the test_splits flag to direct_test_splits. This avoids possible flag conflicts. commit 48c70cc30742b45b17a1d18ece2f0d079bee3915 Author: arne-alex <108519096+arne-alex@users.noreply.github.com> Date: Mon Nov 14 21:33:02 2022 +0100 Merge pull request #23333: Track time on Cloud Dataflow streaming data reads and export via heartbeats commit 9c83de646ab52bd0b05e3346190dd55cd68b2a8b Author: Johanna Öjeling <51084516+johannaojeling@users.noreply.github.com> Date: Mon Nov 14 21:19:44 2022 +0100 Add more tests for S3 filesystem (#24138) commit 9e9c6d797ba52b460f83131431c8e53aebbbc9ac Merge: d5d76b9745 c600444e1d Author: Ning Kang <ningkang0957@gmail.com> Date: Mon Nov 14 12:06:15 2022 -0800 Merge pull request #24029 from apache/dependabot/npm_and_yarn/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/loader-utils-1.4.1 Bump loader-utils from 1.4.0 to 1.4.1 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel commit d5d76b974592d45de368ab641647ca5cc4ec12ec Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 15:03:28 2022 -0500 Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils (#23969) * Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils * Add TODO about java.time migration commit 330cc2010c9f4a2d4e30318bf50a4109ec1cd392 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 12:02:10 2022 -0800 Cleanup stale BQ datasets (#24158) * Cleanup stale BQ datasets * addressing comments commit 4a044999b8ed4bcd41f816f3a23ccb5da00c4c38 Merge: e563b9dd2f 5bd75c25de Author: Heejong Lee <heejong@gmail.com> Date: Mon Nov 14 11:16:00 2022 -0800 Merge pull request #24076 from chamikaramj/multilang_java_updates Updates Multi-lang Java quickstart commit e563b9dd2f3aa0484e6cdc08869991b5e438023e Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Mon Nov 14 20:56:35 2022 +0300 [Tour Of Beam] verify that unit exists when saving progress (#24118) * AIO * Update learning/tour-of-beam/backend/integration_tests/auth_test.go Co-authored-by: Danny McCormick <dannymccormick@google.com> * nit Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 774923e0dd089de870bfa5c77063ae2b28f79347 Merge: 71785de528 1ad0cbc445 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 09:52:26 2022 -0800 Merge pull request #24141: Fix checkArgument format in GcsPath commit 71785de52864313c2e3b14fe72a2a63281343617 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 11:54:37 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.17.10 to 1.18.0 in /sdks (#24151) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.17.10 to 1.18.0. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.17.10...config/v1.18.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 50d591d6cb3e799bee4e29dfc593c693a86e6276 Author: Bruno Volpato <bvolpato@google.com> Date: Mon Nov 14 11:50:01 2022 -0500 Change DataflowBatchWorkerHarness doWork error level to INFO (#24135) commit 5a72696bfda09fdb905ba8e58b636f8494ef955f Merge: ee0a5836d6 0633fe9634 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:12:12 2022 -0800 Merge pull request #24149: Remove extraneous jetbrains annotation commit ee0a5836d69b776834eb3bd9b2bd02eb5252c333 Merge: d001a69e1a 137799672e Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:11:00 2022 -0800 Merge pull request #24132: Fix checkArgument format string in AvroIO commit d001a69e1a58701d6ed4fcb5e3fb7a0921301dad Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 10:56:54 2022 -0500 Test Dataproc 2.1 with Flink load tests (#24129) * Test Dataproc 2.1 with Flink load tests * Minor fix flink_cluster script commit caabd9be52887ad70c8a4269395c893811ac6a84 Author: Israel Herraiz <ihr@google.com> Date: Mon Nov 14 16:03:39 2022 +0100 Make MonotonicWatermarkEstimator work like its Java SDK equivalent (#24146) * Make MonotonicWatermarkEstimator work like its Java SDK equivalent The current implementation of MonotonicWatermarkEstimator raises an exception with late messages, which makes the watermark estimator barely usable in real world scenarios. This PR fixes #20041 by making this watermark estimator work like its Java SDK equivalent (`WatermarkEstimators.MonotonicallyIncreasing`). * Update unit tests too * Make linter happy commit 451f6b3e7f58d0a3782ad942c6a1fd9f63932024 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 09:48:23 2022 -0500 Bump golang.org/x/net from 0.1.0 to 0.2.0 in /sdks (#24153) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.1.0 to 0.2.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/compare/v0.1.0...v0.2.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2bb03d62e2d7dc2d8e39040fc9adebccbde74fde Merge: 4e39ef2041 623083cd0a Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 15:01:13 2022 +0100 Merge pull request #24000: [Website] Change headers size from h4,h3 to h2 commit 563c66d6fd32165da14a07747f2764c17a5d24ea Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Wed Sep 7 18:28:42 2022 +0600 [Website] update pre tag copy link styles #23064 commit 4e39ef20410ee51c6040317bcd60171e64c5171f Merge: 223768f782 105ed6fedc Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 10:55:33 2022 +0100 Merge pull request #24115: [Website] update go-dependencies.md java-dependencies.md links commit 223768f782f771f0033b8d0686d86cf4c71fad75 Merge: aa0a35dabf a9da2abee6 Author: Kenn Knowles <kenn@apache.org> Date: Sun Nov 13 18:53:13 2022 -0800 Merge pull request #24136: Fix checkArgument format string in ExecutionStateTracker commit 0633fe9634fe61df7cbc0ecac205d81124fd504a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 15:15:16 2022 -0800 Remove extraneous jetbrains annotation commit 83f1bc19b95935e60ca1f4027d4b60c7e738a84a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 14:16:09 2022 -0800 Fix arguments to checkState in BatchViewOverrides commit 1ad0cbc44594d8405bf4b07a126265238013a02a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 13:41:02 2022 -0800 Fix checkArgument format in GcsPath commit aa0a35dabf9c2a0d9822faff06d939d9a77a3ab6 Author: Kenn Knowles <kenn@apache.org> Date: Fri Nov 11 20:26:30 2022 -0800 Fix checkArgument format string in TestStream (#24134) commit a9da2abee6455bc2cf0f18ba5f6cd7bbaeae669f Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 16:54:27 2022 -0800 Fix checkArgument format string in ExecutionStateTracker commit 369e2ba8622d3474c14c39b941b2c618842d1e47 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Fri Nov 11 19:46:07 2022 -0500 Add a ValidatesContainer integration test for use_sibling_sdk_workers (#24099) commit 137799672eb559a7586262e6a8a73d1ab3580e44 Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 15:30:01 2022 -0800 Fix checkArgument format string in AvroIO commit 5d2dbf957e4e82fb3980726940df02ac67e563cd Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Fri Nov 11 15:57:28 2022 -0500 Update staging of Python wheels (#24114) Fixes https://github.com/apache/beam/issues/24110 commit c2021bee1eba0322b43c90841397859048296b21 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 15:33:14 2022 -0500 Bump google.golang.org/api from 0.102.0 to 0.103.0 in /sdks (#24049) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.102.0 to 0.103.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.102.0...v0.103.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6557c91c79480b9d90573d52d257a11c2b160196 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 11:47:12 2022 -0800 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24112) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.29.1 to 1.29.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.29.1...service/s3/v1.29.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 96f9da1ab652156cd143d57e3aa3d94836338f2b Aut…

commit 95c82ec5be90e64f8e62072d4b7742db77b0558f Merge: 7066061152 c796d856f7 Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Tue Mar 28 16:07:43 2023 +0600 Merge pull request #441 from akvelon/issue25283-tobf-sucfuu [ToB] [Frontend] Save code for unauthenticated users commit c796d856f74dfb3d3371c9421e2c09eb4a5196a8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Mar 28 16:05:08 2023 +0600 _isSnippetTypeSavable (#25283) commit 7b487a0b4c2fd83512bc2da53bf403d60520ae3d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Mar 28 15:49:29 2023 +0600 disabled saving solution code (#25283) commit be3e3460527263146eb13c440056d553ae867938 Author: Alexey Inkin <alexey.inkin@akvelon.com> Date: Tue Mar 28 13:22:05 2023 +0400 Improve comments, remove unused code (#25283) commit 3434381ed159e646949e5e736a491b95e0cad66d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 27 20:28:39 2023 +0600 getSavedDescriptor (#25283) commit 7dbb6b44dd7dcb5e0835f62d4d9454b4c5cb7c9a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 27 20:28:07 2023 +0600 _getStandardOrEmptyDescriptor (#25283) commit b9d76e61660b4180efed79cfb19470aad84e7e00 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 27 18:19:49 2023 +0600 comments (#25283) commit cc7ff54d6c231285dc38eb048da753cb7ddf59a8 Merge: 8410493f38 7066061152 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 21:23:19 2023 +0600 Merge branch 'issue24538-save-user-code' into issue25283-tobf-sucfuu commit 8410493f388ab91f42bb04c26a087331dc25f2e5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 20:37:49 2023 +0600 comments (#25283) commit 7066061152a1415297be5b1fb4729a3322b9676f Merge: 879795b632 ddae966f33 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 20:03:34 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit 8575c6e8f1f4a837c62403e941cf31e1511cf290 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 12:33:28 2023 +0600 addressing comments (#25283) commit 172f18f33dc5648e0d5a6105430a10e2baaca8fb Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 20 19:26:20 2023 +0600 comment fixes (#25283) commit 099b0f19955dc119e0de53a098d424d4cf2d6ad3 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 17 13:28:42 2023 +0600 changes for demo (#25283) commit 426d43ee066b02816ceef4deab62f96ee855fdc8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Mar 15 21:42:15 2023 +0600 untested multi-file saving (#25283) commit 879795b632f5863ae9f477c30f40cb18164cb205 Merge: 1521ab9d93 04c2de61e5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Mar 14 20:39:29 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit 8c7807bbb47906cfc4d68c6a30dc740325688bf5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Mar 2 20:24:01 2023 +0600 setUnit & HiveLocalStorageCache (#25283) commit 6b54331285e912d3e989ffcc6d7a5c446616ea2f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Feb 28 18:32:03 2023 +0600 unfinished ContentExampleLoadingDescriptor (#25283) commit 20e097ef00ab04cc25c4fca0598de4d4cd6a7b9a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 27 12:03:59 2023 +0600 deleted google_sign_in from pubspec (#25283) commit bdf6f4948f9a9a8624f2c4417dd9645b6e188f7e Merge: f755775829 1521ab9d93 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 27 12:01:19 2023 +0600 Merge branch 'issue24538-save-user-code' into issue25283-tobf-sucfuu commit 1521ab9d93f1ea1d964a58ac0f7b87213702cd01 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 23 19:36:50 2023 +0600 CodeBlock widget (#25529) commit b638355e33cfbc1cd73e8ccaf7b8b65dfa6c7184 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 23 19:34:00 2023 +0600 added missing notifyListeners (#24538) commit 5d22b4cda7a944cbbd650e6673ed55d63f4b5dc1 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 23 19:33:01 2023 +0600 scrollbar in markdown code block (#25529) commit f7557758290b48fda83b8abaf6487bce36ea6c5e Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 11:37:58 2023 +0600 resolve deprecation warnings todo (#25283) commit 56264acd2da1a565f3dbb98dab1a41dff54ef54b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 11:30:22 2023 +0600 flutter v3.7 support (#24538) commit 08989ab7e0a8bf7623aff19de10e22b38ecb20dd Merge: 9e3666c706 40838f7644 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 11:00:47 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit 6860d1b367e739abc73e35e48774869ae016872f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 20 11:39:36 2023 +0600 Flutter v3.7.3 support for ToB (#25283) commit 9e3666c706ff3bc6a3a02ccaf33738a116820502 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 8 17:29:53 2023 +0600 set sdkId to fix a bug (#24538) commit 5381d9e0c10d29e2e605c941d0b74d0db16a7b27 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 13:37:00 2023 +0600 _Buttons padding (#24538) commit 4a419a611c8e7e28e178fb0488bd308d6c52add8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 11:26:47 2023 +0600 moved hints to the left (#24538) commit 5637c4171a65ccb4a87019b492245b8cdaec38a5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 11:22:07 2023 +0600 scrollable hints (#24538) commit c7c51b1234b87fe374bce6f9b7dc27d299605b00 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 11:00:15 2023 +0600 commit to relaunch RAT check (#24538) commit 8eed5d3174fdf193b9a4c62631e3687c2c505d6f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 14:53:06 2023 +0600 comment fixes (#24538) commit 7f943d51dcc975f39dc666b19e107c8956fb2048 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:48:46 2023 +0600 organized TourNotifier (#24538) commit 52227d538363a74fb96eb7410eb1d606ef34429f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:37:55 2023 +0600 deleted unitProgressByUnitId getter (#24538) commit 4e3d3cfe134688a8e14813092d1819e97a552b01 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:31:54 2023 +0600 alphabetic order (#24538) commit bfc759666b71f4c265285633f1b57bb3e549eabd Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:26:10 2023 +0600 getUnitSavedSnippetId (#24538) commit a836d0180e328d0bcecf614eb7178734422446ff Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 2 18:57:05 2023 +0600 comment fixes(#24538) commit 01611b02f9947d4401ed056528b92e707c619a27 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 1 20:26:36 2023 +0600 comment fixes (#24538) comment fixes (#24538) commit fff0bfa7e4f3e68ae5ac6e7191f7600b22236dca Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 27 01:29:59 2023 +0600 save code status (#24538) commit d4401c9156a8599cb44b2f92e189906615eff5cf Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Jan 26 14:14:18 2023 +0600 comment fixes (#24538) commit f3d1c0f66fad5072008434516ec93bb23bf1f138 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Jan 26 10:48:50 2023 +0600 snippet type enum switcher (#24538) commit 25dec321066e89e2ce9f1f1574b42d002c61fa80 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Jan 25 00:48:14 2023 +0600 comment fixes (#24538) commit f2d2f8fa304b19c6ebf7411745365e5cb59c3557 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 21:32:19 2023 +0600 reverted to unitProgress (#24538) commit a5310e0eb21d00dc8e3256562e5530bc21fa496c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 21:28:40 2023 +0600 missing notifyListeners (#24538) commit 18ecd23b17cc9397311c2a0a51bbef25c62e2e0b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 21:26:52 2023 +0600 reset is shown immediately (#24538) commit ea8acc0d68528fd435a2134feb1ad35076114fa4 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 18:58:41 2023 +0600 removed comment after discussion (#24538) commit 92b4ec47c0526523da672f4472f2776a46e40de5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 09:06:57 2023 +0600 tested with show solution (#24538) commit ebce111cc242e6f2323717c6fda0761d1e69bdc9 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sun Jan 22 22:28:47 2023 +0600 set show solution false on reset (#24538) commit ece905cfb6d8abe5fc64f3f8d023699a17047fd5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sun Jan 22 22:08:53 2023 +0600 pre-review fixes (#24538) commit 725e8d108678c0b9da1718a478468207c1e4c446 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 20 19:06:51 2023 +0600 post-merge commit (#24538) commit b6cb6b93e92df4571df84bdbc524a815d7ca2308 Merge: eb23673d01 428ec97e30 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 20 19:06:03 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit eb23673d0178f90b313fb18c591505f8952eb24b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 20 18:39:35 2023 +0600 setSaveCodeListener to activeFileController.codeController (#24538) commit d3b0085253c985e52351bf541587d447db08c94d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Jan 17 15:28:17 2023 +0600 merge fixes (#24538) commit 8cd106d00f6369b8dec1f92ca44190ded3378d30 Merge: a8502624ca a5e6d90081 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Jan 17 15:27:42 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit a8502624ca74909e27e5f90508c0bea22619b382 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Dec 27 19:06:23 2022 +0600 getUnitSnippets in unit progress (#24538) commit 8dd70984cdb3aebd676c2d92f4508b3b0d6a7ae5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 8 22:33:32 2022 +0600 user menu without padding commit 90b9e3d49adf6c075fbdeef881ee161b109fd1df Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 8 22:32:37 2022 +0600 save user code barely works commit 44172c57c2d806b6521fc966febabb949e3e77a7 Merge: d442e63668 59d0fd3129 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 8 14:15:37 2022 +0600 Merge branch 'issue24394-tobf-hint' into issue24538-tobf-save-user-code commit d442e63668b15330bd8a68fb18312f5cc06e5640 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Dec 7 15:22:57 2022 +0600 no duplicate slash with reg exp (#24538) commit e76860497a6983cd28e29f3d4065c9331ac32660 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Dec 6 10:11:12 2022 +0600 updated import (#24538) commit 63745d564ace082d34be5360cfd264cfaef6e86a Merge: d15913b289 af936d0b11 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Dec 6 10:02:57 2022 +0600 Merge branch 'issue23692_auth' into issue24538-tobf-save-user-code commit 59d0fd3129bcab3b55a66f671898ce6c5bcc84df Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 20:50:12 2022 +0600 solution managed inside of tour notifier (#24394) commit 5ca484661d330a0d7bea371a767f8d37402db520 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 15:57:30 2022 +0600 show solution snippets (#24394) commit 5e3ac5857e6fbe585e8f6b43cca75265b59e25c2 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 14:20:43 2022 +0600 renamed to unit content widget (#24394) commit 372b97fb5cd457125137c3ceb71332acecbc9117 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 12:01:39 2022 +0600 group progress indicator size adjustment (#24394) commit 48ea57d7c97a817458741d7062955add3b758c76 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 11:50:08 2022 +0600 title in unit content (#24394) commit 338c67317cdc9eec5a4046af9e5599737408baf2 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 11:01:57 2022 +0600 extracted widgets, hints emptiness check (#24394) commit 36d3105f52fad0aa37ddaeaac340ee31a8a4e86b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 21:12:05 2022 +0600 hint in dialog (#24394) commit af936d0b115bf42f21310719dc1b18e580010421 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 14:59:52 2022 +0600 added firebase_options.dart into gradle rat exclusions & added a missing license (#23692) commit 9e1bbb081c575358445750694d27e393ebd72406 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 14:02:46 2022 +0600 specific imports (#23692) commit 9720c134b5733fb439ae2b4df12bff07c359a95b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 13:06:57 2022 +0600 moved dismissible overlay (#23692) commit 869b40cefa9222b5a61db0125f6c54340dfc918c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 13:06:40 2022 +0600 extracted overlay body (#23692) commit c8db1ef778f401a9029c113a784faeb94919d660 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 13:05:12 2022 +0600 refined open overlay (#23692) commit 8b639d08b3d6101ba9b13a6f6fe32b7cc7261559 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 12:50:35 2022 +0600 all caches extend cache (#23692) commit a10795205507a66b8cd90ed8caaf112663a60c8c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 12:47:19 2022 +0600 renamed user progress cache to unit progress (#23692) commit e215ec4c676e88d85fad3ba08e222ac19d8953b8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Nov 30 10:37:05 2022 +0600 generated files commit 96e41f7d01c50e2cfa96fbc120fca69b3d29cb39 Merge: 1e38328e56 37fb90c98d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Nov 30 10:30:05 2022 +0600 Merge from master commit 1e38328e564f72b4e18439c60610902423223dbd Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 15:40:15 2022 +0600 removed "fix exception" todo after filing an issue (#23692) commit b6c4e3e66c30fa4cbdfbbefd1a41e649c32b9ecb Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:52:29 2022 +0600 added async (#23692) commit 27b26e41a81f07faac060c3d2e45f5b7bea19d5a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:31:24 2022 +0600 clearUpdatingUnitId (#23692) commit 4b2d855920d6d63dfff4412fc9bdfb9ddbbdd695 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:30:12 2022 +0600 renamed user progress model to unit progress (#23692) commit ef637403aa114f43f82ac005254632f2d35e0bad Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:09:58 2022 +0600 rearranged completeUnit (#23692) commit fb1fea1af8069ae02ba261781ca923cd40796c97 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 11:43:06 2022 +0600 comment fixes (4) commit 749d8b19bea072be84cace76bd1fb998a3a76531 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 11:03:34 2022 +0600 pubspec.lock commit 0a76b4019647ebcbec76739b078b01fddd37789b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 11:03:17 2022 +0600 pubspec.lock ignored only in PGC commit c6004b0f220be0959b1fadc1fcffb04e4004cd97 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 21:17:16 2022 +0600 no final else commit d55622e30df308eb61a42ecd1f03930012ec0cb8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 21:15:15 2022 +0600 missing await commit 65dc5219943a948e7922da44ce34b54474bffaf1 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 21:12:22 2022 +0600 updating & blocking complete unit button (#23692) commit 1bbc09279604dd2025ca6c93cfc85d2df9bece09 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 14:36:25 2022 +0600 comment fixes (3) commit 860780836bdf8cdbbcbe5952232abc3544effc9e Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 12:44:52 2022 +0600 pubspec.lock in gitignore commit a575717869b239f59374caf74f81db74191c3781 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 12:41:44 2022 +0600 deleted pubspec.lock commit c2f74a59ed6a364d04a9cb0d11ea0e1f0ce5ffbf Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sun Nov 27 21:05:49 2022 +0600 review comments (2) commit adbdeb6eec823178d487a3a3b3bba07ceda6efa9 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 15:15:57 2022 +0600 deleted file (#23692) commit 6ed38acb2beb4b960eb7a77da1282683e2805bc0 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 15:14:07 2022 +0600 unused import (#23692) commit 2c90ced90376df81d0989c233995320a73c7feb1 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 14:55:05 2022 +0600 login_overlay (#23692) commit 6d21e1ea3f0837616a185e4b1f0d48398799791d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 14:52:08 2022 +0600 kOpenLoginOverlay (#23692) commit 64965a1e1f49f0240b4e27a387aad099caade1b2 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 14:44:19 2022 +0600 canCompleteCurrentUnit (#23692) commit 7fbaf3843394530602a87805fa274ace7620ae29 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Nov 23 15:21:56 2022 +0600 new configs & todos commit 0b6fc8f805d3417f819ebafadc40f2571d73df99 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 22 21:07:35 2022 +0600 candidate repository commit 3df27bc7fc71f12d48b3aa0bf5d58e57ba1327b9 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 22 15:12:05 2022 +0600 untested refinement (1) commit 52838e500fc106414dbb47d0c05fe4042c812509 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 21:54:53 2022 +0600 notifier objects naming commit 2068ce9e8da8b6caa41e1be09dfa04e26e2ca567 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 21:50:25 2022 +0600 deleted show sdk selector commit a6597d434a3aed640fc230976afd3637217d8792 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 21:44:55 2022 +0600 untested refinement commit 10b287b89456368b146fd9bc1edc38bf0c28fc73 Merge: 4aedd5a607 5ad067f7cc Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Mon Nov 21 19:06:30 2022 +0600 Merge pull request #324 from akvelon/tour-header-sdk-selector SDK selector in tour scaffold commit 5ad067f7ccdb976fcdb26102d983bea936a9ba83 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 17:55:38 2022 +0600 non-nullable onChanged commit 4aedd5a6072e92dbfd7487652619e7a14e291375 Merge: 6f53e30d39 70c0d61d3a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 16:06:38 2022 +0600 SDK selector with fixed comments commit 6f53e30d39ba793850153a2a6077fbe6bbab2b51 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 16:04:31 2022 +0600 cache license commit 70c0d61d3a96e120d312f1aecc6a38e73e551697 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sat Nov 19 20:46:18 2022 +0600 comments (0) commit de273baf06cc76e4da1f94064131f0093e265f0c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 20:51:31 2022 +0600 hide back button after navigating from welcome commit 3a10e8d7633bb4d6d9bd39ba5515aacae6208554 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 20:48:13 2022 +0600 show sdk selector in welcome screen commit de0f7c34f2a686f22137c464d8815a5f6c2cb9c4 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 20:00:17 2022 +0600 sdk selection works with auth commit 3c534d864b5a36cf4f63b64af0332753fd420b9a Merge: 12a685d1f0 b78bbd33dc Author: Darkhan Nausharipov <nausharipov@gmail.com> Date: Fri Nov 18 19:41:06 2022 +0600 Merge remote-tracking branch 'origin/tour-header-sdk-selector' into auth_sdk_demo commit 12a685d1f00140981c8241a2477b5ee0707e47be Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 19:36:34 2022 +0600 unit file fix commit d25eaf52df834b655421319b14e43e0afc7c63f1 Merge: 5297d2548c cf56af2799 Author: Darkhan Nausharipov <nausharipov@gmail.com> Date: Fri Nov 18 19:22:22 2022 +0600 Merge branch 'master' into issue23692_auth commit 5297d2548ced6027c58b75b9d917fbfeb7052369 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Oct 19 19:50:43 2022 +0600 auth, complete unit, user progress AuthNotifier draft (#23692) Comments (#23692) Comments (#23692)(1) sign in with google works (#23692) new configs (#23692) get user progress draft (#23692) comment fixes (#23692) sign in in IntroTextBody (#23692) reverted config (#23692) comment fixes (#23692) WIP before rebase (merge) (#23692) Squashed commit of the following: commit bff4919ff00ec3b5d7186efde41c884dfc4c8344 Merge: 79ba69483a ce8d618c77 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Thu Nov 17 10:34:02 2022 +0100 Merge pull request #24186: Uses _all to follow alias/datastreams when estimating index size commit 79ba69483a84ea0278d0b0ddb141200739607c77 Merge: 245fea9040 b7e860a762 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 20:47:40 2022 -0800 Merge pull request #24218: Update Python wheel format for RC validation commit 245fea904014cd58d4148807463dbaa40000774c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Nov 16 18:12:33 2022 -0800 Bump loader-utils from 1.4.1 to 1.4.2 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (#24191) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit e1de8e78deeb5d17617fda6591429eaaf8abb8a2 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 20:48:06 2022 -0500 Fix PythonLint (#24219) commit b7e860a7621771c300dcec625655f87e62591323 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:28:31 2022 -0800 updates commit c2feb09ea49dd815b69c65e531ce34128756d988 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:06:08 2022 -0800 updates commit ce8d618c77d23e20a1ddb128bb8183048597d096 Author: egalpin <egalpin@users.noreply.github.com> Date: Wed Nov 16 16:43:57 2022 -0800 Adds test for following aliases when estimating index size commit 959719d01c627328c0ca2849d2b7e2c9b322d4d1 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 15:16:06 2022 -0800 Temporary update Python RC validation job commit b952b41788acc20edbe5b75b2196f30dbf8fdeb0 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 14:18:12 2022 -0500 Python TextIO Performance Test (#23951) * Python TextIO Performance Test * Add filebasedio_perf_test module for unified test framework for Python file-based IOs * Fix MetricsReader publishes metrics duplicately if more than one load test declared. This is because MetricsReader.publishers was static class variable * Fix pylint * Distribute Python performance tests random time at a day instead of all at 3PM * Add information about length conversion commit 017f2cbde124af40a43be99ec88289fcf63c1c95 Merge: fef8acdbc0 88dba4f494 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 10:39:52 2022 -0800 Merge pull request #24187: Add a reference to Java RunInference example commit fef8acdbc0ecbcc85b49144adaf8830e3bc6b2de Merge: 6e9187e67e ead245539d Author: Ahmet Altay <aaltay@gmail.com> Date: Wed Nov 16 10:24:53 2022 -0800 Merge pull request #24199 from Laksh47/issue#24196 refs: issue-24196, fix broken hyperlink commit 6e9187e67e1bd8f73997f437f0ed4c29880ed73b Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Wed Nov 16 22:33:50 2022 +0600 [Tour of Beam] [Frontend] Content tree URLs (#23776) * Content tree navigation (#23593) Unit content navigation (#23593) Update URL on node click (#23593) Active unit color (#23593) removeListener in unit (#23593) First unit is opened on group title click (#23593) WIP by Alexey Inkin (#23593) selectedUnitColor (#23593) Unit borderRadius (#23593) RegExp todo (#23593) added referenced collection package to remove warning (#23593) small refinement (#23593) expand on group tap, padding, openNode (#23593) group expansion bug fix (#23593) selected & unselected progress indicators (#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (#23593) * fixed _getNodeAncestors (#23593) * get sdkId (#23593) * addressing comments (#23593) * sdkId getter & StatelessExpansionTile (#23593) * expand & collapse group (#23593) * StatelessExpansionTile (#23593) * license (#23593) * ValueChanged and ValueKey in StatelessExpansionTile (#23593) Co-authored-by: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Co-authored-by: Alexey Inkin <alexey.inkin@akvelon.com> commit b33fac2aa533d77cfa47f88466c8cd6bd3f3e864 Author: Bruno Volpato <bvolpato@google.com> Date: Wed Nov 16 10:51:11 2022 -0500 Use only ValueProviders in SpannerConfig (#24156) commit 5f013ab6567ec75b460b2081d7f89d332320caff Author: Robert Burke <lostluck@users.noreply.github.com> Date: Wed Nov 16 07:23:10 2022 -0800 revert upgrade to go 1.19 for action unit tests (#24189) commit 9337f4dbecc929886f8559949a082a649fd9d1bb Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 10:18:42 2022 -0500 Fix Python PostCommit Example CustomPTransformIT on portable (#24159) * Fix Python PostCommit Examples on portable * Fix custom_ptransform pipeline options gets modified * Specify flinkConfDir commit ead245539d01dec0f3e08699c1e1cc6777a5ef0e Author: Laksh <lakshmanansathya@gmail.com> Date: Wed Nov 16 09:32:46 2022 -0500 refs: issue-24196, fix broken hyperlink commit e83a996d4374d467d95bcfad7166905622ec615c Merge: 2fc56ec663 ffdee0b6ed Author: Jan Lukavský <je.ik@seznam.cz> Date: Wed Nov 16 15:15:31 2022 +0100 Merge pull request #24192: Re-use serializable pipeline options when already available. commit ffdee0b6edb8638c78a65ec85c727ea5dde1cb2f Author: Jozef Vilcek <jvilcek@zetaglobal.com> Date: Mon Nov 14 16:48:18 2022 +0100 Re-use serializable pipeline options when already available (#24192) commit 88dba4f494829b2b3530b767fb8c5252e0d2ba44 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Tue Nov 15 16:21:22 2022 -0800 Add a reference to Java RunInference example commit 2fc56ec663e335cfcf37dc57d471f79b601414f4 Merge: f763186987 83f1bc19b9 Author: Kenn Knowles <kenn@apache.org> Date: Tue Nov 15 16:16:47 2022 -0800 Merge pull request #24142: Fix arguments to checkState in BatchViewOverrides commit f763186987c00ba1d26efdc35406436a1fa69a9a Merge: c2bc2135e9 0d7ca04182 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 15:25:20 2022 -0800 Addresses #24161 Updated README of Interactive Beam commit c2bc2135e9bce715990a5d5551e2bc2dc0311da4 Author: Doug Judd <nuggetwheat@gmail.com> Date: Tue Nov 15 14:48:26 2022 -0800 Strip FGAC database role from changestreams metadata requests (#24177) Co-authored-by: Doug Judd <nuggetwheat@google.com> commit af637974f96ad1b5110d7dea3f9a26c68e19a51b Author: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Date: Tue Nov 15 17:16:43 2022 -0500 Add custom inference function support to the PyTorch model handler (#24062) * Initial type def and function signature * [Draft] Add custom inference fn support to Pytorch Model Handler * Formatting * Split out default * Remove Keyed version for testing * Move device optimization * Make default available for import, add to test classes * Remove incorrect default from keyed test * Keyed impl * Fix device arg * custom inference test * formatting * Add helpers to define custom inference functions using model methods * Trailing whitespace * Unit tests * Fix incorrect getattr syntax * Type typo * Fix docstring * Fix keyed helper, add basic generate route * Modify generate() to be different than forward() * formatting * Remove extra generate() def commit a014637106970a0a0e9eb7944aa5caf79fa5fd37 Author: egalpin <egalpin@users.noreply.github.com> Date: Tue Nov 15 13:57:54 2022 -0800 Uses _all to follow alias/datastreams when estimating index size Fixes #24117 commit 0d7ca041823bc2b09f76f86fdfd1d0b9508c9c88 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:57:27 2022 -0800 Minor update commit e8fc759d756f4a987e41d2b9da56b906a6cd7736 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:52:18 2022 -0800 Updated README of Interactive Beam Removed deprecated cache_dir runner param in favor of the cache_root global option. commit 08d5f72e5f35d41f3e9fa9fe799caea6bed1b7a7 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Tue Nov 15 16:34:21 2022 -0500 [Python]Support pipe operator as Union (PEP -604) (#24106) Fixes https://github.com/apache/beam/issues/21972 commit 526e7a58b62682582c27173ab21ed8667ddab766 Author: Scott Strong <scott.strong87@gmail.com> Date: Tue Nov 15 16:26:45 2022 -0500 Using Teardown context instead of deprecated finalize (#24180) * Using Teardown context instead of deprecated finalize * making function public Co-authored-by: Scott Strong <scott.strong@wunderkind.co> commit fb4d1d4dea7b26ed538a9f6aca0ed41e8c300e37 Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 16:25:22 2022 -0500 Fix broken json for notebook (#24183) commit f98db2008a97f4546d036ddf0dddfee8c87eb58a Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 15 12:49:23 2022 -0800 Update automation to use Go 1.19 (#24175) Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com> commit e5f58504eef1fdeebe0402cda8a2df259169c704 Author: Brian Hulette <bhulette@google.com> Date: Tue Nov 15 12:25:13 2022 -0800 Add error reporting for BatchConverter match failure (#24022) * add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else commit 3037747f66f0d71d65b6c65745b4f8942c22f05a Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 14:13:04 2022 -0500 Fix broken notebook (#24179) commit b2b1c739ce37690923891934ee317f799db937a2 Author: MakarkinSAkvelon <67736809+MakarkinSAkvelon@users.noreply.github.com> Date: Tue Nov 15 21:53:06 2022 +0500 [Playground] Move Playground in GKE and Infrastructure change (#23928) * changes to updated master branch * Change workflow * ingress changes * Certificate was added * Updates for cloud build backend * Update main.tf * Create main.tf * Create variables.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Create output.tf * Update output.tf * Update output.tf * Update main.tf * Update build.gradle.kts * Update output.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update output.tf * Update main.tf * Update main.tf * Update output.tf * Create variables.tf * Update main.tf * Update main.tf * Delete playground/terraform/infrastructure/cluddns directory * Update main.tf * Update output.tf * Update output.tf * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update README.md * Update README.md * helm folder name was changed * Update README.md * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Updates to readme * Fix DNS name * HelmChart was changed * Some workflows were changed * Remove unused file * playground-examples return * add license information * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * remove "stg" folder * Update README.md * Remove unused files * DNS Removed * var name changed * remove DNSName from var file * 1 * Clear terraform * remove unused records * gradle check * grade last change * issue fix * fix * 1 * run * test * Index creation for Gradle * Add IndexCreation in gradle * Update README.md * Update README.md * Fix names for Frontend * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Redis fix * services fix * Update variables.tf * change order in gradle * Fix Config.g.dart file issue * Update README.md * Playground workflow update Co-authored-by: Sergey Makarkin <sergey.makarkin@akvelon.com> Co-authored-by: Sergey Makarkin <sergey_makarkin@quicktest1.c.apache-beam-testing.internal> Co-authored-by: ruslan-ikhsan <ruslan.ikhsanov@akvelon.com> Co-authored-by: Alex Kosolapov <alex.kosolapov@gmail.com> commit 85df5f2eb2f299e28b36be0cce7b9c19d62124da Author: Yi Hu <yathu@google.com> Date: Tue Nov 15 11:38:13 2022 -0500 Eliminate CalciteUtil.CharType logical type (#24013) * Eliminate CalciteUtils.CharType logical type * Replace CalciteUtils.CharType to String Note that CalciteUtils still omits the precision of BINARY/VARBINARY/CHAR/VARCHAR as what it originally did. Support of the precision of these calcite types involves make use of making use of the overload method RelDataTypeFactory.createSqlType(var1, var2). * Replace every reference of CalciteUtil.CharType to generic PassThroughLogicalType check * Add TODO to Support sql types with arguments * Use VariableString in LogicalTypeTestCase commit f349f41010c5b238ff6020f7de718f938eef3c5e Author: alexeyinkin <alexey.inkin@akvelon.com> Date: Tue Nov 15 20:04:01 2022 +0400 Configure flutter_code_editor options with Hugo shortcode (#23926) (#24031) * Configure flutter_code_editor options with Hugo shortcode (#23926) * Minor fixes (#23926) * Refactor after review (#23926) commit 0f4ca6363b3ce0e5de3ad36517bb406aa6391a18 Author: Rebecca Szper <98840847+rszper@users.noreply.github.com> Date: Tue Nov 15 06:10:13 2022 -0800 Editorial review of the ML notebooks. (#24125) * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Update examples/notebooks/beam-ml/custom_remote_inference.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback * Update examples/notebooks/beam-ml/run_inference_sklearn.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 5bd34ede026253326ebff1a7e4f9edb5f71b4a2c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 15 07:17:28 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24131) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.3.2 to 1.11.39. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.3.2...feature/s3/manager/v1.11.39) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2ee809fa0ca7689dd0279e186ebc02d9569a8429 Merge: e3b9bdb2e6 563c66d6fd Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Tue Nov 15 11:01:14 2022 +0100 Merge pull request #23065: [Website] Update copy icon styles commit e3b9bdb2e607d85a4017ba7839000e92a0ad83c4 Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:40:50 2022 +0100 [Dockerized Jenkins] Fix build of dockerized jenkins (fixes #24053) (#24054) commit faaac2ab6e010374cb2be0e95a5dd345836a2a2c Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:38:59 2022 +0100 [Dockerized Jenkins] Update README how to use local repo (#24055) commit 689e70b5131620540faf52e2f1e2dca7a36f269d Author: Damon <damondouglas@users.noreply.github.com> Date: Mon Nov 14 17:34:29 2022 -0800 Implement embedded WebAssembly example (#24081) commit e1bf6c42950e8013f35e35fb9fee8017e01e5010 Merge: eddac84126 10337d2868 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 15:22:14 2022 -0800 Merge pull request #24160 Rename the test_splits flag to direct_test_splits. commit eddac841261228a2c63fa9b225c520ae0f853806 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 15:05:05 2022 -0800 More dataset templates to clean up (#24162) commit 2adb68bd12743566cc89b596bf204d7c807eb62d Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 13:28:13 2022 -0800 Adding a quickstart to README for the TS SDK (#23509) * More of a quickstart for the TS SDK * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 10337d28685ad5712e2ad8608977ec5c5e0e6b6b Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 12:46:32 2022 -0800 Rename the test_splits flag to direct_test_splits. This avoids possible flag conflicts. commit 48c70cc30742b45b17a1d18ece2f0d079bee3915 Author: arne-alex <108519096+arne-alex@users.noreply.github.com> Date: Mon Nov 14 21:33:02 2022 +0100 Merge pull request #23333: Track time on Cloud Dataflow streaming data reads and export via heartbeats commit 9c83de646ab52bd0b05e3346190dd55cd68b2a8b Author: Johanna Öjeling <51084516+johannaojeling@users.noreply.github.com> Date: Mon Nov 14 21:19:44 2022 +0100 Add more tests for S3 filesystem (#24138) commit 9e9c6d797ba52b460f83131431c8e53aebbbc9ac Merge: d5d76b9745 c600444e1d Author: Ning Kang <ningkang0957@gmail.com> Date: Mon Nov 14 12:06:15 2022 -0800 Merge pull request #24029 from apache/dependabot/npm_and_yarn/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/loader-utils-1.4.1 Bump loader-utils from 1.4.0 to 1.4.1 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel commit d5d76b974592d45de368ab641647ca5cc4ec12ec Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 15:03:28 2022 -0500 Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils (#23969) * Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils * Add TODO about java.time migration commit 330cc2010c9f4a2d4e30318bf50a4109ec1cd392 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 12:02:10 2022 -0800 Cleanup stale BQ datasets (#24158) * Cleanup stale BQ datasets * addressing comments commit 4a044999b8ed4bcd41f816f3a23ccb5da00c4c38 Merge: e563b9dd2f 5bd75c25de Author: Heejong Lee <heejong@gmail.com> Date: Mon Nov 14 11:16:00 2022 -0800 Merge pull request #24076 from chamikaramj/multilang_java_updates Updates Multi-lang Java quickstart commit e563b9dd2f3aa0484e6cdc08869991b5e438023e Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Mon Nov 14 20:56:35 2022 +0300 [Tour Of Beam] verify that unit exists when saving progress (#24118) * AIO * Update learning/tour-of-beam/backend/integration_tests/auth_test.go Co-authored-by: Danny McCormick <dannymccormick@google.com> * nit Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 774923e0dd089de870bfa5c77063ae2b28f79347 Merge: 71785de528 1ad0cbc445 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 09:52:26 2022 -0800 Merge pull request #24141: Fix checkArgument format in GcsPath commit 71785de52864313c2e3b14fe72a2a63281343617 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 11:54:37 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.17.10 to 1.18.0 in /sdks (#24151) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.17.10 to 1.18.0. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.17.10...config/v1.18.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 50d591d6cb3e799bee4e29dfc593c693a86e6276 Author: Bruno Volpato <bvolpato@google.com> Date: Mon Nov 14 11:50:01 2022 -0500 Change DataflowBatchWorkerHarness doWork error level to INFO (#24135) commit 5a72696bfda09fdb905ba8e58b636f8494ef955f Merge: ee0a5836d6 0633fe9634 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:12:12 2022 -0800 Merge pull request #24149: Remove extraneous jetbrains annotation commit ee0a5836d69b776834eb3bd9b2bd02eb5252c333 Merge: d001a69e1a 137799672e Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:11:00 2022 -0800 Merge pull request #24132: Fix checkArgument format string in AvroIO commit d001a69e1a58701d6ed4fcb5e3fb7a0921301dad Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 10:56:54 2022 -0500 Test Dataproc 2.1 with Flink load tests (#24129) * Test Dataproc 2.1 with Flink load tests * Minor fix flink_cluster script commit caabd9be52887ad70c8a4269395c893811ac6a84 Author: Israel Herraiz <ihr@google.com> Date: Mon Nov 14 16:03:39 2022 +0100 Make MonotonicWatermarkEstimator work like its Java SDK equivalent (#24146) * Make MonotonicWatermarkEstimator work like its Java SDK equivalent The current implementation of MonotonicWatermarkEstimator raises an exception with late messages, which makes the watermark estimator barely usable in real world scenarios. This PR fixes #20041 by making this watermark estimator work like its Java SDK equivalent (`WatermarkEstimators.MonotonicallyIncreasing`). * Update unit tests too * Make linter happy commit 451f6b3e7f58d0a3782ad942c6a1fd9f63932024 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 09:48:23 2022 -0500 Bump golang.org/x/net from 0.1.0 to 0.2.0 in /sdks (#24153) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.1.0 to 0.2.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/compare/v0.1.0...v0.2.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2bb03d62e2d7dc2d8e39040fc9adebccbde74fde Merge: 4e39ef2041 623083cd0a Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 15:01:13 2022 +0100 Merge pull request #24000: [Website] Change headers size from h4,h3 to h2 commit 563c66d6fd32165da14a07747f2764c17a5d24ea Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Wed Sep 7 18:28:42 2022 +0600 [Website] update pre tag copy link styles #23064 commit 4e39ef20410ee51c6040317bcd60171e64c5171f Merge: 223768f782 105ed6fedc Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 10:55:33 2022 +0100 Merge pull request #24115: [Website] update go-dependencies.md java-dependencies.md links commit 223768f782f771f0033b8d0686d86cf4c71fad75 Merge: aa0a35dabf a9da2abee6 Author: Kenn Knowles <kenn@apache.org> Date: Sun Nov 13 18:53:13 2022 -0800 Merge pull request #24136: Fix checkArgument format string in ExecutionStateTracker commit 0633fe9634fe61df7cbc0ecac205d81124fd504a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 15:15:16 2022 -0800 Remove extraneous jetbrains annotation commit 83f1bc19b95935e60ca1f4027d4b60c7e738a84a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 14:16:09 2022 -0800 Fix arguments to checkState in BatchViewOverrides commit 1ad0cbc44594d8405bf4b07a126265238013a02a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 13:41:02 2022 -0800 Fix checkArgument format in GcsPath commit aa0a35dabf9c2a0d9822faff06d939d9a77a3ab6 Author: Kenn Knowles <kenn@apache.org> Date: Fri Nov 11 20:26:30 2022 -0800 Fix checkArgument format string in TestStream (#24134) commit a9da2abee6455bc2cf0f18ba5f6cd7bbaeae669f Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 16:54:27 2022 -0800 Fix checkArgument format string in ExecutionStateTracker commit 369e2ba8622d3474c14c39b941b2c618842d1e47 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Fri Nov 11 19:46:07 2022 -0500 Add a ValidatesContainer integration test for use_sibling_sdk_workers (#24099) commit 137799672eb559a7586262e6a8a73d1ab3580e44 Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 15:30:01 2022 -0800 Fix checkArgument format string in AvroIO commit 5d2dbf957e4e82fb3980726940df02ac67e563cd Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Fri Nov 11 15:57:28 2022 -0500 Update staging of Python wheels (#24114) Fixes https://github.com/apache/beam/issues/24110 commit c2021bee1eba0322b43c90841397859048296b21 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 15:33:14 2022 -0500 Bump google.golang.org/api from 0.102.0 to 0.103.0 in /sdks (#24049) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.102.0 to 0.103.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.102.0...v0.103.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6557c91c79480b9d90573d52d257a11c2b160196 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 11:47:12 2022 -0800 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24112) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.29.1 to 1.29.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.29.1...service/s3/v1.29.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 96f9da1ab652156cd143d57e3aa3d94836338f2b Author: Yi Hu <yathu@google.com> Date: Fri Nov 11 14:34:22 2022 -0500 More cleanup containers (#24105) * More cleanup containers * prebuilt_beam_sdk/beam_python_prebuilt_sdk no longer exists in gcr.io Add correct paths to cleanup * Bump grace time to 15 days * Run job daily as in code comment * Set grace period to 30 days commit 836766ddb83d37463e4b036f156b8d7e15e3864b Author: Ritesh Ghorse <riteshghorse@gmail.com> Date: Fri Nov 11 14:30:30 2022 -0500 upgrade testcontainer dependency (#24123) commit 9fcd20c3712536f2d4580beead678cdbb6fd4746 Author: Damon <damondouglas@users.noreply.github.com> Date: Fri Nov 11 11:12:11 2022 -0800 Implement PubsubRowToMessage transform (#23897) * Begin PubsubRowToMessage Impl * Complete working draft * Unit tests validate user and non-user fields * Finish tests on supporting methods * Pass checks before finalizing tests * WIP * fix timestamp * finalize tests * Finalize code comments * Clean up check findings * Add InputSchemaFactory * Patch code comment typo commit 3a6fcc1ca4d07a467464ed1214a94b5c9c147295 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Fri Nov 11 19:13:45 2022 +0300 disable (#24121) commit 027fb142038b45c443d87af96cac082264c43188 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 10:26:23 2022 -0500 Bump cloud.google.com/go/bigtable from 1.17.0 to 1.18.0 in /sdks (#24113) Bumps [cloud.google.com/go/bigtable](https://github.com/googleapis/google-cloud-go) from 1.17.0 to 1.18.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.17.0...pubsub/v1.18.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/bigtable dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit b2d28a64bb0837dd97e8be7f97f31f88d380f110 Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 11 08:45:14 2022 -0500 Add TFX support in pydoc (#23960) * Add TFX support in pydoc * Wording commit 105ed6fedcb6ff33d4a43940c342a653e67cb25e Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Thu Nov 10 17:47:54 2022 +0600 [Website] update go-dependencies.md java-dependencies.md programming-guide.md links #24084 commit 4b96193250a63b27721a8c5e1a3bd6ecff983093 Author: Brian Hulette <bhulette@google.com> Date: Thu Nov 10 16:51:06 2022 -0800 Remove TheNeuralBit from the pool of Python reviewers (#24108) commit 8f8f089a5f565f179905984cef3522a0577d9219 Author: Sanil Jain <snjain@linkedin.com> Date: Thu Nov 10 16:47:31 2022 -0800 Wire SamzaPipelineOptions to Exeption listener interface (#24109) commit b3186ba91f9bb22764d47b78eb2b7ff017080f75 Author: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> Date: Thu Nov 10 18:04:50 2022 -0500 Support using BigQueryIO Storage Read API with SchemaTransforms (#23827) * support schema transform for bq direct read method * use vendor Strings import * add BigQueryServices argument to config object * suppress nullability errors * add package-info.java file to providers subdirectory * removing accidentally created file * added documentation and moved configuration filclass into provider class * validate config params before expansion. config has a validaate me method * updated URN identifier to follow standards commit a4a94da1936d8cd2f8d1145fd2e8329fe06a2990 Author: Oleh Borysevych <oleg.borisevich@akvelon.com> Date: Fri Nov 11 00:28:43 2022 +0200 fixing linter error (#24104) commit 156a6099d71a155c3379340a0b98256306e1755e Author: Danny McCormick <dannymccormick@google.com> Date: Thu Nov 10 17:26:15 2022 -0500 Add blog post on new ML resources (#24071) * Create ml-resources.md * Add ensemble notebook image * Add image link * Fix image link * Add ml-landing page image * Add image + move around * Add paragraph on upcoming changes * Remove bad whitespace commit 38742d40c895f5d38ee6ee5eb328d11b3262307e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Nov 10 16:12:40 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.5.0 to 1.17.10 in /sdks (#24080) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.5.0 to 1.17.10. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.5.0...config/v1.17.10) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 92cef32915c40d353b51529a87d8f0131b0a3538 Author: BjornPrime <32173247+BjornPrime@users.noreply.github.com> Date: Thu Nov 10 15:30:37 2022 -0500 Num failed inferences (#23830) * created test_increment_num_failed_inferences and test_num_failed_inferences_no_failures * added assertRaises to test_increment_num_failed_inferences * added num_failed_inferences to _MetricsCollector * changed error handling and update() implementation * updated metric name in tests * removed unnecessary else blocking * removed unnecessary inference_args from test_increment_failed_batches_counter() * changed final test_increment_failed_batches_counter assertion * clarified error handling and updated failed_batches_counter initialization * decreased examples array length to 1 to ensure repeatability * troubleshooting tests * trying to get test_increment_failed_batches_counter to fail as expected * corrected assertion details * simplified assertRaises and added reminder comment to assertEqual counter * lint test * lint test passed, resetting pre-commit-config.yaml * fixed lingering linting issues * shortened comment line to comply with linting * formatter worked its magic commit 2341f61d48d838ddaf2e4fb990a49987244c3513 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Thu Nov 10 21:24:55 2022 +0300 [Tour Of Beam] handle CORS pre-flight requests (#24083) * cors * README * nit * nit * -headers, -cache * allow-headers * +1h cache,comment commit 36b0c8f1a76865e4dc3b1eaec1566c3ef9fd4345 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Thu Nov 10 21:21:21 2022 +0300 [Playground] update snippet by persistence_key (#24056) * proto * AIO * -frontend * testify * +license * -build * index * skipKey * Update playground/backend/internal/db/datastore/datastore_db.go Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit df553d1a4c6c6cc9b37ada2a6c84c88d8aef38b9 Author: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> Date: Thu Nov 10 13:17:36 2022 -0500 Add random string at the end of BigQuery query job name to make it resilient to retries (#24041) * add random string at the end of query job name * use deterministic temp table name commit e439f4120ef4c25aa36e5b03756dc7391bdbd211 Author: Pablo <pabloem@users.noreply.github.com> Date: Thu Nov 10 10:06:26 2022 -0800 Improving stale container cleanup script (#24040) * Improving stale container cleanup script * Avoid also latest image * Update .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh Co-authored-by: Yi Hu <huuyyi@gmail.com> * Update .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh Co-authored-by: Yi Hu <huuyyi@gmail.com> Co-authored-by: Yi Hu <huuyyi@gmail.com> commit 8d585242e158babd8dd2ca5a6d5a49d353be2935 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Thu Nov 10 12:58:41 2022 -0500 [Python]Set pickle library at the Pipeline creation stage (#24069) Fixes https://github.com/apache/beam/issues/21615 commit 08b6a524fecc333a12ce42971733ef64ae7d02f1 Author: Janek Bevendorff <janek.bevendorff@uni-weimar.de> Date: Thu Nov 10 18:37:09 2022 +0100 [BEAM-12792] Install pipline dependencies to temporary venv (#16658) commit 74f87b0282a23aa01ecb63288013311d207960aa Merge: 38a85b288a 383ea77bc9 Author: Robert Bradshaw <robertwb@gmail.com> Date: Thu Nov 10 09:10:37 2022 -0800 Merge pull request #23985 Support dynamic sharding in the worker. commit 38a85b288ad10171c444d4092eb969ca5de58af3 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Nov 10 10:39:15 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24077) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.11.1 to 1.29.1. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-g…

* Squashed commit of the following: commit 9f42763a819ae82ae7fc7fe495d43cabae00df25 Merge: 6b8ba1afc7 04c2de61e5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Mar 14 20:05:42 2023 +0600 Merge branch 'master' into issue25255-delete-account commit 6b8ba1afc7155f2357ca3a5c5f23394f770e2ae7 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 21:41:34 2023 +0600 flutter v3.7 support (#25255) commit dc44ba77db8edd7fdf063ed88ee97e7c8e45bc8d Merge: 3f3747cfa0 8cfee7d05e Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 21:34:45 2023 +0600 Merge branch 'master' into issue25255-delete-account commit 3f3747cfa0d48e0a5df367894269a0840b615156 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Feb 14 18:28:43 2023 +0600 comment fixes (#25255) commit a5110e4ef360e3957c3e539ec11fb5a0124838f7 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Feb 14 11:00:55 2023 +0600 progress dialog is undismissible (#25255) commit 2cb7c585da3ebe0fb599dca5ce6887bfa0f65310 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 13 22:50:18 2023 +0600 updated to app_state 0.9.2 (#25255) commit 7626f8fef3ea42ba792cf3a9869dfb36a827191c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 13 22:40:25 2023 +0600 pop progress dialog with navigatorKey (#25255) commit 12882fbce983e26547c06bc23e10f35adde87e2b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 10 21:46:08 2023 +0600 addressing review (#25255) commit 440d0b04e913d368bdb8bc46f018dee74eca5a63 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 10 21:17:03 2023 +0600 addressing review comments (#25255) commit 6005103ac778669665518c7312ed726a27c99d8d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 10 15:13:54 2023 +0600 addressing review comments (#25255) commit fde6ace2e43ecc559426e733faea49dbb491ec7b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 9 21:18:15 2023 +0600 showOverlay (#25255) commit 09a5526aa9f415081914e60b0737ceeef2686502 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 9 20:08:48 2023 +0600 addException in deleteAccount (#25255) commit a2f43bf4d5850032306192490bc485e8e70aaa45 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 9 20:00:04 2023 +0600 BeamRouterDelegate for showing toasts (#25255) commit f1a2cecd9369dfe98cd1fa635fa1aef700f0a3a5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 9 19:54:12 2023 +0600 comment fixes (#25255) commit 8c3e4b5205e257edf2efc6e2faf44d2d9a15d7ab Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 9 14:40:07 2023 +0600 toastNotifier todo (#25255) commit eda3f16254e6bebb503359efd2c62a4b0bd48091 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 9 14:39:43 2023 +0600 showProgressOverlay (#25255) commit 2ab46ec1199414d3a2335d2d00523f15446a5124 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 8 19:07:00 2023 +0600 GitHub auth untested (#25255) commit 17b8c429d59842e04f2ff6b072fd9f0674e56a9f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Feb 7 14:51:56 2023 +0600 comment fixes (#25255) commit bc4d930945bc76e261c7a815ff54961b862032ca Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 23:05:50 2023 +0600 todo (#25255) commit d67cf474ceb5932b996438953536a87a355a6077 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 22:51:55 2023 +0600 pre-review fixes (#25255) commit eda0bfec9a04d2a26d923085410dde66fb3fe235 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 22:39:12 2023 +0600 delete my account (#25255) commit a34b1701c822cde7c41d7e3b98947cc5d1124a05 Merge: aee2c844ad f9f8c3ad6a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 21:05:19 2023 +0600 hint warning (#25116) commit f9f8c3ad6a3a3e4ddecda49807066aeae4787223 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 21:03:21 2023 +0600 text and popup (#25116) * Squashed commit of the following: commit 95c82ec5be90e64f8e62072d4b7742db77b0558f Merge: 7066061152 c796d856f7 Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Tue Mar 28 16:07:43 2023 +0600 Merge pull request #441 from akvelon/issue25283-tobf-sucfuu [ToB] [Frontend] Save code for unauthenticated users commit c796d856f74dfb3d3371c9421e2c09eb4a5196a8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Mar 28 16:05:08 2023 +0600 _isSnippetTypeSavable (#25283) commit 7b487a0b4c2fd83512bc2da53bf403d60520ae3d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Mar 28 15:49:29 2023 +0600 disabled saving solution code (#25283) commit be3e3460527263146eb13c440056d553ae867938 Author: Alexey Inkin <alexey.inkin@akvelon.com> Date: Tue Mar 28 13:22:05 2023 +0400 Improve comments, remove unused code (#25283) commit 3434381ed159e646949e5e736a491b95e0cad66d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 27 20:28:39 2023 +0600 getSavedDescriptor (#25283) commit 7dbb6b44dd7dcb5e0835f62d4d9454b4c5cb7c9a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 27 20:28:07 2023 +0600 _getStandardOrEmptyDescriptor (#25283) commit b9d76e61660b4180efed79cfb19470aad84e7e00 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 27 18:19:49 2023 +0600 comments (#25283) commit cc7ff54d6c231285dc38eb048da753cb7ddf59a8 Merge: 8410493f38 7066061152 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 21:23:19 2023 +0600 Merge branch 'issue24538-save-user-code' into issue25283-tobf-sucfuu commit 8410493f388ab91f42bb04c26a087331dc25f2e5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 20:37:49 2023 +0600 comments (#25283) commit 7066061152a1415297be5b1fb4729a3322b9676f Merge: 879795b632 ddae966f33 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 20:03:34 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit 8575c6e8f1f4a837c62403e941cf31e1511cf290 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 24 12:33:28 2023 +0600 addressing comments (#25283) commit 172f18f33dc5648e0d5a6105430a10e2baaca8fb Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Mar 20 19:26:20 2023 +0600 comment fixes (#25283) commit 099b0f19955dc119e0de53a098d424d4cf2d6ad3 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Mar 17 13:28:42 2023 +0600 changes for demo (#25283) commit 426d43ee066b02816ceef4deab62f96ee855fdc8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Mar 15 21:42:15 2023 +0600 untested multi-file saving (#25283) commit 879795b632f5863ae9f477c30f40cb18164cb205 Merge: 1521ab9d93 04c2de61e5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Mar 14 20:39:29 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit 8c7807bbb47906cfc4d68c6a30dc740325688bf5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Mar 2 20:24:01 2023 +0600 setUnit & HiveLocalStorageCache (#25283) commit 6b54331285e912d3e989ffcc6d7a5c446616ea2f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Feb 28 18:32:03 2023 +0600 unfinished ContentExampleLoadingDescriptor (#25283) commit 20e097ef00ab04cc25c4fca0598de4d4cd6a7b9a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 27 12:03:59 2023 +0600 deleted google_sign_in from pubspec (#25283) commit bdf6f4948f9a9a8624f2c4417dd9645b6e188f7e Merge: f755775829 1521ab9d93 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 27 12:01:19 2023 +0600 Merge branch 'issue24538-save-user-code' into issue25283-tobf-sucfuu commit 1521ab9d93f1ea1d964a58ac0f7b87213702cd01 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 23 19:36:50 2023 +0600 CodeBlock widget (#25529) commit b638355e33cfbc1cd73e8ccaf7b8b65dfa6c7184 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 23 19:34:00 2023 +0600 added missing notifyListeners (#24538) commit 5d22b4cda7a944cbbd650e6673ed55d63f4b5dc1 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 23 19:33:01 2023 +0600 scrollbar in markdown code block (#25529) commit f7557758290b48fda83b8abaf6487bce36ea6c5e Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 11:37:58 2023 +0600 resolve deprecation warnings todo (#25283) commit 56264acd2da1a565f3dbb98dab1a41dff54ef54b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 11:30:22 2023 +0600 flutter v3.7 support (#24538) commit 08989ab7e0a8bf7623aff19de10e22b38ecb20dd Merge: 9e3666c706 40838f7644 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 22 11:00:47 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit 6860d1b367e739abc73e35e48774869ae016872f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 20 11:39:36 2023 +0600 Flutter v3.7.3 support for ToB (#25283) commit 9e3666c706ff3bc6a3a02ccaf33738a116820502 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 8 17:29:53 2023 +0600 set sdkId to fix a bug (#24538) commit 5381d9e0c10d29e2e605c941d0b74d0db16a7b27 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 13:37:00 2023 +0600 _Buttons padding (#24538) commit 4a419a611c8e7e28e178fb0488bd308d6c52add8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 11:26:47 2023 +0600 moved hints to the left (#24538) commit 5637c4171a65ccb4a87019b492245b8cdaec38a5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 11:22:07 2023 +0600 scrollable hints (#24538) commit c7c51b1234b87fe374bce6f9b7dc27d299605b00 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Feb 6 11:00:15 2023 +0600 commit to relaunch RAT check (#24538) commit 8eed5d3174fdf193b9a4c62631e3687c2c505d6f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 14:53:06 2023 +0600 comment fixes (#24538) commit 7f943d51dcc975f39dc666b19e107c8956fb2048 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:48:46 2023 +0600 organized TourNotifier (#24538) commit 52227d538363a74fb96eb7410eb1d606ef34429f Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:37:55 2023 +0600 deleted unitProgressByUnitId getter (#24538) commit 4e3d3cfe134688a8e14813092d1819e97a552b01 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:31:54 2023 +0600 alphabetic order (#24538) commit bfc759666b71f4c265285633f1b57bb3e549eabd Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Feb 3 12:26:10 2023 +0600 getUnitSavedSnippetId (#24538) commit a836d0180e328d0bcecf614eb7178734422446ff Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Feb 2 18:57:05 2023 +0600 comment fixes(#24538) commit 01611b02f9947d4401ed056528b92e707c619a27 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Feb 1 20:26:36 2023 +0600 comment fixes (#24538) comment fixes (#24538) commit fff0bfa7e4f3e68ae5ac6e7191f7600b22236dca Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 27 01:29:59 2023 +0600 save code status (#24538) commit d4401c9156a8599cb44b2f92e189906615eff5cf Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Jan 26 14:14:18 2023 +0600 comment fixes (#24538) commit f3d1c0f66fad5072008434516ec93bb23bf1f138 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Jan 26 10:48:50 2023 +0600 snippet type enum switcher (#24538) commit 25dec321066e89e2ce9f1f1574b42d002c61fa80 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Jan 25 00:48:14 2023 +0600 comment fixes (#24538) commit f2d2f8fa304b19c6ebf7411745365e5cb59c3557 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 21:32:19 2023 +0600 reverted to unitProgress (#24538) commit a5310e0eb21d00dc8e3256562e5530bc21fa496c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 21:28:40 2023 +0600 missing notifyListeners (#24538) commit 18ecd23b17cc9397311c2a0a51bbef25c62e2e0b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 21:26:52 2023 +0600 reset is shown immediately (#24538) commit ea8acc0d68528fd435a2134feb1ad35076114fa4 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 18:58:41 2023 +0600 removed comment after discussion (#24538) commit 92b4ec47c0526523da672f4472f2776a46e40de5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Jan 23 09:06:57 2023 +0600 tested with show solution (#24538) commit ebce111cc242e6f2323717c6fda0761d1e69bdc9 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sun Jan 22 22:28:47 2023 +0600 set show solution false on reset (#24538) commit ece905cfb6d8abe5fc64f3f8d023699a17047fd5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sun Jan 22 22:08:53 2023 +0600 pre-review fixes (#24538) commit 725e8d108678c0b9da1718a478468207c1e4c446 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 20 19:06:51 2023 +0600 post-merge commit (#24538) commit b6cb6b93e92df4571df84bdbc524a815d7ca2308 Merge: eb23673d01 428ec97e30 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 20 19:06:03 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit eb23673d0178f90b313fb18c591505f8952eb24b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Jan 20 18:39:35 2023 +0600 setSaveCodeListener to activeFileController.codeController (#24538) commit d3b0085253c985e52351bf541587d447db08c94d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Jan 17 15:28:17 2023 +0600 merge fixes (#24538) commit 8cd106d00f6369b8dec1f92ca44190ded3378d30 Merge: a8502624ca a5e6d90081 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Jan 17 15:27:42 2023 +0600 Merge branch 'master' into issue24538-save-user-code commit a8502624ca74909e27e5f90508c0bea22619b382 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Dec 27 19:06:23 2022 +0600 getUnitSnippets in unit progress (#24538) commit 8dd70984cdb3aebd676c2d92f4508b3b0d6a7ae5 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 8 22:33:32 2022 +0600 user menu without padding commit 90b9e3d49adf6c075fbdeef881ee161b109fd1df Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 8 22:32:37 2022 +0600 save user code barely works commit 44172c57c2d806b6521fc966febabb949e3e77a7 Merge: d442e63668 59d0fd3129 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 8 14:15:37 2022 +0600 Merge branch 'issue24394-tobf-hint' into issue24538-tobf-save-user-code commit d442e63668b15330bd8a68fb18312f5cc06e5640 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Dec 7 15:22:57 2022 +0600 no duplicate slash with reg exp (#24538) commit e76860497a6983cd28e29f3d4065c9331ac32660 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Dec 6 10:11:12 2022 +0600 updated import (#24538) commit 63745d564ace082d34be5360cfd264cfaef6e86a Merge: d15913b289 af936d0b11 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Dec 6 10:02:57 2022 +0600 Merge branch 'issue23692_auth' into issue24538-tobf-save-user-code commit 59d0fd3129bcab3b55a66f671898ce6c5bcc84df Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 20:50:12 2022 +0600 solution managed inside of tour notifier (#24394) commit 5ca484661d330a0d7bea371a767f8d37402db520 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 15:57:30 2022 +0600 show solution snippets (#24394) commit 5e3ac5857e6fbe585e8f6b43cca75265b59e25c2 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 14:20:43 2022 +0600 renamed to unit content widget (#24394) commit 372b97fb5cd457125137c3ceb71332acecbc9117 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 12:01:39 2022 +0600 group progress indicator size adjustment (#24394) commit 48ea57d7c97a817458741d7062955add3b758c76 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 11:50:08 2022 +0600 title in unit content (#24394) commit 338c67317cdc9eec5a4046af9e5599737408baf2 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Dec 2 11:01:57 2022 +0600 extracted widgets, hints emptiness check (#24394) commit 36d3105f52fad0aa37ddaeaac340ee31a8a4e86b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 21:12:05 2022 +0600 hint in dialog (#24394) commit af936d0b115bf42f21310719dc1b18e580010421 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 14:59:52 2022 +0600 added firebase_options.dart into gradle rat exclusions & added a missing license (#23692) commit 9e1bbb081c575358445750694d27e393ebd72406 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 14:02:46 2022 +0600 specific imports (#23692) commit 9720c134b5733fb439ae2b4df12bff07c359a95b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 13:06:57 2022 +0600 moved dismissible overlay (#23692) commit 869b40cefa9222b5a61db0125f6c54340dfc918c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 13:06:40 2022 +0600 extracted overlay body (#23692) commit c8db1ef778f401a9029c113a784faeb94919d660 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 13:05:12 2022 +0600 refined open overlay (#23692) commit 8b639d08b3d6101ba9b13a6f6fe32b7cc7261559 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 12:50:35 2022 +0600 all caches extend cache (#23692) commit a10795205507a66b8cd90ed8caaf112663a60c8c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Dec 1 12:47:19 2022 +0600 renamed user progress cache to unit progress (#23692) commit e215ec4c676e88d85fad3ba08e222ac19d8953b8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Nov 30 10:37:05 2022 +0600 generated files commit 96e41f7d01c50e2cfa96fbc120fca69b3d29cb39 Merge: 1e38328e56 37fb90c98d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Nov 30 10:30:05 2022 +0600 Merge from master commit 1e38328e564f72b4e18439c60610902423223dbd Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 15:40:15 2022 +0600 removed "fix exception" todo after filing an issue (#23692) commit b6c4e3e66c30fa4cbdfbbefd1a41e649c32b9ecb Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:52:29 2022 +0600 added async (#23692) commit 27b26e41a81f07faac060c3d2e45f5b7bea19d5a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:31:24 2022 +0600 clearUpdatingUnitId (#23692) commit 4b2d855920d6d63dfff4412fc9bdfb9ddbbdd695 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:30:12 2022 +0600 renamed user progress model to unit progress (#23692) commit ef637403aa114f43f82ac005254632f2d35e0bad Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 13:09:58 2022 +0600 rearranged completeUnit (#23692) commit fb1fea1af8069ae02ba261781ca923cd40796c97 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 11:43:06 2022 +0600 comment fixes (4) commit 749d8b19bea072be84cace76bd1fb998a3a76531 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 11:03:34 2022 +0600 pubspec.lock commit 0a76b4019647ebcbec76739b078b01fddd37789b Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 29 11:03:17 2022 +0600 pubspec.lock ignored only in PGC commit c6004b0f220be0959b1fadc1fcffb04e4004cd97 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 21:17:16 2022 +0600 no final else commit d55622e30df308eb61a42ecd1f03930012ec0cb8 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 21:15:15 2022 +0600 missing await commit 65dc5219943a948e7922da44ce34b54474bffaf1 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 21:12:22 2022 +0600 updating & blocking complete unit button (#23692) commit 1bbc09279604dd2025ca6c93cfc85d2df9bece09 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 14:36:25 2022 +0600 comment fixes (3) commit 860780836bdf8cdbbcbe5952232abc3544effc9e Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 12:44:52 2022 +0600 pubspec.lock in gitignore commit a575717869b239f59374caf74f81db74191c3781 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 28 12:41:44 2022 +0600 deleted pubspec.lock commit c2f74a59ed6a364d04a9cb0d11ea0e1f0ce5ffbf Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sun Nov 27 21:05:49 2022 +0600 review comments (2) commit adbdeb6eec823178d487a3a3b3bba07ceda6efa9 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 15:15:57 2022 +0600 deleted file (#23692) commit 6ed38acb2beb4b960eb7a77da1282683e2805bc0 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 15:14:07 2022 +0600 unused import (#23692) commit 2c90ced90376df81d0989c233995320a73c7feb1 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 14:55:05 2022 +0600 login_overlay (#23692) commit 6d21e1ea3f0837616a185e4b1f0d48398799791d Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 14:52:08 2022 +0600 kOpenLoginOverlay (#23692) commit 64965a1e1f49f0240b4e27a387aad099caade1b2 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Thu Nov 24 14:44:19 2022 +0600 canCompleteCurrentUnit (#23692) commit 7fbaf3843394530602a87805fa274ace7620ae29 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Nov 23 15:21:56 2022 +0600 new configs & todos commit 0b6fc8f805d3417f819ebafadc40f2571d73df99 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 22 21:07:35 2022 +0600 candidate repository commit 3df27bc7fc71f12d48b3aa0bf5d58e57ba1327b9 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Tue Nov 22 15:12:05 2022 +0600 untested refinement (1) commit 52838e500fc106414dbb47d0c05fe4042c812509 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 21:54:53 2022 +0600 notifier objects naming commit 2068ce9e8da8b6caa41e1be09dfa04e26e2ca567 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 21:50:25 2022 +0600 deleted show sdk selector commit a6597d434a3aed640fc230976afd3637217d8792 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 21:44:55 2022 +0600 untested refinement commit 10b287b89456368b146fd9bc1edc38bf0c28fc73 Merge: 4aedd5a607 5ad067f7cc Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Mon Nov 21 19:06:30 2022 +0600 Merge pull request #324 from akvelon/tour-header-sdk-selector SDK selector in tour scaffold commit 5ad067f7ccdb976fcdb26102d983bea936a9ba83 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 17:55:38 2022 +0600 non-nullable onChanged commit 4aedd5a6072e92dbfd7487652619e7a14e291375 Merge: 6f53e30d39 70c0d61d3a Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 16:06:38 2022 +0600 SDK selector with fixed comments commit 6f53e30d39ba793850153a2a6077fbe6bbab2b51 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Mon Nov 21 16:04:31 2022 +0600 cache license commit 70c0d61d3a96e120d312f1aecc6a38e73e551697 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Sat Nov 19 20:46:18 2022 +0600 comments (0) commit de273baf06cc76e4da1f94064131f0093e265f0c Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 20:51:31 2022 +0600 hide back button after navigating from welcome commit 3a10e8d7633bb4d6d9bd39ba5515aacae6208554 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 20:48:13 2022 +0600 show sdk selector in welcome screen commit de0f7c34f2a686f22137c464d8815a5f6c2cb9c4 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 20:00:17 2022 +0600 sdk selection works with auth commit 3c534d864b5a36cf4f63b64af0332753fd420b9a Merge: 12a685d1f0 b78bbd33dc Author: Darkhan Nausharipov <nausharipov@gmail.com> Date: Fri Nov 18 19:41:06 2022 +0600 Merge remote-tracking branch 'origin/tour-header-sdk-selector' into auth_sdk_demo commit 12a685d1f00140981c8241a2477b5ee0707e47be Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Fri Nov 18 19:36:34 2022 +0600 unit file fix commit d25eaf52df834b655421319b14e43e0afc7c63f1 Merge: 5297d2548c cf56af2799 Author: Darkhan Nausharipov <nausharipov@gmail.com> Date: Fri Nov 18 19:22:22 2022 +0600 Merge branch 'master' into issue23692_auth commit 5297d2548ced6027c58b75b9d917fbfeb7052369 Author: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Date: Wed Oct 19 19:50:43 2022 +0600 auth, complete unit, user progress AuthNotifier draft (#23692) Comments (#23692) Comments (#23692)(1) sign in with google works (#23692) new configs (#23692) get user progress draft (#23692) comment fixes (#23692) sign in in IntroTextBody (#23692) reverted config (#23692) comment fixes (#23692) WIP before rebase (merge) (#23692) Squashed commit of the following: commit bff4919ff00ec3b5d7186efde41c884dfc4c8344 Merge: 79ba69483a ce8d618c77 Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Thu Nov 17 10:34:02 2022 +0100 Merge pull request #24186: Uses _all to follow alias/datastreams when estimating index size commit 79ba69483a84ea0278d0b0ddb141200739607c77 Merge: 245fea9040 b7e860a762 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 20:47:40 2022 -0800 Merge pull request #24218: Update Python wheel format for RC validation commit 245fea904014cd58d4148807463dbaa40000774c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Nov 16 18:12:33 2022 -0800 Bump loader-utils from 1.4.1 to 1.4.2 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (#24191) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit e1de8e78deeb5d17617fda6591429eaaf8abb8a2 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 20:48:06 2022 -0500 Fix PythonLint (#24219) commit b7e860a7621771c300dcec625655f87e62591323 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:28:31 2022 -0800 updates commit c2feb09ea49dd815b69c65e531ce34128756d988 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 17:06:08 2022 -0800 updates commit ce8d618c77d23e20a1ddb128bb8183048597d096 Author: egalpin <egalpin@users.noreply.github.com> Date: Wed Nov 16 16:43:57 2022 -0800 Adds test for following aliases when estimating index size commit 959719d01c627328c0ca2849d2b7e2c9b322d4d1 Author: Chamikara Jayalath <chamikara@apache.org> Date: Wed Nov 16 15:16:06 2022 -0800 Temporary update Python RC validation job commit b952b41788acc20edbe5b75b2196f30dbf8fdeb0 Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 14:18:12 2022 -0500 Python TextIO Performance Test (#23951) * Python TextIO Performance Test * Add filebasedio_perf_test module for unified test framework for Python file-based IOs * Fix MetricsReader publishes metrics duplicately if more than one load test declared. This is because MetricsReader.publishers was static class variable * Fix pylint * Distribute Python performance tests random time at a day instead of all at 3PM * Add information about length conversion commit 017f2cbde124af40a43be99ec88289fcf63c1c95 Merge: fef8acdbc0 88dba4f494 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Wed Nov 16 10:39:52 2022 -0800 Merge pull request #24187: Add a reference to Java RunInference example commit fef8acdbc0ecbcc85b49144adaf8830e3bc6b2de Merge: 6e9187e67e ead245539d Author: Ahmet Altay <aaltay@gmail.com> Date: Wed Nov 16 10:24:53 2022 -0800 Merge pull request #24199 from Laksh47/issue#24196 refs: issue-24196, fix broken hyperlink commit 6e9187e67e1bd8f73997f437f0ed4c29880ed73b Author: Darkhan Nausharipov <31556582+nausharipov@users.noreply.github.com> Date: Wed Nov 16 22:33:50 2022 +0600 [Tour of Beam] [Frontend] Content tree URLs (#23776) * Content tree navigation (#23593) Unit content navigation (#23593) Update URL on node click (#23593) Active unit color (#23593) removeListener in unit (#23593) First unit is opened on group title click (#23593) WIP by Alexey Inkin (#23593) selectedUnitColor (#23593) Unit borderRadius (#23593) RegExp todo (#23593) added referenced collection package to remove warning (#23593) small refinement (#23593) expand on group tap, padding, openNode (#23593) group expansion bug fix (#23593) selected & unselected progress indicators (#23593) * AnimatedBuilders instead of StatefulWidgets in unit & group (#23593) * fixed _getNodeAncestors (#23593) * get sdkId (#23593) * addressing comments (#23593) * sdkId getter & StatelessExpansionTile (#23593) * expand & collapse group (#23593) * StatelessExpansionTile (#23593) * license (#23593) * ValueChanged and ValueKey in StatelessExpansionTile (#23593) Co-authored-by: darkhan.nausharipov <darkhan.nausharipov@kzn.akvelon.com> Co-authored-by: Alexey Inkin <alexey.inkin@akvelon.com> commit b33fac2aa533d77cfa47f88466c8cd6bd3f3e864 Author: Bruno Volpato <bvolpato@google.com> Date: Wed Nov 16 10:51:11 2022 -0500 Use only ValueProviders in SpannerConfig (#24156) commit 5f013ab6567ec75b460b2081d7f89d332320caff Author: Robert Burke <lostluck@users.noreply.github.com> Date: Wed Nov 16 07:23:10 2022 -0800 revert upgrade to go 1.19 for action unit tests (#24189) commit 9337f4dbecc929886f8559949a082a649fd9d1bb Author: Yi Hu <yathu@google.com> Date: Wed Nov 16 10:18:42 2022 -0500 Fix Python PostCommit Example CustomPTransformIT on portable (#24159) * Fix Python PostCommit Examples on portable * Fix custom_ptransform pipeline options gets modified * Specify flinkConfDir commit ead245539d01dec0f3e08699c1e1cc6777a5ef0e Author: Laksh <lakshmanansathya@gmail.com> Date: Wed Nov 16 09:32:46 2022 -0500 refs: issue-24196, fix broken hyperlink commit e83a996d4374d467d95bcfad7166905622ec615c Merge: 2fc56ec663 ffdee0b6ed Author: Jan Lukavský <je.ik@seznam.cz> Date: Wed Nov 16 15:15:31 2022 +0100 Merge pull request #24192: Re-use serializable pipeline options when already available. commit ffdee0b6edb8638c78a65ec85c727ea5dde1cb2f Author: Jozef Vilcek <jvilcek@zetaglobal.com> Date: Mon Nov 14 16:48:18 2022 +0100 Re-use serializable pipeline options when already available (#24192) commit 88dba4f494829b2b3530b767fb8c5252e0d2ba44 Author: Chamikara Jayalath <chamikaramj@gmail.com> Date: Tue Nov 15 16:21:22 2022 -0800 Add a reference to Java RunInference example commit 2fc56ec663e335cfcf37dc57d471f79b601414f4 Merge: f763186987 83f1bc19b9 Author: Kenn Knowles <kenn@apache.org> Date: Tue Nov 15 16:16:47 2022 -0800 Merge pull request #24142: Fix arguments to checkState in BatchViewOverrides commit f763186987c00ba1d26efdc35406436a1fa69a9a Merge: c2bc2135e9 0d7ca04182 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 15:25:20 2022 -0800 Addresses #24161 Updated README of Interactive Beam commit c2bc2135e9bce715990a5d5551e2bc2dc0311da4 Author: Doug Judd <nuggetwheat@gmail.com> Date: Tue Nov 15 14:48:26 2022 -0800 Strip FGAC database role from changestreams metadata requests (#24177) Co-authored-by: Doug Judd <nuggetwheat@google.com> commit af637974f96ad1b5110d7dea3f9a26c68e19a51b Author: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Date: Tue Nov 15 17:16:43 2022 -0500 Add custom inference function support to the PyTorch model handler (#24062) * Initial type def and function signature * [Draft] Add custom inference fn support to Pytorch Model Handler * Formatting * Split out default * Remove Keyed version for testing * Move device optimization * Make default available for import, add to test classes * Remove incorrect default from keyed test * Keyed impl * Fix device arg * custom inference test * formatting * Add helpers to define custom inference functions using model methods * Trailing whitespace * Unit tests * Fix incorrect getattr syntax * Type typo * Fix docstring * Fix keyed helper, add basic generate route * Modify generate() to be different than forward() * formatting * Remove extra generate() def commit a014637106970a0a0e9eb7944aa5caf79fa5fd37 Author: egalpin <egalpin@users.noreply.github.com> Date: Tue Nov 15 13:57:54 2022 -0800 Uses _all to follow alias/datastreams when estimating index size Fixes #24117 commit 0d7ca041823bc2b09f76f86fdfd1d0b9508c9c88 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:57:27 2022 -0800 Minor update commit e8fc759d756f4a987e41d2b9da56b906a6cd7736 Author: Ning Kang <ningkang0957@gmail.com> Date: Tue Nov 15 13:52:18 2022 -0800 Updated README of Interactive Beam Removed deprecated cache_dir runner param in favor of the cache_root global option. commit 08d5f72e5f35d41f3e9fa9fe799caea6bed1b7a7 Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Tue Nov 15 16:34:21 2022 -0500 [Python]Support pipe operator as Union (PEP -604) (#24106) Fixes https://github.com/apache/beam/issues/21972 commit 526e7a58b62682582c27173ab21ed8667ddab766 Author: Scott Strong <scott.strong87@gmail.com> Date: Tue Nov 15 16:26:45 2022 -0500 Using Teardown context instead of deprecated finalize (#24180) * Using Teardown context instead of deprecated finalize * making function public Co-authored-by: Scott Strong <scott.strong@wunderkind.co> commit fb4d1d4dea7b26ed538a9f6aca0ed41e8c300e37 Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 16:25:22 2022 -0500 Fix broken json for notebook (#24183) commit f98db2008a97f4546d036ddf0dddfee8c87eb58a Author: Robert Burke <lostluck@users.noreply.github.com> Date: Tue Nov 15 12:49:23 2022 -0800 Update automation to use Go 1.19 (#24175) Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com> commit e5f58504eef1fdeebe0402cda8a2df259169c704 Author: Brian Hulette <bhulette@google.com> Date: Tue Nov 15 12:25:13 2022 -0800 Add error reporting for BatchConverter match failure (#24022) * add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else commit 3037747f66f0d71d65b6c65745b4f8942c22f05a Author: Danny McCormick <dannymccormick@google.com> Date: Tue Nov 15 14:13:04 2022 -0500 Fix broken notebook (#24179) commit b2b1c739ce37690923891934ee317f799db937a2 Author: MakarkinSAkvelon <67736809+MakarkinSAkvelon@users.noreply.github.com> Date: Tue Nov 15 21:53:06 2022 +0500 [Playground] Move Playground in GKE and Infrastructure change (#23928) * changes to updated master branch * Change workflow * ingress changes * Certificate was added * Updates for cloud build backend * Update main.tf * Create main.tf * Create variables.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Create output.tf * Update output.tf * Update output.tf * Update main.tf * Update build.gradle.kts * Update output.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update variables.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update main.tf * Update output.tf * Update main.tf * Update main.tf * Update output.tf * Create variables.tf * Update main.tf * Update main.tf * Delete playground/terraform/infrastructure/cluddns directory * Update main.tf * Update output.tf * Update output.tf * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Update README.md * Update README.md * helm folder name was changed * Update README.md * Update build.gradle.kts * Update build.gradle.kts * Update build.gradle.kts * Updates to readme * Fix DNS name * HelmChart was changed * Some workflows were changed * Remove unused file * playground-examples return * add license information * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * remove "stg" folder * Update README.md * Remove unused files * DNS Removed * var name changed * remove DNSName from var file * 1 * Clear terraform * remove unused records * gradle check * grade last change * issue fix * fix * 1 * run * test * Index creation for Gradle * Add IndexCreation in gradle * Update README.md * Update README.md * Fix names for Frontend * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Redis fix * services fix * Update variables.tf * change order in gradle * Fix Config.g.dart file issue * Update README.md * Playground workflow update Co-authored-by: Sergey Makarkin <sergey.makarkin@akvelon.com> Co-authored-by: Sergey Makarkin <sergey_makarkin@quicktest1.c.apache-beam-testing.internal> Co-authored-by: ruslan-ikhsan <ruslan.ikhsanov@akvelon.com> Co-authored-by: Alex Kosolapov <alex.kosolapov@gmail.com> commit 85df5f2eb2f299e28b36be0cce7b9c19d62124da Author: Yi Hu <yathu@google.com> Date: Tue Nov 15 11:38:13 2022 -0500 Eliminate CalciteUtil.CharType logical type (#24013) * Eliminate CalciteUtils.CharType logical type * Replace CalciteUtils.CharType to String Note that CalciteUtils still omits the precision of BINARY/VARBINARY/CHAR/VARCHAR as what it originally did. Support of the precision of these calcite types involves make use of making use of the overload method RelDataTypeFactory.createSqlType(var1, var2). * Replace every reference of CalciteUtil.CharType to generic PassThroughLogicalType check * Add TODO to Support sql types with arguments * Use VariableString in LogicalTypeTestCase commit f349f41010c5b238ff6020f7de718f938eef3c5e Author: alexeyinkin <alexey.inkin@akvelon.com> Date: Tue Nov 15 20:04:01 2022 +0400 Configure flutter_code_editor options with Hugo shortcode (#23926) (#24031) * Configure flutter_code_editor options with Hugo shortcode (#23926) * Minor fixes (#23926) * Refactor after review (#23926) commit 0f4ca6363b3ce0e5de3ad36517bb406aa6391a18 Author: Rebecca Szper <98840847+rszper@users.noreply.github.com> Date: Tue Nov 15 06:10:13 2022 -0800 Editorial review of the ML notebooks. (#24125) * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Editorial review of the ML notebooks. * Update examples/notebooks/beam-ml/custom_remote_inference.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback * Update examples/notebooks/beam-ml/run_inference_sklearn.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Danny McCormick <dannymccormick@google.com> * Updating based on feedback Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 5bd34ede026253326ebff1a7e4f9edb5f71b4a2c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Nov 15 07:17:28 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24131) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.3.2 to 1.11.39. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.3.2...feature/s3/manager/v1.11.39) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2ee809fa0ca7689dd0279e186ebc02d9569a8429 Merge: e3b9bdb2e6 563c66d6fd Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Tue Nov 15 11:01:14 2022 +0100 Merge pull request #23065: [Website] Update copy icon styles commit e3b9bdb2e607d85a4017ba7839000e92a0ad83c4 Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:40:50 2022 +0100 [Dockerized Jenkins] Fix build of dockerized jenkins (fixes #24053) (#24054) commit faaac2ab6e010374cb2be0e95a5dd345836a2a2c Author: Moritz Mack <mmack@talend.com> Date: Tue Nov 15 10:38:59 2022 +0100 [Dockerized Jenkins] Update README how to use local repo (#24055) commit 689e70b5131620540faf52e2f1e2dca7a36f269d Author: Damon <damondouglas@users.noreply.github.com> Date: Mon Nov 14 17:34:29 2022 -0800 Implement embedded WebAssembly example (#24081) commit e1bf6c42950e8013f35e35fb9fee8017e01e5010 Merge: eddac84126 10337d2868 Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 15:22:14 2022 -0800 Merge pull request #24160 Rename the test_splits flag to direct_test_splits. commit eddac841261228a2c63fa9b225c520ae0f853806 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 15:05:05 2022 -0800 More dataset templates to clean up (#24162) commit 2adb68bd12743566cc89b596bf204d7c807eb62d Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 13:28:13 2022 -0800 Adding a quickstart to README for the TS SDK (#23509) * More of a quickstart for the TS SDK * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> * Update sdks/typescript/README.md Co-authored-by: Danny McCormick <dannymccormick@google.com> Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 10337d28685ad5712e2ad8608977ec5c5e0e6b6b Author: Robert Bradshaw <robertwb@gmail.com> Date: Mon Nov 14 12:46:32 2022 -0800 Rename the test_splits flag to direct_test_splits. This avoids possible flag conflicts. commit 48c70cc30742b45b17a1d18ece2f0d079bee3915 Author: arne-alex <108519096+arne-alex@users.noreply.github.com> Date: Mon Nov 14 21:33:02 2022 +0100 Merge pull request #23333: Track time on Cloud Dataflow streaming data reads and export via heartbeats commit 9c83de646ab52bd0b05e3346190dd55cd68b2a8b Author: Johanna Öjeling <51084516+johannaojeling@users.noreply.github.com> Date: Mon Nov 14 21:19:44 2022 +0100 Add more tests for S3 filesystem (#24138) commit 9e9c6d797ba52b460f83131431c8e53aebbbc9ac Merge: d5d76b9745 c600444e1d Author: Ning Kang <ningkang0957@gmail.com> Date: Mon Nov 14 12:06:15 2022 -0800 Merge pull request #24029 from apache/dependabot/npm_and_yarn/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/loader-utils-1.4.1 Bump loader-utils from 1.4.0 to 1.4.1 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel commit d5d76b974592d45de368ab641647ca5cc4ec12ec Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 15:03:28 2022 -0500 Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils (#23969) * Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils * Add TODO about java.time migration commit 330cc2010c9f4a2d4e30318bf50a4109ec1cd392 Author: Pablo <pabloem@users.noreply.github.com> Date: Mon Nov 14 12:02:10 2022 -0800 Cleanup stale BQ datasets (#24158) * Cleanup stale BQ datasets * addressing comments commit 4a044999b8ed4bcd41f816f3a23ccb5da00c4c38 Merge: e563b9dd2f 5bd75c25de Author: Heejong Lee <heejong@gmail.com> Date: Mon Nov 14 11:16:00 2022 -0800 Merge pull request #24076 from chamikaramj/multilang_java_updates Updates Multi-lang Java quickstart commit e563b9dd2f3aa0484e6cdc08869991b5e438023e Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Mon Nov 14 20:56:35 2022 +0300 [Tour Of Beam] verify that unit exists when saving progress (#24118) * AIO * Update learning/tour-of-beam/backend/integration_tests/auth_test.go Co-authored-by: Danny McCormick <dannymccormick@google.com> * nit Co-authored-by: Danny McCormick <dannymccormick@google.com> commit 774923e0dd089de870bfa5c77063ae2b28f79347 Merge: 71785de528 1ad0cbc445 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 09:52:26 2022 -0800 Merge pull request #24141: Fix checkArgument format in GcsPath commit 71785de52864313c2e3b14fe72a2a63281343617 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 11:54:37 2022 -0500 Bump github.com/aws/aws-sdk-go-v2/config from 1.17.10 to 1.18.0 in /sdks (#24151) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.17.10 to 1.18.0. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.17.10...config/v1.18.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 50d591d6cb3e799bee4e29dfc593c693a86e6276 Author: Bruno Volpato <bvolpato@google.com> Date: Mon Nov 14 11:50:01 2022 -0500 Change DataflowBatchWorkerHarness doWork error level to INFO (#24135) commit 5a72696bfda09fdb905ba8e58b636f8494ef955f Merge: ee0a5836d6 0633fe9634 Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:12:12 2022 -0800 Merge pull request #24149: Remove extraneous jetbrains annotation commit ee0a5836d69b776834eb3bd9b2bd02eb5252c333 Merge: d001a69e1a 137799672e Author: Kenn Knowles <kenn@apache.org> Date: Mon Nov 14 08:11:00 2022 -0800 Merge pull request #24132: Fix checkArgument format string in AvroIO commit d001a69e1a58701d6ed4fcb5e3fb7a0921301dad Author: Yi Hu <yathu@google.com> Date: Mon Nov 14 10:56:54 2022 -0500 Test Dataproc 2.1 with Flink load tests (#24129) * Test Dataproc 2.1 with Flink load tests * Minor fix flink_cluster script commit caabd9be52887ad70c8a4269395c893811ac6a84 Author: Israel Herraiz <ihr@google.com> Date: Mon Nov 14 16:03:39 2022 +0100 Make MonotonicWatermarkEstimator work like its Java SDK equivalent (#24146) * Make MonotonicWatermarkEstimator work like its Java SDK equivalent The current implementation of MonotonicWatermarkEstimator raises an exception with late messages, which makes the watermark estimator barely usable in real world scenarios. This PR fixes #20041 by making this watermark estimator work like its Java SDK equivalent (`WatermarkEstimators.MonotonicallyIncreasing`). * Update unit tests too * Make linter happy commit 451f6b3e7f58d0a3782ad942c6a1fd9f63932024 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Nov 14 09:48:23 2022 -0500 Bump golang.org/x/net from 0.1.0 to 0.2.0 in /sdks (#24153) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.1.0 to 0.2.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/compare/v0.1.0...v0.2.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2bb03d62e2d7dc2d8e39040fc9adebccbde74fde Merge: 4e39ef2041 623083cd0a Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 15:01:13 2022 +0100 Merge pull request #24000: [Website] Change headers size from h4,h3 to h2 commit 563c66d6fd32165da14a07747f2764c17a5d24ea Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Wed Sep 7 18:28:42 2022 +0600 [Website] update pre tag copy link styles #23064 commit 4e39ef20410ee51c6040317bcd60171e64c5171f Merge: 223768f782 105ed6fedc Author: Alexey Romanenko <33895511+aromanenko-dev@users.noreply.github.com> Date: Mon Nov 14 10:55:33 2022 +0100 Merge pull request #24115: [Website] update go-dependencies.md java-dependencies.md links commit 223768f782f771f0033b8d0686d86cf4c71fad75 Merge: aa0a35dabf a9da2abee6 Author: Kenn Knowles <kenn@apache.org> Date: Sun Nov 13 18:53:13 2022 -0800 Merge pull request #24136: Fix checkArgument format string in ExecutionStateTracker commit 0633fe9634fe61df7cbc0ecac205d81124fd504a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 15:15:16 2022 -0800 Remove extraneous jetbrains annotation commit 83f1bc19b95935e60ca1f4027d4b60c7e738a84a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 14:16:09 2022 -0800 Fix arguments to checkState in BatchViewOverrides commit 1ad0cbc44594d8405bf4b07a126265238013a02a Author: Kenneth Knowles <klk@google.com> Date: Sat Nov 12 13:41:02 2022 -0800 Fix checkArgument format in GcsPath commit aa0a35dabf9c2a0d9822faff06d939d9a77a3ab6 Author: Kenn Knowles <kenn@apache.org> Date: Fri Nov 11 20:26:30 2022 -0800 Fix checkArgument format string in TestStream (#24134) commit a9da2abee6455bc2cf0f18ba5f6cd7bbaeae669f Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 16:54:27 2022 -0800 Fix checkArgument format string in ExecutionStateTracker commit 369e2ba8622d3474c14c39b941b2c618842d1e47 Author: Ryan Thompson <ryanthompson591@gmail.com> Date: Fri Nov 11 19:46:07 2022 -0500 Add a ValidatesContainer integration test for use_sibling_sdk_workers (#24099) commit 137799672eb559a7586262e6a8a73d1ab3580e44 Author: Kenneth Knowles <klk@google.com> Date: Fri Nov 11 15:30:01 2022 -0800 Fix checkArgument format string in AvroIO commit 5d2dbf957e4e82fb3980726940df02ac67e563cd Author: Anand Inguva <34158215+AnandInguva@users.noreply.github.com> Date: Fri Nov 11 15:57:28 2022 -0500 Update staging of Python wheels (#24114) Fixes https://github.com/apache/beam/issues/24110 commit c2021bee1eba0322b43c90841397859048296b21 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 15:33:14 2022 -0500 Bump google.golang.org/api from 0.102.0 to 0.103.0 in /sdks (#24049) Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.102.0 to 0.103.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.102.0...v0.103.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6557c91c79480b9d90573d52d257a11c2b160196 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 11:47:12 2022 -0800 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24112) Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.29.1 to 1.29.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.29.1...service/s3/v1.29.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 96f9da1ab652156cd143d57e3aa3d94836338f2b Author: Yi Hu <yathu@google.com> Date: Fri Nov 11 14:34:22 2022 -0500 More cleanup containers (#24105) * More cleanup containers * prebuilt_beam_sdk/beam_python_prebuilt_sdk no longer exists in gcr.io Add correct paths to cleanup * Bump grace time to 15 days * Run job daily as in code comment * Set grace period to 30 days commit 836766ddb83d37463e4b036f156b8d7e15e3864b Author: Ritesh Ghorse <riteshghorse@gmail.com> Date: Fri Nov 11 14:30:30 2022 -0500 upgrade testcontainer dependency (#24123) commit 9fcd20c3712536f2d4580beead678cdbb6fd4746 Author: Damon <damondouglas@users.noreply.github.com> Date: Fri Nov 11 11:12:11 2022 -0800 Implement PubsubRowToMessage transform (#23897) * Begin PubsubRowToMessage Impl * Complete working draft * Unit tests validate user and non-user fields * Finish tests on supporting methods * Pass checks before finalizing tests * WIP * fix timestamp * finalize tests * Finalize code comments * Clean up check findings * Add InputSchemaFactory * Patch code comment typo commit 3a6fcc1ca4d07a467464ed1214a94b5c9c147295 Author: Evgeny Antyshev <eantyshev@gmail.com> Date: Fri Nov 11 19:13:45 2022 +0300 disable (#24121) commit 027fb142038b45c443d87af96cac082264c43188 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Nov 11 10:26:23 2022 -0500 Bump cloud.google.com/go/bigtable from 1.17.0 to 1.18.0 in /sdks (#24113) Bumps [cloud.google.com/go/bigtable](https://github.com/googleapis/google-cloud-go) from 1.17.0 to 1.18.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.17.0...pubsub/v1.18.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/bigtable dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit b2d28a64bb0837dd97e8be7f97f31f88d380f110 Author: Danny McCormick <dannymccormick@google.com> Date: Fri Nov 11 08:45:14 2022 -0500 Add TFX support in pydoc (#23960) * Add TFX support in pydoc * Wording commit 105ed6fedcb6ff33d4a43940c342a653e67cb25e Author: bulat safiullin <bulat.safiullin@akvelon.com> Date: Thu Nov 10 17:47:54 2022 +0600 [Website] update go-dependencies.md java-dependencies.md programming-guide.md links #24084 commit 4b96193250a63b27721a8c5e1a3bd6ecff983093 Author: Brian Hulette <bhulette@google.com> Date: Thu Nov 10 16:51:06 2022 -0800 Remove TheNeuralBit from the pool of Python reviewers (#24108) commit 8f8f089a5f565f179905984cef3522a0577d9219 Author: Sanil Jain <snjain@linkedin.com> Date: Thu Nov 10 16:47:31 2022 -0800 Wire SamzaPipelineOptions to Exeption listener interface (#24109) commit b3186ba91f9bb22764d47b78eb2b7ff017080f75 Author: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> Date: Thu Nov 10 18:04:50 2022 -0500 Support using BigQueryIO Storage Read API with SchemaTransforms (#23827) * support schema transform for bq direct read method * use vendor Strings import * add BigQueryServices argument to config object * suppress nu…

add error reporting for BatchConverters

180ff0c

github-actions bot added the python label Nov 7, 2022

TheNeuralBit added 2 commits November 8, 2022 16:11

Test pytorch

d5d1fa8

Finish up torch tests

80e1efe

TheNeuralBit changed the title ~~[WIP] Add error reporting for BatchConverters~~ Add error reporting for BatchConverter match failure Nov 9, 2022

TheNeuralBit added 2 commits November 8, 2022 16:31

yapf

cf9e856

yapf

1a2096b

github-actions bot added the Next Action: Reviewers label Nov 9, 2022

robertwb approved these changes Nov 14, 2022

View reviewed changes

sdks/python/apache_beam/typehints/batch.py Outdated Show resolved Hide resolved

Remove else

4118956

TheNeuralBit merged commit e5f5850 into apache:master Nov 15, 2022

This was referenced Nov 17, 2022

Teach Azure Filesystem to authenticate using DefaultAzureCredential in the Python SDK #24212

Merged

Fix PythonLint #24219

Merged

andreigurau pushed a commit to andreigurau/beam that referenced this pull request Nov 17, 2022

Add error reporting for BatchConverter match failure (apache#24022)

2c7242c

* add error reporting for BatchConverters * Test pytorch * Finish up torch tests * yapf * yapf * Remove else

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add error reporting for BatchConverter match failure #24022

Add error reporting for BatchConverter match failure #24022

TheNeuralBit commented Nov 7, 2022 •

edited

Loading

codecov bot commented Nov 7, 2022 •

edited

Loading

TheNeuralBit commented Nov 9, 2022

github-actions bot commented Nov 9, 2022

robertwb left a comment

TheNeuralBit commented Nov 15, 2022

Add error reporting for BatchConverter match failure #24022

Add error reporting for BatchConverter match failure #24022

Conversation

TheNeuralBit commented Nov 7, 2022 • edited Loading

GitHub Actions Tests Status (on master branch)

codecov bot commented Nov 7, 2022 • edited Loading

Codecov Report

TheNeuralBit commented Nov 9, 2022

github-actions bot commented Nov 9, 2022

robertwb left a comment

Choose a reason for hiding this comment

TheNeuralBit commented Nov 15, 2022

TheNeuralBit commented Nov 7, 2022 •

edited

Loading

codecov bot commented Nov 7, 2022 •

edited

Loading