Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the custom implementation of buffered input stream reader #36673

Merged
merged 1 commit into from
Apr 2, 2024

Conversation

jbfbell
Copy link
Contributor

@jbfbell jbfbell commented Mar 29, 2024

This removes our custom message reader and uses a standard one. It also will log every 1000 empty lines until a max of 10000 is reached and report the number of empty lines at the end.

Copy link

vercel bot commented Mar 29, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 2, 2024 5:42pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label Mar 29, 2024
Copy link
Contributor Author

jbfbell commented Mar 29, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jbfbell and the rest of your teammates on Graphite Graphite

@jbfbell jbfbell changed the title bump version Remove the custom implementation of buffered input stream reader Mar 29, 2024
@jbfbell jbfbell marked this pull request as ready for review March 29, 2024 00:34
@jbfbell jbfbell requested review from a team as code owners March 29, 2024 00:34
@jbfbell jbfbell force-pushed the jbfbell/use_standard_stream_reader branch 3 times, most recently from 3e6985a to cbd5205 Compare March 29, 2024 18:16
@jbfbell jbfbell requested review from a team and stephane-airbyte March 29, 2024 19:00
Copy link
Contributor

@evantahler evantahler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay! Using the same reader the platform uses, e.g. the built-in java one, seems like a good way to not be dropping data

baos.write(b.toInt())
lastWasNewLine = false
emptyLines++
// We've occasionally seen this loop not exit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're not adding any loop, so I don't think we're making the problem worse. And once every 1k lines should not be a problem. If it is, it's a problem on the side that's generating those empty lines IMO

@jbfbell jbfbell force-pushed the jbfbell/use_standard_stream_reader branch from cbd5205 to 624ccae Compare April 2, 2024 17:41
@jbfbell jbfbell merged commit 0d4adbb into master Apr 2, 2024
27 checks passed
@jbfbell jbfbell deleted the jbfbell/use_standard_stream_reader branch April 2, 2024 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants