Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sources not setting Event::transactional #870

Merged
merged 9 commits into from
Mar 25, 2021
Merged

Conversation

mfelsche
Copy link
Member

@mfelsche mfelsche commented Mar 24, 2021

Pull request

Description

  • Fix sources not setting Event::transactional
  • Ensure sinks only emit ack/fail insights when the event is transactional, so we spare some processing if it is not needed.
  • Fix kafka onramp origin_url not containing the partition id, although the docs state it should.

Related

Checklist

  • The RFC, if required, has been submitted and approved
  • Any user-facing impact of the changes is reflected in docs.tremor.rs
  • The code is tested
  • Use of unsafe code is reasoned about in a comment
  • Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
  • The performance impact of the change is measured (see below)

Performance

roughly equivalent.

@mfelsche mfelsche added offramp Offramps operator Pipeline Operators labels Mar 24, 2021
@mfelsche mfelsche self-assigned this Mar 24, 2021
Matthias Wahl added 3 commits March 24, 2021 20:50
…nal().

Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
@coveralls
Copy link
Collaborator

coveralls commented Mar 24, 2021

Coverage Status

Coverage increased (+0.3%) to 82.181% when pulling 5f5b695 on transactionality into 19cf123 on main.

@Licenser
Copy link
Member

grafik
I love it!!! 🚀 👍

@darach
Copy link
Member

darach commented Mar 24, 2021 via email

@Licenser
Copy link
Member

All jokes aside, I don't see any code that would explain a 20% perf improvement, any idea how it improved?

@darach
Copy link
Member

darach commented Mar 24, 2021 via email

@Licenser
Copy link
Member

20mb/s in those numbers so 7.8%?

Ja too late for math, but I don't see then 7.8% either so I'm curious :D

@mfelsche
Copy link
Member Author

Seems to be local fluctuation due to whatever background noise through other processes on my machine. Also got a run on my branch with 255MB/s. Gonna do some more runs.

@mfelsche
Copy link
Member Author

Jep, all noise, consider the throughput roughly the same for this benchmark.

Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
@mfelsche mfelsche marked this pull request as ready for review March 24, 2021 23:28
Matthias Wahl added 3 commits March 25, 2021 00:34
Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
@mfelsche mfelsche mentioned this pull request Mar 25, 2021
6 tasks
Matthias Wahl added 2 commits March 25, 2021 13:27
Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Copy link
Member

@Licenser Licenser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@mfelsche mfelsche merged commit 2c6f0da into main Mar 25, 2021
@mfelsche mfelsche deleted the transactionality branch March 25, 2021 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
offramp Offramps operator Pipeline Operators
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sources do not set field Event::transactional
4 participants