Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OBSDATA-3156] Apply Confluent Patches on top of Druid 28.0.1 #171

Merged
merged 115 commits into from
Dec 18, 2023

Conversation

pagrawal10
Copy link

@pagrawal10 pagrawal10 commented Nov 28, 2023

Apply Confluent patches on top of Druid 28.0.1 https://confluentinc.atlassian.net/wiki/spaces/OBSERVE/pages/1506182651/Druid+Patches

Druid 28 release notes can be found here

Description

Fixed the bug ...

Renamed the class ...

Added a forbidden-apis entry ...

Release note

For tips about how to write a good release note, see Release notes.


Key changed/added classes in this PR
  • MyFoo
  • OurBar
  • TheirBaz

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Travis Thompson and others added 30 commits November 23, 2023 16:02
The extension packaging included both shaded and unshaded dependencies
in the classpath. Shading should not be necessary in this case.

Also excludes guava dependencies, which are already provided by Druid
and don't need to be added to the extensions jars.
* METRICS-1302: Added prefix support for resource labels.

* Addressed review comments.

* Added and moved configs to ingestion spec, optimized code.

* Addressed review comments

* Updated metric dimesnion and other review comments

* Flipped ternary operator

* Moved from NullHandling to StringUtils.

* Removed unnecessary HashMap.

* Removed verbosity for instance variables.
* Added getters for configs, labels for distribution metric.

* Addressed review comments

* Removed extra brackets in JsonProperty.
Align protobuf dependencies to use the main pom one
- fix millisecond resolution being dropped when converting timestamps
- remove unnecessary conversion of ByteBuffer to ByteString
- make test code a little more concise
- remove the need to parse timestamps into their own column
- reduce the number of times we copy maps of labels
- pre-size hashmaps and arrays when possible
- use loops instead of streams in critical sections

Combined these changes improve parsing performance by about 15%
- added benchmark for reference
…tFormat (#26)

InputRowParsers have been deprecated in favor or InputFormat.
This implements the InputFormat version of the OpenCensus Protobuf
parser, and deprecates the existing InputRowParser implementation.

- the existing InputRowParser behavior is unchanged.
- the InputFormat behaves like the InputRowParser, except for the
  default resource prefix which now defaults to "resource." instead of
  empty.
- both implementations internally delegate to OpenCensusProtobufReader,
  which is covered by the existing InputRowParser tests.
pom.xml Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
@pagrawal10 pagrawal10 requested a review from xvrl December 12, 2023 14:55
@pagrawal10 pagrawal10 changed the base branch from 28.0.0-confluent to 28.0.1-confluent December 14, 2023 06:46
@pagrawal10 pagrawal10 changed the title [OBSDATA-3156] Apply Confluent Patches on top of Druid 28 [OBSDATA-3156] Apply Confluent Patches on top of Druid 28.0.1 Dec 14, 2023
@@ -29,6 +30,8 @@
public interface RowFunction
{
Object eval(Row row);

List<String> evalDimension(Row row);
default List<String> evalDimension(Row row)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a test. And push the test upstream

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be cleaner to add the test for this upstream and once that PR is merged, cherry-pick it in a separate PR. Otherwise, there might be some changes suggested in the test and would require me to keep updating it in both places.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please create the test upstream then

@pagrawal10 pagrawal10 merged commit c5397d0 into 28.0.1-confluent Dec 18, 2023
3 checks passed
@pagrawal10 pagrawal10 deleted the OBSDATA-3156-druid-28-with-cflt-patches branch December 18, 2023 06:06
pagrawal10 added a commit that referenced this pull request Dec 18, 2023
@pagrawal10 pagrawal10 restored the OBSDATA-3156-druid-28-with-cflt-patches branch December 18, 2023 07:04
@pagrawal10 pagrawal10 deleted the OBSDATA-3156-druid-28-with-cflt-patches branch December 19, 2023 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.