Local file-based name resolver with SQLite + Update components-contrib to register Blob Storage v2 #7038

ItalyPaleAle · 2023-10-13T23:05:38Z

This PR updates components-contrib to the latest commit, so it also requires registering the new Azure Blob Storage v2 components

Addresses a longstanding request (> 2 years!) to have an alternative name resolver component for local development, for situations where mDNS can't be used (and even in those situations, a local name resolver is more convenient for local development).

As discussed in dapr/components-contrib#1380, this is implemented with SQLite, which is fast and efficient.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

go.mod

codecov · 2023-10-13T23:27:21Z

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (90835fa) 65.01% compared to head (44033e5) 65.07%.

❗ Current head 44033e5 differs from pull request most recent head 437cde1. Consider uploading reports for the commit 437cde1 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7038      +/-   ##
==========================================
+ Coverage   65.01%   65.07%   +0.05%     
==========================================
  Files         221      231      +10     
  Lines       20799    21060     +261     
==========================================
+ Hits        13523    13704     +181     
- Misses       6138     6214      +76     
- Partials     1138     1142       +4

Files	Coverage Δ
pkg/messaging/direct_messaging.go	`55.63% <0.00%> (ø)`
pkg/runtime/runtime.go	`73.18% <86.36%> (+0.03%)`	⬆️

... and 23 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

JoshVanL

One thing, otherwise LGTM 👍

Could you add a simple service invocation integration test using SQLite?

pkg/messaging/direct_messaging.go

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle · 2023-11-01T23:21:11Z

The PR in contrib was merged so this is good to go!! 🚀

Could you add a simple service invocation integration test using SQLite?

No, we should not use tests in the runtime to validate a component. There are some “ITs” for this component in components-contrib (implemented as unit tests because we don’t have an infra to do otherwise). We should add an infra in components-contrib to test nameresolvers, also for the other nameresolvers that don’t have a test (Consul)

ItalyPaleAle · 2023-11-01T23:32:33Z

pkg/runtime/runtime.go

-		nr.AppPort:      strconv.Itoa(a.runtimeConfig.appConnectionConfig.Port),
-		nr.HostAddress:  a.hostAddress,
-		nr.AppID:        a.runtimeConfig.id,
+	resolverMetadata.Instance = nr.Instance{


A small API change in contrib. Now properties such as address and port are passed in a strongly-typed struct rather than in a weakly-typed map.

go.mod

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Addressed

yaron2 · 2023-11-02T05:27:10Z

Integration tests seem to be consistently failing. I reran the tests 3 times. Are there known issues in master branch with integration tests? cc @JoshVanL

ItalyPaleAle · 2023-11-02T16:33:06Z

Integration tests seem to be consistently failing. I reran the tests 3 times. Are there known issues in master branch with integration tests? cc @JoshVanL

Looks like the test that's failing is the reminders rebalancing, and it's failing consistently in this PR and also when I run it locally. It appears the issue is that placement table dissemination doesn't happen in time.

It's really weird as this PR doesn't touch anything related to placement 😮

I am investigating

ItalyPaleAle · 2023-11-02T18:17:42Z

The issue seems to be an unrelated bug introduced by a separate commit in components-contrib. I'm going to fix or revert that commit.

--EDIT Fixed

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

…b to register Blob Storage v2 (dapr#7038) * Added support for SQLite nameresolver Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Added temp replacement Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Use a strongly-typed Instance struct for name resolvers Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Removed mod replacement Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Some test fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Updated pinned contrib Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Updated pinned components-contrib again Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Mod tidy Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> --------- Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Cassandra Coyle <cassie@diagrid.io>

ItalyPaleAle added 2 commits October 13, 2023 22:22

Added support for SQLite nameresolver

0f04b62

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Added temp replacement

42a7462

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle requested review from a team as code owners October 13, 2023 23:05

ItalyPaleAle commented Oct 13, 2023

View reviewed changes

go.mod Outdated Show resolved Hide resolved

ItalyPaleAle mentioned this pull request Oct 13, 2023

[Proposal] Provide alternative service resolution mechanism besides mdns #3256

Closed

JoshVanL requested changes Oct 14, 2023

View reviewed changes

pkg/messaging/direct_messaging.go Show resolved Hide resolved

Use a strongly-typed Instance struct for name resolvers

7538aab

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle marked this pull request as draft October 17, 2023 15:40

ItalyPaleAle marked this pull request as ready for review November 1, 2023 23:21

Merge branch 'master' into sqlite-nr

048ec56

ItalyPaleAle commented Nov 1, 2023

View reviewed changes

yaron2 previously requested changes Nov 2, 2023

View reviewed changes

go.mod Outdated Show resolved Hide resolved

Removed mod replacement

44033e5

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle added 5 commits November 2, 2023 21:37

Some test fixes

eff2d3d

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Updated pinned contrib

b1de04a

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Merge branch 'master' of https://github.com/dapr/dapr into sqlite-nr

0d54a7a

Updated pinned components-contrib again

63b3ddc

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Merge branch 'master' of https://github.com/dapr/dapr into sqlite-nr

6d73bde

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

ItalyPaleAle changed the title ~~Local file-based name resolver with SQLite~~ Local file-based name resolver with SQLite + Update components-contrib to register Blob Storage v2 Nov 3, 2023

ItalyPaleAle added 2 commits November 6, 2023 08:48

Merge branch 'master' into sqlite-nr

1e81d4a

Mod tidy

437cde1

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

yaron2 approved these changes Nov 6, 2023

View reviewed changes

yaron2 merged commit c2cbcd0 into dapr:master Nov 6, 2023

hhunter-ms mentioned this pull request Dec 20, 2023

Local file-based name resolver w SQLite dapr/docs#3914

Closed

ItalyPaleAle added this to the v1.13 milestone Jan 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Local file-based name resolver with SQLite + Update components-contrib to register Blob Storage v2 #7038

Local file-based name resolver with SQLite + Update components-contrib to register Blob Storage v2 #7038

ItalyPaleAle commented Oct 13, 2023 •

edited

Loading

codecov bot commented Oct 13, 2023 •

edited

Loading

JoshVanL left a comment •

edited

Loading

ItalyPaleAle commented Nov 1, 2023

ItalyPaleAle Nov 1, 2023

yaron2 commented Nov 2, 2023

ItalyPaleAle commented Nov 2, 2023

ItalyPaleAle commented Nov 2, 2023 •

edited

Loading

Local file-based name resolver with SQLite + Update components-contrib to register Blob Storage v2 #7038

Local file-based name resolver with SQLite + Update components-contrib to register Blob Storage v2 #7038

Conversation

ItalyPaleAle commented Oct 13, 2023 • edited Loading

codecov bot commented Oct 13, 2023 • edited Loading

Codecov Report

JoshVanL left a comment • edited Loading

Choose a reason for hiding this comment

ItalyPaleAle commented Nov 1, 2023

ItalyPaleAle Nov 1, 2023

Choose a reason for hiding this comment

yaron2 commented Nov 2, 2023

ItalyPaleAle commented Nov 2, 2023

ItalyPaleAle commented Nov 2, 2023 • edited Loading

ItalyPaleAle commented Oct 13, 2023 •

edited

Loading

codecov bot commented Oct 13, 2023 •

edited

Loading

JoshVanL left a comment •

edited

Loading

ItalyPaleAle commented Nov 2, 2023 •

edited

Loading