Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract provider module to go-ipfs-provider #6421

Merged
merged 4 commits into from
Jul 3, 2019
Merged

Extract provider module to go-ipfs-provider #6421

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
20c8c85
Add go-ipfs-provider as dependency
michaelavila Jul 3, 2019
40af13e
Remove existing provider system (will fail)
michaelavila Jul 3, 2019
10de165
Change to construct provider from go-ipfs-provider
michaelavila Jul 3, 2019
72fef7f
Update to provider with timestamp based queue
michaelavila Jul 3, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion core/core.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ import (
ipnsrp "github.com/ipfs/go-ipfs/namesys/republisher"
"github.com/ipfs/go-ipfs/p2p"
"github.com/ipfs/go-ipfs/pin"
"github.com/ipfs/go-ipfs/provider"
"github.com/ipfs/go-ipfs/repo"

bserv "github.com/ipfs/go-blockservice"
bstore "github.com/ipfs/go-ipfs-blockstore"
exchange "github.com/ipfs/go-ipfs-exchange-interface"
"github.com/ipfs/go-ipfs-provider"
ipld "github.com/ipfs/go-ipld-format"
logging "github.com/ipfs/go-log"
mfs "github.com/ipfs/go-mfs"
Expand Down
2 changes: 1 addition & 1 deletion core/coreapi/coreapi.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ import (
"github.com/ipfs/go-ipfs/core/node"
"github.com/ipfs/go-ipfs/namesys"
"github.com/ipfs/go-ipfs/pin"
"github.com/ipfs/go-ipfs/provider"
"github.com/ipfs/go-ipfs/repo"

bserv "github.com/ipfs/go-blockservice"
"github.com/ipfs/go-ipfs-blockstore"
"github.com/ipfs/go-ipfs-exchange-interface"
offlinexch "github.com/ipfs/go-ipfs-exchange-offline"
"github.com/ipfs/go-ipfs-provider"
offlineroute "github.com/ipfs/go-ipfs-routing/offline"
ipld "github.com/ipfs/go-ipld-format"
logging "github.com/ipfs/go-log"
Expand Down
19 changes: 14 additions & 5 deletions core/node/provider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,13 @@ import (
"time"

"github.com/ipfs/go-ipfs/core/node/helpers"
"github.com/ipfs/go-ipfs/provider"
q "github.com/ipfs/go-ipfs/provider/queue"
"github.com/ipfs/go-ipfs/provider/simple"
"github.com/ipfs/go-ipfs/pin"
"github.com/ipfs/go-ipfs/repo"

"github.com/ipfs/go-ipfs-provider"
q "github.com/ipfs/go-ipfs-provider/queue"
"github.com/ipfs/go-ipfs-provider/simple"
ipld "github.com/ipfs/go-ipld-format"
"github.com/libp2p/go-libp2p-core/routing"
"go.uber.org/fx"
)
Expand Down Expand Up @@ -101,9 +104,9 @@ func SimpleProviders(reprovideStrategy string, reprovideInterval string) fx.Opti
case "":
keyProvider = fx.Provide(simple.NewBlockstoreProvider)
case "roots":
keyProvider = fx.Provide(simple.NewPinnedProvider(true))
keyProvider = fx.Provide(pinnedProviderStrategy(true))
case "pinned":
keyProvider = fx.Provide(simple.NewPinnedProvider(false))
keyProvider = fx.Provide(pinnedProviderStrategy(false))
default:
return fx.Error(fmt.Errorf("unknown reprovider strategy '%s'", reprovideStrategy))
}
Expand All @@ -115,3 +118,9 @@ func SimpleProviders(reprovideStrategy string, reprovideInterval string) fx.Opti
fx.Provide(SimpleReprovider(reproviderInterval)),
)
}

func pinnedProviderStrategy(onlyRoots bool) interface{} {
return func(pinner pin.Pinner, dag ipld.DAGService) simple.KeyChanFunc {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@magik6k @Kubuxu Any thoughts on a preferred alternative to this?

The reason for this is that the "old" reprovider depends on pin.Pinner which is a module within go-ipfs. To break that dependency I introduced a Pinner type in the go-ipfs-provider repository, but that is not the type of Pinner that's in the dependency injection container. To get around this I do the constructing of the dependency here, within go-ipfs, so that I can depend on pin.Pinner. The benefit, as I see it, is that this change is relatively small.

We could move the Pinner out as well into its own repository, but I really don't want to take on that large of a change right now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to leave the NewPInnedProvider thing as a provider implementation within go-ipfs and extract the more generalistic ones (namely BlockstoreProvider). This looks very weird right now, and I don't need the PinnedProvider anyways.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Offline @magik6k said he's ok with this. Ultimately, he has some work in progress in #6387 that will change how all of this works for the better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hsanjuan I think that makes sense. Let me try it. And now that I think about it, ipfs-lite maybe doesn't have the concept of a Pinner, is that true?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's true, no Pinner. Pinner is a go-ipfs specific concept (for now)

return simple.NewPinnedProvider(onlyRoots, pinner, dag)
}
}
2 changes: 2 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ require (
github.com/ipfs/go-ipfs-exchange-offline v0.0.1
github.com/ipfs/go-ipfs-files v0.0.3
github.com/ipfs/go-ipfs-posinfo v0.0.1
github.com/ipfs/go-ipfs-provider v0.1.1
github.com/ipfs/go-ipfs-routing v0.1.0
github.com/ipfs/go-ipfs-util v0.0.1
github.com/ipfs/go-ipld-cbor v0.0.2
Expand Down Expand Up @@ -112,6 +113,7 @@ require (
go4.org v0.0.0-20190313082347-94abd6928b1d // indirect
golang.org/x/sync v0.0.0-20190423024810-112230192c58 // indirect
golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb
google.golang.org/appengine v1.4.0 // indirect
gopkg.in/cheggaaa/pb.v1 v1.0.28
gotest.tools/gotestsum v0.3.4
)
Expand Down
4 changes: 4 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,10 @@ github.com/ipfs/go-ipfs-posinfo v0.0.1 h1:Esoxj+1JgSjX0+ylc0hUmJCOv6V2vFoZiETLR6
github.com/ipfs/go-ipfs-posinfo v0.0.1/go.mod h1:SwyeVP+jCwiDu0C313l/8jg6ZxM0qqtlt2a0vILTc1A=
github.com/ipfs/go-ipfs-pq v0.0.1 h1:zgUotX8dcAB/w/HidJh1zzc1yFq6Vm8J7T2F4itj/RU=
github.com/ipfs/go-ipfs-pq v0.0.1/go.mod h1:LWIqQpqfRG3fNc5XsnIhz/wQ2XXGyugQwls7BgUmUfY=
github.com/ipfs/go-ipfs-provider v0.1.0 h1:lYSVVxWpL0KJw1PLj3+DAn0zuVfc+z93wzUXS09ZjZk=
github.com/ipfs/go-ipfs-provider v0.1.0/go.mod h1:gzVZZXC4zhr2r+MkNR21/+FS54oc7VfTKtDT2mdDxD8=
github.com/ipfs/go-ipfs-provider v0.1.1 h1:nsC6oWr6bDJ4H7pZfZJqAk6oXaHsrqnwhpQIqTdSDic=
github.com/ipfs/go-ipfs-provider v0.1.1/go.mod h1:gzVZZXC4zhr2r+MkNR21/+FS54oc7VfTKtDT2mdDxD8=
github.com/ipfs/go-ipfs-routing v0.0.1/go.mod h1:k76lf20iKFxQTjcJokbPM9iBXVXVZhcOwc360N4nuKs=
github.com/ipfs/go-ipfs-routing v0.1.0 h1:gAJTT1cEeeLj6/DlLX6t+NxD9fQe2ymTO6qWRDI/HQQ=
github.com/ipfs/go-ipfs-routing v0.1.0/go.mod h1:hYoUkJLyAUKhF58tysKpids8RNDPO42BVMgK5dNsoqY=
Expand Down
28 changes: 0 additions & 28 deletions provider/offline.go
View file
Open in desktop

This file was deleted.

26 changes: 0 additions & 26 deletions provider/provider.go
View file
Open in desktop

This file was deleted.

148 changes: 0 additions & 148 deletions provider/queue/queue.go
View file
Open in desktop

This file was deleted.

Loading