[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

migrate to datastore-backed peerstore #6080

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

migrate to datastore-backed peerstore #6080

wants to merge 1 commit into from

Conversation

raulk
Copy link
Member
@raulk raulk commented Mar 14, 2019 •
edited

The datastore-backed libp2p peerstore is now production ready. Let's roll it out.

Could someone enumerate a few tests in go-ipfs land that would make intensive use of the peerstore? I'd like to benchmark memory and gc before we merge.

EDIT: do we want to make the peerstore choice and options configurable?

@raulk raulk requested review from Stebalien and Kubuxu March 14, 2019 10:10
@ghost ghost assigned raulk Mar 14, 2019
@ghost ghost added the status/in-progress In progress label Mar 14, 2019
@Stebalien
Copy link
Member
Stebalien commented Mar 14, 2019 •
edited by raulk

🎉 Awesome! 🎉

  1. How will this interact with the default non-block datastore, leveldb? IIRC, most testing was against badger. Answer: Just fine (on my machine, that is).
  2. This is going to write a lot of data (over time). We may need to look into GC: Data may remain after garbage collecting everything go-ds-badger#54. However, badger isn't the default so this may be fine.
  3. We definitely need to fix Periodically garbage collect go-ds-badger#51 first.

For now, we should probably add an experimental option to the config (https://github.com/ipfs/go-ipfs-config/blob/master/experiments.go).

Concretely:

@raulk
Copy link
Member Author
raulk commented Mar 14, 2019

Makes sense, @Stebalien. Will look into those when I get some spare cycles. Thanks for the quick follow-up!

@raulk raulk mentioned this pull request Apr 8, 2019
Closed
@raulk
support experimental datastore-backed peerstore.
286941d 
License: MIT
Signed-off-by: Raúl Kripalani <raul@protocol.ai>
@raulk raulk mentioned this pull request Apr 14, 2019
Closed
@whyrusleeping
Copy link
Member

@raulk i'm getting a panic:

14:55:18.225 ERROR peerstore/: failed to update ttls for peer QmfDqERcEewXnrM8pD2QCsw9XrGQ8KerHPgG66y1A9fFvC: open /home/whyrusleeping/.ipfs/datastore/1597823.ldb: too many open files
 addr_book.go:266
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xfb79de]

goroutine 7754831 [running]:
github.com/libp2p/go-libp2p-peerstore/pstoreds.(*dsAddrBook).UpdateAddrs(0xc00012a200, 0xc00ecba180, 0x22, 0x7ffffffffffffffe, 0x8bb2c97000)
        /home/whyrusleeping/gopkg/pkg/mod/github.com/libp2p/go-libp2p-peerstore@v0.0.1/pstoreds/addr_book.go:269 +0x7e
github.com/libp2p/go-libp2p/p2p/protocol/identify.(*netNotifiee).Disconnected(0xc000126690, 0x1a26760, 0xc00010c000, 0x1a20860, 0xc008d07ec0)
        /home/whyrusleeping/gopkg/pkg/mod/github.com/libp2p/go-libp2p@v0.0.5/p2p/protocol/identify/id.go:457 +0x145
github.com/libp2p/go-libp2p-swarm.(*Conn).doClose.func1.1(0x1a16a80, 0xc000126690)
        /home/whyrusleeping/gopkg/pkg/mod/github.com/libp2p/go-libp2p-swarm@v0.0.1/swarm_conn.go:75 +0x5b
github.com/libp2p/go-libp2p-swarm.(*Swarm).notifyAll.func1(0xc002c649d4, 0xc01c20be00, 0x1a16a80, 0xc000126690)
        /home/whyrusleeping/gopkg/pkg/mod/github.com/libp2p/go-libp2p-swarm@v0.0.1/swarm.go:448 +0x60
created by github.com/libp2p/go-libp2p-swarm.(*Swarm).notifyAll
        /home/whyrusleeping/gopkg/pkg/mod/github.com/libp2p/go-libp2p-swarm@v0.0.1/swarm.go:446 +0x119

@raulk raulk mentioned this pull request Apr 16, 2019
Merged
@whyrusleeping
Copy link
Member

@raulk could you rebase this on latest master? There are a few conflicts and i'd like to continue testing with some changes that landed in master.

@Stebalien Stebalien mentioned this pull request Apr 23, 2019
Closed
@momack2 momack2 added this to In Progress in ipfs/go-ipfs May 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Stebalien Stebalien Awaiting requested review from Stebalien

@Kubuxu Kubuxu Awaiting requested review from Kubuxu

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
status/in-progress In progress
Projects
No open projects
ipfs/go-ipfs
In Progress
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@raulk @Stebalien @whyrusleeping