• Created https://github.com/ipfs/js-kubo-rpc-client.
• Used https://support.github.com/contact?tags=rr-forks&subject=Detach%20Fork&flow=detach_fork to detach the forked repo from ipfs/js-ipfs

Trying to generate documentation, but I can't seem to do so and can't find instructions at https://github.com/ipfs/js-ipfs

It seems like aegir doesn't support generating docs anymore (aegir docs command no longer exists), and they used to use typedoc, so I'm going to use https://github.com/TypeStrong/typedoc directly

Action is running right now: https://github.com/ipfs/js-kubo-rpc-client/actions/runs/2886171949

Updated the action, ipfs/js-kubo-rpc-client@cc2823b, and the site is now visible at https://ipfs.github.io/js-kubo-rpc-client/

Created PR to add js-kubo-rpc-client to ipfs/github-mgmt#43

Using personal NPM_TOKEN for publish js-kubo-rpc-client to npm automatically with github action via aegir until I get access to npm service account to create more permanent NPM_TOKEN

Unified CI should be enabled for js-kubo-rpc-client once protocol/.github#371 and ipfs/github-mgmt#43 are merged

Updated NPM_TOKEN to use automated token from service account

update https://github.com/ipfs/js-ipfsd-ctl to support both ipfsHttpModule and kuboRpcModule
(we will also rename binary package, but that is tracked separately in ipfs/npm-kubo#51)

Starting work on this now.

published https://www.npmjs.com/package/js-kubo-rpc-client so I can bring the lib into https://github.com/ipfs/js-ipfsd-ctl

decide if js-kubo-rpc-client CI should reuse tests against Core APIs interface-ipfs-core

@lidel @BigLep @aschmahmann, who should I talk to about this? Which stakeholders make the final call for this item?

Created milestone for tracking each task: https://github.com/ipfs/js-kubo-rpc-client/milestone/1

decide if js-kubo-rpc-client CI should reuse tests against Core APIs interface-ipfs-core

@lidel @BigLep @aschmahmann, who should I talk to about this? Which stakeholders make the final call for this item?

@SgtPooki : We need to make sure that kubo (RPC server) and the js-kupo-rpc-client can interopoerate and that we don't break compatibility between them. We should take whatever tests we had in the past to bootstrap our efforts, but we no longer need to adhere to the "Core API" concept. Basically, I would write tests that invoke the js-kubo-rpc client directly, rather than through legacy "Core API" interfaces.

So it sounds like Kubo RPC Client should have it's own tests, stripped out from interface-ipfs-core. And we should only bring over the tests that are applicable for kubo's supported RPC calls.

I can make sense of that I think. Thanks

@SgtPooki and @lidel : question on ipfs/interop when this work is done. Will ipfs/interop assert that a kubo node and a js-ipfs node can continue to interact with each other (e.g., discover each other, transport data)? Will the way that ipfs/interop accomplish this by orchestrating the kubo node using the js-kubo-rpc-client and the js-ipfs node using ipfs-http-client?

I think for now, we want to do a surgical swap: drop-in replace of ipfs-http-client with kubo-rpc-client when the backend node is Kubo, but all existing tests remain and should pass.

Seems we are back to the underlying question if both ipfs-http-client and kubo-rpc-client should conform to the same JS Interface (Core API) or not.

My understanding is:

• Iroh does not implement Kubo RPC at all, and there are no plans to do that.
  • The only interface other implementations use for benchmarking is HTTP Gateway.
• One of the reasons why we were asked to rename go-ipfs to Kubo was to make it clear that RPC at /api/v0/ is Kubo-specific, and other implementations are free to go their own way, create own APIs.
• Until now, Core API in JS-IPFS was forced to conform to whatever Kubo legacy existed.
  • Creation of kubo-rpc-client aims at removing this co-dependency, allowing Kubo RPC and JS-IPFS RPC to go their own ways.
  • Someone will ask: if these are not distinct creatures, then why did we create kubo-rpc-client in the first place?

At the same time we have years of legacy code that assumes both projects expose the same API.
Issue at hand with js-ipfsd-ctl is only a symptom of a problem that we will see over and over again.
Sadly, this is one of the reasons why Stewards tried to postpone the split and rename for so long – it is a ton of work and bad DX for preexisting users.

Some thoughts on minimizing the pain

Accounting for the legacy we live with, and being pragmatic (Ideally, we don't want to spend the rest of 2022 on this), I see three ways going forward, would appreciate sanity check from @achingbrain: how feasible (and painful) below are?

1. Make kubo-rpc-client conform to Core API types from JS-IPFS.
   • We admit defeat for now, and keep them joined at a hip, but the roles are reversed and it is JS-IPFS wilggling the kubo-rpc-client from now.
   • Main win is that nobody has to refactor 999 projects (incl. dragon lairs like ipfs/interop)
2. Create distinct types for kubo-rpc-client but keep using ipfsd-ctl
   • This may be the least painful way to do things, complexity and refactor-wise.
   • Migration plan: js-ipfsd-ctl exposes existing JS API as api but when node type is go, kubo-rpc-client is used for talking to Kubo, and api is replaced with a stubs that throw error asking people to use api from kuboRpc field instead.
     • Good: there is no silent breakage caused by api having different behavior when Kubo is used: this ensures GO people are forced to refactor their code to switch to kuboRpc field and Kubo types ( api field's types remain as Core API).
     • Bad: every internal app that uses ipfsd-ctl with Kubo daemon will have to be refactored (incl. dragon lairs like ipfs/interop)
3. Create kubod-ctl
   • Same as 2, but with more work: adds maintenance burden of a separate package.
   • We could update ipfsd-ctl to throw error when go type is used, pointing people at this new package

Thanks for the comments here @lidel. A couple of thoughts.

1. Please huddle to sync synchronously if it would help for getting decisions made. Maybe the js-ipfs triage time could be used (or afterwards since that is early for @SgtPooki ).
2. This probably goes without saying, but just so it's clear what I care about, I don't care as much about having a JS Kubo RPC client that is named as such (although I do think it's ideal for users to have a name that's clear). My chief concern is allowing the Kubo team to own their RPC client without any considerations for js-ipfs and vice versa. Decoupling is the key. If in the process, it's just as easy to rename, then great - please do it. If renaming anything is causing pain, please feel free to skip. I want to optimize for future developer velocity more so than user clarity.

In js-ipfs triage we have come up with a 4th option:

The neat trick of the Core API is that it's supported by js-ipfs in-process and both js-ipfs and Kubo over http. We have leveraged this to write the interop suite which is a) massively useful in proving the two implementations can talk to each other and b) about to cause significant type-related pain now that we want to allow the RPC APIs to diverge.

The 4th option is to just type the parts of the API that the interop suite uses, store those types in the interop suite and have it use them instead of the types from the Core API.

The theory is that both kubo-rpc-client and ipfs-http-client will have overlap with these types so tsc should be happy with them, and then both implementations (just Kubo really since js-ipfs is getting No New Features:tm:) are free to innovate in other areas of the API.

When Kubo breaks the API so badly that the interop suite types are no longer valid, hopefully we'll have the shiny new version of IPFS-on-js that will have it's own interop suite that maybe orchestrates kubo via some sort of API external to it a la go-libp2p-daemon's simple protobuf-based API and doesn't depend on the RPC client at all.

type the parts of the API that the interop suite uses, store those types in the interop suite and have it use them instead of the types from the Core API.

Excellent. The interop work living in the interop package makes the most sense and should keep us all unblocked for the longest period. Type augmentations should help keep the diverging types manageable.

FYI I made a "CI test that fails when a new command is added to Kubo and is not supported by kubo-rpc-client yet" as followup that isn' required for closing this issue since we didn't have that with the ipfs-http-client.

Note that the latest work to close out the last action item in this issue (ipfs/js-kubo-rpc-client#5) is being handled via pull request ipfs/js-kubo-rpc-client#83.

I'll be meeting with @hacdias tomorrow (2022-12-06) to go over some of these tests and confirm next steps and handoff requirements.

Note that the original interface tests were testing against go-ipfs@0.12, and so the done criteria for closing this issue does not include updating interface tests to 0.17.

ninja-edit: There are technically two last items as seen on the milestone tracking the closing of this issue: https://github.com/ipfs/js-kubo-rpc-client/milestone/1, but the "revert patch-only release changes" is trivial so was not mentioned in the original comment.

Latest actions required prior to handoff: ipfs/js-kubo-rpc-client#83 (comment)

Note that ipfs/js-kubo-rpc-client#83 has been merged!

We should be good to close this. will defer to @BigLep and @lidel

Thanks @SgtPooki for bringing this to this point. The done criteria are satisfied, thus I think it's reasonable to close this out. I know there are additional tasks to do to further harden js-kubo-rpc-client and those are in ipfs/js-kubo-rpc-client#56

Closing - thanks again!

With this issue closed, should we point all users of https://github.com/ipfs/js-ipfs/tree/master/packages/ipfs-http-client to this?

The NPM page could also use a notice: https://www.npmjs.com/package/ipfs-http-client

ipfs-http-client is still used to connect to js-ipfs, so not all users, but it could use a message saying to use the kubo rpc client with kubo.