[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Re-enable interface tests and get them succeeding #83

Merged

Conversation

SgtPooki
Copy link
Member
@SgtPooki SgtPooki commented Nov 23, 2022

Some tests are disabled (temporarily) (19 of them) and will be looked at later with @hacdias, prior to handing project off to Kubo team.

prioritized action items from talk with @hacdias

@SgtPooki SgtPooki linked an issue Nov 23, 2022 that may be closed by this pull request
17 tasks
Comment on lines 61 to 62
expect(peer).to.have.a.property('muxer')
expect(peer).to.have.a.property('streams')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems like at some point, these properties were not supposed to be returned if verbose: true was not passed, however, they are being returned.

Also, we probably shouldn't be asserting the lack of a property.. only the existence of properties.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When verbose is not true, Kubo does not return values for them (JSON keys are present, but with empty/default value):

$ ipfs swarm peers --enc=json | jq
{
      "Addr": "/ip4/97.115.80.237/udp/57908/quic",
      "Peer": "12D3KooWMMnfUUJBzZEA6rv6NyN6QUdJaRk62bc9zECXJNmKYPNP",
      "Latency": "",
      "Muxer": "",
      "Direction": 0,
      "Streams": null
    }

so what you want to test here is to ensure they have these empty values when verbose is not passed.

@SgtPooki SgtPooki force-pushed the 56-investigate-and-document-broken-interface-ipfs-core-tests branch from 7e8c039 to e370e56 Compare November 29, 2022 18:35
@SgtPooki SgtPooki marked this pull request as ready for review November 29, 2022 18:37
@SgtPooki SgtPooki self-assigned this Nov 29, 2022
Copy link
Member Author
@SgtPooki SgtPooki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ready for review @hacdias. 19 total tests skipped if I counted things correctly.

test/interface-tests.spec.js Outdated Show resolved Hide resolved
test/interface-tests.spec.js Outdated Show resolved Hide resolved
test/interface-tests.spec.js Outdated Show resolved Hide resolved
test/interface-tests.spec.js Outdated Show resolved Hide resolved
test/interface-tests.spec.js Outdated Show resolved Hide resolved
test/interface-tests.spec.js Outdated Show resolved Hide resolved
test/interface-tests.spec.js Outdated Show resolved Hide resolved
@SgtPooki SgtPooki changed the title [WIP] feat: Re-enable interface tests and get them succeeding feat: Re-enable interface tests and get them succeeding Nov 29, 2022
package.json Outdated Show resolved Hide resolved
Copy link
Member
@hacdias hacdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, it looks good to me. It's also important to mention that I'm not extremely used to this codebase. Besides the other inlined questions, I would like to understand why there are so many MFS-related tests that were removed (except for mode, and mtime)?

@SgtPooki
Copy link
Member Author

Overall, it looks good to me. It's also important to mention that I'm not extremely used to this codebase. Besides the other inlined questions, I would like to understand why there are so many MFS-related tests that were removed (except for mode, and mtime)?

@hacdias this codebase is a complete beast. I think it might be worth getting @achingbrain or @lidel to take a peek, but one major callout is that not all RPC APIs listed at https://docs.ipfs.tech/reference/kubo/rpc/#getting-started are implemented, and some additional tests that are not related to kubo specific RPC calls were supported. i.e. things that were in ipfs-http-client as 'js' or 'proc' but not in kubo.

regarding MFS related tests being removed, the thing to look for is whether the tests being removed were marked as "unsupported" by go-ipfs/kubo in https://github.com/ipfs/js-ipfs/blob/74aee8b3d78f233c3199a3e9a6c0ac628a31a433/packages/ipfs/test/interface-http-go.js. I attempted to leave things that looked like they should be supported when comparing to https://docs.ipfs.tech/reference/kubo/rpc/#getting-started, but there are a ton of tests and I may have missed something.

If some of these MFS tests should be supported by Kubo/go-ipfs, I can re-add them and leave them skipped? this is part of the struggle with pulling ipfs-http-client out. We either need to

  1. strip things and then rebuild the tests (what I recommend now)
  2. or keep the old tests and fix them (what I have been recommending)

After fixing many of these tests, I would highly recommend, and personally prefer #1. The existing tests contain a lot of very odd things that get tests to work in a non-deterministic way and coming at this from an angle of "we need to test X functionality that isn't currently tested" should allow for a much better test than from the angle of "this old test needs modified/updated."

@BigLep
Copy link
Contributor
BigLep commented Dec 13, 2022

General comment: my understanding is that https://docs.ipfs.tech/reference/kubo/rpc/ is the source of truth of Kubo RPC APIs that we should ideally have tested. (This is because it's generated from the Kubo itself.)

Assuming that is the source of truth, I would expect we can disable any HTTP calls in js-kubo-rpc-client not pertaining to that surface area.

Any places where the Kubo RPC API is not covered by previous (ipfs-http-client) code can be left uncovered for now (and ideally have backlog items).

Using MFS specifically, it's probably worth tracing out:

  1. What HTTP paths are those tests hitting?
  2. Why does that not show up in https://docs.ipfs.tech/reference/kubo/rpc/ ? Is it it intentional? If that isn't a supported API, then I think it's safe to remove the test.

But yes, I know there is a lot more intricacy here than I'm aware of, and it may be helpful if @lidel has any other color.

@hacdias
Copy link
Member
hacdias commented Dec 14, 2022

I agree that someone else should also take a look before we merge, since it's such a massive PR.

Regarding the tests, I would also go with option 1.

Copy link
Member
@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @SgtPooki and everyone involved, this was a Herculean effort ❤️

Overall lgtm, removals make sense: now that we are Kubo-only, we dont need to duplicate go test and sharness, all we need to test here is if HTTP RPC executes the same way CLI would.

I left some comments / questions, but nothing blocking – feel free to extract any folllow-up tasks related to my comments to separate issues (or as a checkbox in #56)

return {
server,
echoServer,
pinningService,
env: {
NODE_OPTIONS: '--no-experimental-fetch',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: when we run Node with this, does it impact the runtime of tests, or is it run in a separate process?

(if it is the same process, it is a problem: we should be running tests with default Node LTS and no special parameters)

Copy link
Member Author
@SgtPooki SgtPooki Dec 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I fully understand your question, but I'll give some context around this:

  1. nodeJS 18 (not being tested currently for node tests in ipfs-http-client, so scope creep for this task) adopted native fetch, see https://nodejs.org/en/blog/announcements/v18-release-announce/#fetch-experimental.
    • This was previously under an opt-in flag --experimental-fetch, but in 18 was switched to opt-out with --no-experimental-fetch. Basically, any discrepancies between node-fetch and undici's fetch will need to be handled if we don't override global fetch in node with node-fetch.
    • I have a hunch that some of these discrepancies were one of the problems I had with unblocking the test/node/agent.js test.
  2. When we run tests, the flow is:

I think item 2 above tells us the answer to your question is that this is a separate process.

steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
node-version: lts/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️

This project is managed via github.com/protocol/.github (config).

Does this mean the changes to this file from this PR will be overwritten by https://github.com/protocol/.github/blob/master/templates/.github/workflows/js-test-and-release.yml the next time CI sync runs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@galargh can you confirm?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's going to get overwritten. Whenever https://github.com/protocol/.github/actions/workflows/dispatch.yml runs, it updates uCI files in all the repositories where it's configured (if needed of course). That's why files managed by uCI have this header:

# File managed by web3-bot. DO NOT EDIT.
# See https://github.com/protocol/.github/ for details.

Here's an automated PR that wants to update the workflow here: #85 (it would have been automerged if only test-node on windows passed).

If there's a change we want to introduce that would benefit all JS repos participating in uCI, then this would be the place to do it: https://github.com/protocol/.github/blob/master/templates/.github/workflows/js-test-and-release.yml

Side note: the way Unified CI provisions files is different from how GitHub Management does it. For example, if we were to modify https://github.com/ipfs/js-kubo-rpc-client/blob/master/.github/workflows/stale.yml which is managed here https://github.com/ipfs/github-mgmt/blob/eed32757b57354659c07d738d36ecfbc25a033ef/github/ipfs.yml#L4589-L4591, then the next sync would have noticed that the two diverged and would update the content in the YAML config.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if a project needs to disable node LTS or other node versions for some reason, what is the best way to accomplish that?

Can we make the unified CI workflow dynamic with a default to LTS? I'm not sure the best way to accomplish this but it would allow individual projects to update/override when necessary. Without being overwritten.

src/bitswap/index.js Outdated Show resolved Hide resolved
src/bootstrap/index.js Outdated Show resolved Hide resolved
src/bootstrap/index.js Outdated Show resolved Hide resolved
test/interface-tests/src/dag/import.js Outdated Show resolved Hide resolved
test/interface-tests/src/files/cp.js Show resolved Hide resolved
Comment on lines 61 to 62
expect(peer).to.have.a.property('muxer')
expect(peer).to.have.a.property('streams')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When verbose is not true, Kubo does not return values for them (JSON keys are present, but with empty/default value):

$ ipfs swarm peers --enc=json | jq
{
      "Addr": "/ip4/97.115.80.237/udp/57908/quic",
      "Peer": "12D3KooWMMnfUUJBzZEA6rv6NyN6QUdJaRk62bc9zECXJNmKYPNP",
      "Latency": "",
      "Muxer": "",
      "Direction": 0,
      "Streams": null
    }

so what you want to test here is to ensure they have these empty values when verbose is not passed.

test/node/agent.js Show resolved Hide resolved
test/node/request-api.js Show resolved Hide resolved
@SgtPooki SgtPooki merged commit 2819080 into master Dec 15, 2022
@SgtPooki SgtPooki deleted the 56-investigate-and-document-broken-interface-ipfs-core-tests branch December 15, 2022 22:38
@SgtPooki SgtPooki removed the status/in-progress In progress label Dec 15, 2022
github-actions bot pushed a commit that referenced this pull request Dec 15, 2022
## [2.0.0](v1.0.3...v2.0.0) (2022-12-15)

### ⚠ BREAKING CHANGES

* Re-enable interface tests and get them succeeding (#83)

### Features

* Re-enable interface tests and get them succeeding ([#83](#83)) ([2819080](2819080)), closes [/github.com/ipfs/js-ipfs/blob/6ae5eb7/packages/ipfs/package.json#L93](https://github.com/ipfs//github.com/ipfs/js-ipfs/blob/6ae5eb7/packages/ipfs/package.json/issues/L93) [/github.com//pull/83#issuecomment-1347820355](https://github.com/ipfs//github.com/ipfs/js-kubo-rpc-client/pull/83/issues/issuecomment-1347820355)

### Trivial Changes

* move interface tests back to original name ([56c5c28](56c5c28))
@github-actions
Copy link
Contributor

🎉 This PR is included in version 2.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Pull out relevant tests from interface-ipfs-core
5 participants