[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pay: Remember and update channel_hints across payments #7494

Open
wants to merge 22 commits into
base: master
Choose a base branch
from

Conversation

cdecker
Copy link
Member
@cdecker cdecker commented Jul 25, 2024

Builds on top of #7487.

This PR takes the channel_hints that were previously created,
updated and then discarded as part of a payment, and instead we are
making them long-lived. This means that we share what we learn from
one payment with all later payments, allowing us to skip attempts that
will likely fail anyway, and improve the overall performance
(time-to-completion as well as success rate of payments).

The downside of this is that we are now keeping observations for
prolonged periods, meaning that these observations are no longer valid
when we use them. If used unchanged these earlier observations could
lead us to skip routes that we deem unavailable due to a prior
observation, but which is now back to being usable, which in turn
impacts our ability to complete a payment successfully. The core
contribution in this PR therefore is the time-based relaxation of
learnt constraints. It is based on a leaky-bucket approach, with a
linear refill-rate for depleted channels, such that after 2h we
consider any observation outdated.

Example

If at time t we observe a channel c with cap_c as overall
capacity, and bal_c = 0.5 * cap_c the currently available balance
(i.e., we observed a payment of bal_c + 1 fail to generate this
hint), then we have a refill rate r = cap_c / 7200, and after 1h the
channel is considered to be fully available again, i.e., we don't have
a more restrictive observation, and it is old enough that it does not
add to the structural information we already have from the gossip.

graph TD;
    CLN7487 --> CLN7494 ;
    click CLN7487 "https://github.com/ElementsProject/lightning/pull/7487"
    click CLN7494 "https://github.com/ElementsProject/lightning/pull/7494"
    style CLN7494 fill:#AAFFAA
Loading

@cdecker cdecker force-pushed the 202407-pay-channel-hint-update branch 7 times, most recently from 92b6cc1 to 594f806 Compare July 31, 2024 11:18
@cdecker cdecker force-pushed the 202407-pay-channel-hint-update branch 9 times, most recently from c676551 to 5259236 Compare August 7, 2024 11:59
@cdecker cdecker marked this pull request as ready for review August 7, 2024 15:49
@cdecker cdecker added this to the v24.08 milestone Aug 7, 2024
common/route.h Outdated Show resolved Hide resolved
common/route.c Outdated Show resolved Hide resolved
lightningd/onchain_control.c Outdated Show resolved Hide resolved
@rustyrussell rustyrussell force-pushed the 202407-pay-channel-hint-update branch from 5259236 to acc063d Compare August 9, 2024 05:45
@rustyrussell
Copy link
Contributor

I rebased, removed the final commit (which was from a different PR, and already merged) and added a commit with my suggested changes.

But I'm not sure this PR is complete?

plugins/channel_hint.c Outdated Show resolved Hide resolved
@cdecker cdecker force-pushed the 202407-pay-channel-hint-update branch 3 times, most recently from dc74fd8 to 24437b4 Compare August 9, 2024 10:44
@cdecker cdecker force-pushed the 202407-pay-channel-hint-update branch from 24437b4 to 9fc9e9b Compare August 9, 2024 10:50
@cdecker cdecker marked this pull request as draft August 9, 2024 12:10
@cdecker cdecker force-pushed the 202407-pay-channel-hint-update branch 3 times, most recently from e63e3f9 to 954eea3 Compare August 23, 2024 14:20
@cdecker cdecker marked this pull request as ready for review September 6, 2024 10:55
@cdecker cdecker force-pushed the 202407-pay-channel-hint-update branch from c362f12 to 6b068c5 Compare September 6, 2024 15:29
We relax constraints from the `channel_hint` through a linear refill.
We need to know the overall channel capacity, i.e., the amount_msat
that the channel was funded with, in order to relax the channel_hint
to refill over time.
We attach the hints to the plugin, so they get shared across multiple
payments.
We're getting serious about how we manage the channel_hints, so let's
give them a proper home.
Keeping it in `amount_msat` made the comparisons easier, but it was
the wrong type for this.
If we have a large channel, fail to send through a small amount, and
then add a `channel_hint`, then it can happen that the call to
`channel_hint_set_update` is already late enough that it refills the
1msat we removed from the attempted amount, thus making the edge we
just excluded eligible again, which can lead into an infinite loop.

We slow down the updating of the channel_hints to once every
hysteresis timeout. This allows us to set tight constraints, while not
incurring in the accidental relaxation issue.
Making sure that we don't accidentally create an endless loop.
We were using `channel_hint` to temporarily tweak the graph inside of
a payment. However, these ad-hoc `channel_hints` are stickier than
their predecessors, in that they outlive the payment attempt itself,
and interfere with later ones.

Changelog-Changed: pay: Discarding an overly long or expensive route does not blocklist channels anymore.
I'm not sure how this ever worked, and it must've been racy: we
initiate a number of payments, then proceed to disconnect and
reconnect repeatedly. If a payment is attempted inbetween the
`disconnect` and `connect` the payment will fail. Removing it since it
most likely was just a sledgehammer test that we didn't know how to
fine-tune.

Changelog-None
It was failing because the channel_hint from one attempt would prevent
us from retrying. By changing the amounts so that the channel_hints do
not concern them (value smaller than estimate) we can make things work
as before again.
We were ignoring more reliable information because of the
stricter-is-better logic. This meant that we were also ignoring local
channel information, despite knowing better.

By simplifying the logic to pick the one with the newer timestamp we
ensure that later observations always trump earlier ones. There is a
bit of interplay between the relaxation of the constraints updating
the timestamp, and comparing to real observation timestamps, but that
should not impact us for local channels.
@cdecker cdecker force-pushed the 202407-pay-channel-hint-update branch from 6b068c5 to 1e833ba Compare September 6, 2024 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants