[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Staking] Currency <> Fungible migration #5501

Open
wants to merge 135 commits into
base: master
Choose a base branch
from

Conversation

Ank4n
Copy link
Contributor
@Ank4n Ank4n commented Aug 27, 2024

Migrate staking currency from traits::LockableCurrency to traits::fungible::holds.

Resolves part of #226.

Summary of changes

  • Config: Currency becomes of type Fungible while OldCurrency is the LockableCurrency used before.
  • Lazy migration of accounts. Any ledger updates will create a new hold with no extra reads/writes. A permissionless extrinsic migrate_currency() releases the old lock along with some housekeeping.
  • Staking now adds a provider instead of a consumer. More details later.
  • If hold cannot be applied to all stake, the un-holdable part is force withdrawn from the ledger.
  • Reap stash might fail now if provider cannot be decremented.

Migration stats

Polkadot

Total accounts that can be migrated: 61752
Accounts failing to migrate: 1
Accounts with some stake force withdrawn: 35
Total force withdrawal: 16287.7 DOT

Kusama

Total accounts that can be migrated: 26835
Accounts failing to migrate: 0
Accounts with some stake force withdrawn: 59
Total force withdrawal: 14.5 KSM

Full logs here. Error in polkadot is caused due to the ledger corruption, will be fixed by this runtime update. It is possible some of the force withdraws are happening because of ledger corruption and the migration stats might change after those ledgers are fixed.

Note about locks (freeze) vs holds

With locks or freezes, staking could use total balance of an account. But with holds, the account needs to be left with at least Existential Deposit in free balance. This would also affect nomination pools which till now has been able to stake all funds contributed to it. An alternate version of this PR is #5658 where staking pallet does not add any provider, but means pools and delegated-staking pallet has to provide for these accounts and makes the end to end logic (of provider and consumer ref) lot less intuitive and prone to bug.

One way to handle this issue is add a provider to staking stash. This then replicates old behaviour of staking where accounts could stake their whole balance. The side effect of this is that, for stakers that have been slashed to zero, their stash may not be reaped until all consumers to this account is removed. This is because, dec_provider would fail until consumer == 0.

Note about providers and consumers

Before

pallet-staking added consumers for stash account. This meant the agent/pool account must need a provider (by holding ED or system inc provider). We did an explicit inc provider for agents in pallet-delegated staking to allow consumer inc.

Now

This is more simplified. pallet-staking adds provider for stash accounts, but no consumer. pallet-delegated-staking or pools does not need to add provider anymore (or consumer), and as stated before, helps retain the old behaviour of being able to stake all balance via both direct staking or pool.

TODO

  • Permissionless call to release lock.
  • Migration of consumer (dec) and provider (inc) for direct stakers.
  • force unstake if hold cannot be applied to all stake.
  • Fix try state checks (it thinks nothing is staked for unmigrated ledgers).
  • Bench migrate_currency.
  • Virtual Staker migration test.
  • Ensure total issuance is upto date when minting rewards.

Followup

@Ank4n Ank4n changed the base branch from master to ankan/staking-migrate-currency-to-fungible August 27, 2024 12:20
@command-bot
Copy link
command-bot bot commented Oct 8, 2024

@Ank4n https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7538682 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh". Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 28-d6cf8ab9-4fc7-44c1-9ffb-9a6716c33a8c to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link
command-bot bot commented Oct 8, 2024

@Ank4n Command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh" has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7538682 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7538682/artifacts/download.

@Ank4n Ank4n requested a review from muharem October 22, 2024 11:10
@@ -262,7 +262,8 @@ impl<T: Config, Staking: StakingInterface<Balance = BalanceOf<T>, AccountId = T:
pool_account: Pool<Self::AccountId>,
_: Member<Self::AccountId>,
) -> BalanceOf<T> {
T::Currency::balance(&pool_account.0).saturating_sub(Self::active_stake(pool_account))
// free/liquid balance of the pool account.
T::Currency::balance(&pool_account.0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This number does not change for any pool given the README info. Correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the number does not change.

Longer explanation: Previously, staked (locked) funds were part of free balance. So we calculated transferable balance as free balance - staked balance.

Now, holds are taken away from free part of the balance, so transferable balance = free balance.

@@ -131,6 +131,7 @@ impl onchain::Config for OnChainSeqPhragmen {

#[derive_impl(pallet_staking::config_preludes::TestDefaultConfig)]
impl pallet_staking::Config for Test {
type OldCurrency = Balances;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you add this to TestDefaultConfig and be done with it? And later when you remove it nothing has to change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it have a default? Since its a runtime pallet (depends on system config)?

@@ -797,8 +795,6 @@ impl<T: Config> Pallet<T> {
Self::do_remove_validator(&stash);
Self::do_remove_nominator(&stash);

frame_system::Pallet::<T>::dec_consumers(&stash);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect assets::kill_stake to be called here, shouldn't it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StakingLedger::<T>::kill(&stash) calls that.

}

ensure!(!Self::is_virtual_staker(stash), Error::<T>::VirtualStakerNotAllowed);
let ledger = Self::ledger(Stash(stash.clone()))?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good future refactor is to remove the need for clone in Stash. It should not be needed. Makes a good junior mentor issue.

ensure!(ledger.total == locked, Error::<T>::BadState);

let max_hold = asset::stakeable_balance::<T>(&stash);
let force_withdraw = if max_hold >= locked {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that the StakingLedger is updated via two different methods in if and else is not ideal IMO.

I think a nicely written version would look like:

let mut ledger = todo!(); 
if {
    ledger.total = todo!("this")
} else {
    ledger.total = todo!("that")
}
ledger.update()?;

but yeah, I am not super familiar with these interfaces.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw, this allows us to use something similar to the Rust dropping mechanism for StakingLedger via let mut ledger. Once this is dropped (via fn update() taking in self), storage is updated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My nites aside, the code looks clean and easy to get :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ledger is mutated only in else block? Or do I misunderstand?

@@ -2087,7 +2121,7 @@ pub mod pallet {
let new_total = if let Some(total) = maybe_total {
let new_total = total.min(stash_balance);
// enforce lock == ledger.amount.
asset::update_stake::<T>(&stash, new_total);
asset::update_stake::<T>(&stash, new_total)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can already remove this tx?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes probably. @gpestana ?

if new_balance < Self::minimum_balance() {

// look at total balance to ensure ED is respected.
let new_total_balance = amount.saturating_add(Self::total_balance(who));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is an important change:

In the past amount + Self::balance() had to meed ED requirements, now amount + Self::total_balance.

What is your explanation of this change? Is it affecting any other systme?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The holds intentionally was not included into ED, I think the reason is that with the new fungibles contract we wanna guaranty that holds are always slashable, for the same reason we introduced the hold reasons. To change this you need really strong reasoning and for sure a separate PR.

Copy link
Contributor Author
@Ank4n Ank4n Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a separate PR

A separate PR is fair.

What is your explanation of this change

Without this, any staking reward of less than 1 DOT to an account who staked all their balance would fail.

Also, ED requirement even when an account has an active hold seemed like an oversight to me, since the account can not be killed if it has an active hold.

I think the reason is that with the new fungibles contract we wanna guaranty that holds are always slashable

How is it related? Why would holds not be slashable if free part of the balance does not have ED requirements on its own? All held amount can still be slashed and account would be dusted in that case right? Or am I missing something.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if account has some consumer or provider reference? not staking related

@@ -987,6 +989,19 @@ impl<T: Config<I>, I: 'static> OnUnbalanced<NegativeImbalanceOf<T, I>> for Palle
}
}

/// Implement the `OnUnbalanced` handler for [`frame_support::traits::fungible`] trait currency.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably unrelated to the PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Staking uses fungible now, and treasury is the slash handler for it. While treasury still uses Currency which is not compatible with fungibles. FungibleCompat adds a way for treasury to manage fungible imbalance.

Copy link
Contributor
@kianenigma kianenigma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good! I will approve after this round of comments are addressed.

The new code path for purging session keys seems new, but untested.

@kianenigma
Copy link
Contributor

One meta comment I have is that this pallet is already passed the point of being useful to anyone else beyond the polkadot relay chain.

If we pretend to be general, we need to having migration guides, and so on.

Instead, I suggest that as a part of AHM, we move this + other polkadot-only pallets that might be lingering here to the fellowship repo.

And importantly, rename this pallet to pallet-polkadot-npos. We can keep an older version of this that is bug-free in polkadot-sdk as pallet-staking.

@@ -987,6 +989,19 @@ impl<T: Config<I>, I: 'static> OnUnbalanced<NegativeImbalanceOf<T, I>> for Palle
}
}

/// Implement the `OnUnbalanced` handler for [`frame_support::traits::fungible`] trait currency.
pub struct FungibleCompat<T>(PhantomData<T>);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use tokens::imbalance::ResolveTo type with getter type of account id


let _ = T::Currency::resolve(&Pallet::<T>::account_id(), credit).defensive();

Pallet::<T>::deposit_event(Event::Deposit { value: numeric_amount });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the even already publishing already there on resolve completion, done by implementation of fungibles::Balanced::done_deposit method.

if new_balance < Self::minimum_balance() {

// look at total balance to ensure ED is respected.
let new_total_balance = amount.saturating_add(Self::total_balance(who));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The holds intentionally was not included into ED, I think the reason is that with the new fungibles contract we wanna guaranty that holds are always slashable, for the same reason we introduced the hold reasons. To change this you need really strong reasoning and for sure a separate PR.

@@ -2146,6 +2180,19 @@ pub mod pallet {
);
Ok(())
}

#[pallet::call_index(30)]
Copy link
Contributor
@muharem muharem Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about a doc? (=

so basically we can migrate everything ourself? users no need to do anything? and lazy migrating on ledger update just to make sure nothing happens before

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ledger update does not migrate. I did it this way since 1. its safe (just extra locked funds), and 2. no extra read/write while ledger updates since this code can linger for a while. But practically, we will migrate all accounts via a bot, so there will be no extra locked funds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T2-pallets This PR/Issue is related to a particular pallet.
Projects
Status: In review
Status: Scheduled
Development

Successfully merging this pull request may close these issues.

5 participants