[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/export balances command #166

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Feature/export balances command #166

wants to merge 1 commit into from

Conversation

t2y
Copy link
Contributor
@t2y t2y commented Apr 20, 2019 •
edited by medvedev1088
Loading

I implemented export_balances command to output balances from addresses in blocks/transactions. There are three addresses as below.

  • block.miner
  • transaction.from_address
  • transaction.to_address

The export_balances command uses eth_getbalance to get balance.

$ ethereumetl export_balances --help
Usage: ethereumetl export_balances [OPTIONS]

  Exports balances.

Options:
  -s, --start-block INTEGER  Start block
  -e, --end-block INTEGER    End block  [required]
  -o, --output TEXT          The output file. If not specified stdout is used.
  -b, --batch-size INTEGER   The number of blocks to export at a time.
  -p, --provider-uri TEXT    The URI of the web3 provider e.g.
                             file://$HOME/Library/Ethereum/geth.ipc or
                             https://mainnet.infura.io
  -w, --max-workers INTEGER  The maximum number of workers.
  -c, --chain TEXT           The chain network to connect to.
  -h, --help                 Show this message and exit.

The cli usage is like this.

$ ethereumetl export_balances --start-block 47218 --end-block 47218 --output balances.json
2019-04-20 10:36:53,454 - ProgressLogger [INFO] - Started work. Items to process: 1.
2019-04-20 10:36:54,690 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2019-04-20 10:36:54,690 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:01.235745.
2019-04-20 10:36:54,691 - CompositeItemExporter [INFO] - balance items exported: 5

The output is here. This data format is my requirement, so I can add additional columns as needed.

{
  "block_number": 47218,
  "block_hash": "0x889c421abc62a48641eee140519e6da8c9dc01d85d8f5c4fbc3c13e3c6e4cb3e",
  "address": "0x9746c7e1ef2bd21ff3997fa467593a89cb852bd0",
  "balance": "0xe8d2ca53c30e4d7f50"
}
{
  "block_number": 47218,
  "block_hash": "0x889c421abc62a48641eee140519e6da8c9dc01d85d8f5c4fbc3c13e3c6e4cb3e",
  "address": "0x1406854d149e081ac09cb4ca560da463f3123059",
  "balance": "0x27f3c2a8698527d790"
}
{
  "block_number": 47218,
  "block_hash": "0x889c421abc62a48641eee140519e6da8c9dc01d85d8f5c4fbc3c13e3c6e4cb3e",
  "address": "0xa0e74ae010d51894734c308d612131056bb721ad",
  "balance": "0x5f68e81328279cda8"
}
{
  "block_number": 47218,
  "block_hash": "0x889c421abc62a48641eee140519e6da8c9dc01d85d8f5c4fbc3c13e3c6e4cb3e",
  "address": "0xe6a7a1d47ff21b6321162aea7c6cb457d5476bca",
  "balance": "0xfb1712c43448e19e3c"
}
{
  "block_number": 47218,
  "block_hash": "0x889c421abc62a48641eee140519e6da8c9dc01d85d8f5c4fbc3c13e3c6e4cb3e",
  "address": "0xee80ef3c49d9465c7fc2b3d7373fdbbbc3fe282f",
  "balance": "0x70f89193d7505e40"
}

I'm not sure my requirement is general purpose. I will do extra tasks as below if you will accept my PR.

  • add export_balances into export_all command
  • add ExportBalancessJob into EthStreamerAdapter.export_all() method
  • add README about export_balances
  • tell me another task if needed

Could you review export_balance command?

@medvedev1088
Copy link
Member

Thank you for this PR! I'm going to review it next week as I'm a little busy at the moment.

@t2y t2y force-pushed the add-export-balances-command branch from 511b5e6 to 129ce83 Compare April 23, 2019 23:07
@medvedev1088
Copy link
Member

Hi @t2y. Thanks again for the PR.

Is the intention for the export_balances command to include all addresses whose balance was modified in the given blocks, or is it supposed to output a subset of addresses? If it's intended to include all addresses then it should probably rely on the traces instead of blocks/transactions, because blocks/transactions may miss addresses whose balances were changed in internal transactions, and uncle rewards. For what purposes are you using this command right now?

I've tried solving a similar problem using traces. Check out my article here https://medium.com/google-cloud/how-to-query-balances-for-all-ethereum-addresses-in-bigquery-fb594e4034a7.

@t2y t2y force-pushed the add-export-balances-command branch from 129ce83 to 39b168d Compare May 19, 2019 06:34
@t2y
Copy link
Contributor Author
t2y commented May 19, 2019

@medvedev1088 Thank you for your review, and I'm sorry my reply is too late.

I learnt that getting balances from addresses in blocks/transaction may not be exact. My use case is to detect balance change from simple transfer. I though that the use case of consumer to consumer transfer or a balance change in cryptocurrency exchange may be enough without checking trace.

I'm not sure my simple use case is proper as exporting command or not. I will agree with your idea if my PR is not appropriate, then cancel this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@t2y @medvedev1088