(doc) add aggregate trade data collection

pr8x · Dec 15, 2020 · 0eca6ee · 0eca6ee
1 parent aa9e2db
commit 0eca6ee
Showing 1 changed file with 43 additions and 19 deletions.
diff --git a/DATA_COLLECTION.md b/DATA_COLLECTION.md
@@ -1,39 +1,63 @@
 # Important Disclosure re: Hummingbot Data Collection
-
+
+**Updated as of version 0.34 12/15/2020**
+
 Hummingbot (“Hummingbot”) is open-source software developed and maintained by CoinAlpha, Inc. (“CoinAlpha”, “we”, or “us”) and available at https://github.com/coinalpha/hummingbot.
 
 CoinAlpha collects certain usage information from Users that have configured Hummingbot to allow for the sharing of this data.
 
-## Purpose of data collection
+## Sensitive Data
 
-### Software improvement
+Hummingbot will never collect and/or report sensitive information, such as private keys, API keys, or passwords.
 
-Data collected by CoinAlpha is used to help improve Hummingbot for all users.  It is important that our team understands the usage patterns and detects bugs as soon as possible, so we can best decide how to design future features and prioritize current work.
+## Aggregate Trade Data Collection Notification
 
-### Support
+The open-source Hummingbot client is fulfilling its mission of democratizing market making. The growth in features, connectors, and community members has been incredible for the team to watch. With growth comes additional opportunities but also additional expenses. Our 24/7 online support and world class documentation is provided at no cost to the community. These services are provided by full-time paid employees around the world (did you know that Hummingbot has team members on six different continents?). There are multiple ways open-source projects raise funds. Some offer a scaled down “free” version that benefits from revenues generated by a paid “pro” version. Other projects ask for donations if users find the software beneficial. Hummingbot’s model is somewhat unique in that its users are already “contributing” by adding liquidity and trading volume to our exchange and protocol partners. However, continued growth and development is going to require us to prove those contributions.
 
-Users who provide data allow us to more easily diagnose problems, troubleshoot, and resolve any issues encountered when running Hummingbot.
+To sustain development of the Hummingbot client, we partner with exchanges and protocols to build connectors and/or strategies. These partners need actionable data to convince their stakeholders and community members that a Hummingbot integration is worth the investment. The most common requests include total volume traded, total number users, and which trading pairs. Since the Hummingbot client doesn’t currently collect any data, we currently can't answer these questions well and rely on guesstimates based on volume measured on the Hummingbot Miner platform. However, we know from conversations with many community members that the volume activity on Hummingbot Miner is a small subset of the overall volume facilitated each week by the Hummingbot client. For these reasons, beginning with Hummingbot version 0.34, a “heartbeat” data collection mechanism will be implemented.
 
-## Types of data collected
+## Heartbeat Mechanism Key Points
 
-Hummingbot can collect **error logs** and report it to CoinAlpha.
+- Heartbeat will send aggregate trade volume data to Hummingbot servers every 15 minutes
+- Trade data will include volume totals, order count, trade count (not individual trades), tokens traded (base & quote), and exchange(s) used.
+- Hummingbot client data will include software version, installation type (binary, source, Docker), O/S, and a random unique ClientID*.
+- All data collected will be used exclusively by the Hummingbot team for analytical and marketing purposes only. The data will be neither accessible nor sold to any third party.
 
-***Note***: *Users using Hummingbot to participate in any [Liquidity Mining Campaigns](https://docs.hummingbot.io/liquidity-mining/campaigns/) may separately be subject to additional [data collection terms](https://hummingbot.io/liquidity-mining-policy/).*
+\* More on the benefits of ClientID below
 
-## Sensitive data
+As you can see, the data above is aggregated data intended solely to allow us to obtain more information on high-level usage statistics. To be clear, we are not collecting individual trade-specific data or strategy configurations and we have purposely added code in the client that aggregates this data locally on the Hummingbot instance before sending the summary data to our servers. As for the Hummingbot software data (e.g. version, installation type, and O/S), this will help the development and QA teams to prioritize features and efforts to ensure that we are efficiently addressing our user’s needs.
 
-Hummingbot will never collect and/or report sensitive information, such as private keys, API keys, or passwords.
+An FAQ with more technical details follow, but feel free to move on with your life if you love Hummingbot and trust our decision making around this much needed feature. If you have any questions or concerns not addressed in the FAQ, please reach out to us in the General or Support channels on Discord.
+
+## Frequently Asked Questions
+
+### 1. Where is the heartbeat mechanism configured and what if I want to turn it off?
+
+The `conf_global.yml` file will have the following configuration parameters:
+- `heartbeat_enabled`: default = `True`
+- `heartbeat_interval_min`: default = `15`
+
+Changing the enabled flag to `False` will disable all data reporting. The heartbeat interval parameter may be adjusted if you’d rather send data on a slower or faster interval. Fifteen minutes was chosen and test by the team as a reasonable interval for an active trading Hummingbot instance
+
+### 2. Why should I participate in these data gathering? What’s in it for me?
 
-## How do I opt-in to or opt-out of data sharing?
+The Hummingbot trading bot is free to use and yet is built and maintained by a world class core team of developers and finance industry experts. It allows anyone with any amount of capital to become a market marker and/or an arbitrageur on the world’s largest and most active platforms. You also get 24/7 free online support with knowledgeable, full-time paid support staff. Maybe you could share a bit of anonymous volume data to help us grow the product?
 
-When configuring Hummingbot, you will be asked the following question:
+### 3. What’s the reason/rationale for each piece of data collected?
 
-```
-Would you like to send error logs to hummingbot? (Yes/No) >>>
-```
+| Data Item | Benefit to Hummingbot |
+| :-------  | :--------------- |
+| Total Trade Volume | Used by the Core team to secure new exchange connectors, protocol grants, and other projects seeking liquidity. There is power in numbers and a provable filled order volume increases the team’s ability to sign new deals and move the project forward.
+| Total Orders Created & Total Completed Trades | The Hummingbot client can be viewed as an automated “order generation machine”. Exchanges and protocols are interested in how many orders & trades are flowing through the client. 
+| Base/Quote Asset | Help craft strategies that are most beneficial to the types of markets in which the community is active. Do most users trade highly liquid pairs that require fast execution or should we focus on arbitrage opportunities available on smaller exchanges and smaller market cap tokens? We don’t know the answer without this data.
+| Exchange(s) | Strengthen our partnerships with exchanges used most by our users. Justify the Hummingbot ecosystem as a viable alternative to traditional expensive market making firms.
+| Software Version | Learn how often our users upgrade their clients so we can align our release cycles. Example: If 98% of users are only upgrading once per year, why is the team cranking out a new release roughly every four weeks?
+| Installation Type | Knowing the installation type will help the team better align technical resources to the most popular installations.
+| Operating System | Supporting Windows, Mac, and Linux require development hours, separate QA testing, segmented documentation. With limited resources, the team needs to focus on what the community is using.
+| ClientID | The ClientID is randomly generated and does not track or identify the user in any way. The data stored is strictly limited to the elements detailed here. The unique ID is needed to calculate metrics such as total bots active, average volume per bot, etc. as well as to ensure we are not double counting activity.
 
-Enter `Yes` to opt-in; enter `No` to opt-out.
+### 4. How can I be sure that’s the only data Hummingbot is collecting?
 
-## How this data is used
+Hummingbot is 100% open-source. Once version 0.34 is released, please feel free to review for yourself…even better, please review and post on our Discord channels what you found. We’ve always been open and honest with our community members and you’ll find this case is no different.
 
-When enabled by a user, data is reported by Hummingbot to CoinAlpha.  CoinAlpha uses its own systems as well as third parties (such as Datadog Inc.) to collect and process data.
+We welcome any feedback and can answer any additional questions you may have. Please join our [discord community](https://discord.hummingbot.io) to chat with the team or send us an email at [dev@hummingbot.io](mailto:dev@hummingbot.io).