Create shaper_bufferbloat.rst guide #571
Conversation
Created a guide for FQ_CoDeL to combat Bufferbloat on OPNsense. There are several posts on forum and internet with wrong settings. This should provide proper configuration with optional tuning of FQ_C.
Adjusted some wording and meaning
Had twice 2b > changed to 2a
Added sites for testing bufferbloat
Bufferbloat results after FQ_CoDel configuration and tuning
added shaper_bufferbloat to how-to page
|
I tried to make the guide bit more informative as well provide additional information on the parameters. As it looks like a lot of users do something without knowing why. Or try to look what a parameter does but its hard to find. P.S. This is my 1st official pull request ever, if there is something wrong let me know I will try to adjust it to my best knowledge. |
Adjusted wording and hopefully last typo
last typos found
* Adjusted some explanation for interval & target * Added flows parameter into tuning and important parameters * Added official FQ_C RFC * Typos
|
I am using such a setup to fight bufferbloat on my asymmetric DSL. (250/40) What helped me additionally was creating an additional Shaper rule, that matches "tcp (ACK packets only)" and targeting them with an own "Queue TCP ACK Upload" queue with the highest weight (100), which is part of the Upload Pipe that has CoDel enabled. This seems to reduce latency of TCP handshakes when the Upload is heavily used. I didn't measure anything though, I've just read at a few places that it would be a good choice to do. As reference, here is my whole shaper config, I don't know how many forum posts I've read but I think it's pretty close to your docs, with more pipes and queues for better traffic shaping. |
|
I've read somewhere the same. However, I myself with the configuration I have, didn't yet seen/experienced latency on Upload during load, so its hard for me to test and write about it. However Download on my side is somewhat tricky (Due to my ISP's technological limitation) I think the reason (in this specific case) why this Upload or Download latency may happen is due to the flows parameter, the default 1024 may be in some use-cases too low, which would cause some specific flows classified into same slots (sub-queues), this would cause for them as well a latency if that specific sub-queue is experience it. Even if increase of flows parameters could improve it, ultimately I think creating a separate Queue for TCP ACK, which with CoDeL will create sub-queues for specific TCP ACK flows, is ultimately better way. This is just my rough (positive imagination) estimate, but usually the no-knobs deployment for FQ_CoDeL works in 98% of the time if the BW parameter in the Shaper Pipe is set properly. The 2% are then the setups were it needs tuning additional to the out of the box setup of FQ_CoDeL. I may however maybe try it on Download Pipe to see if there will be improvements. If so we could added it as one of the tuning points into the guide. I like your Shaper config, very diligent. Thank you for the config. |
|
The problem is that there is no rule of thumb as it heavily depends on media type (LAN, Ethernet, DSL, Satelliten, Cable) and latency. E.g. for cable with variing upload speeds you may hit congestion earlier than your pipe kicks in |
|
You are absolutely right, that's the main problem actually. For Download as well Upload alike. For the Shaper/Fq_CoDeL to work all the time properly, you need to hit "persistently" the throughput specified in the BW parameter within the Pipe. Which on these technologies is hardly possible sometimes (or most of the time, really depends). Even thou FQ_CoDeL does have mechanisms such as target & interval (another one is limit but... not usable currently) that measure the delay for each queue and can take action upon it, the fact that you set BW 500 but can reach at a time only 200M will cause a back-pressure which will result in increased latency. This specific I tested, and the results was even if this happens the latency is still lower with FQ_CoDel in such situation than without FQ_CoDeL at all. The margin in these situation was 50-120ms (with FQ_CoDel) vs 1200-2000ms (without FQ_CoDeL) during congestion time, worst case scenario. As for the rule of thumb, there is none, but its recommended to target 85% of advertised ISP BW. Which in theory and most cases should include the worst case scenario throughput. However even this recommendation is recommended to take with grain of salt and needs to be tuned accordingly. Thus doing few runs of Internet speed-tests, to possibly get average throughput and from it target 85% as the BW value in the Shaper is a good start. I would however extent this, to target 85% of the lowest possible throughput as your BW value, if somebody wants to always try prevention of bufferbloat. The caveat is, the lowest possible that sometimes occurs maybe waaay to low thus you are loosing substantial amount of BW. Its a heavy tradeoff. For example if I have 500M and lowest throughput occurrence sometimes is 200M. I could set 85% of that 200M as the BW in Pipe. But I am cutting myself of 300M possible throughput that I am usually able to get just to prevent the occasional latency. |
* Changed some working * Typos * More clarified the problem in regards of FQ_CoDeL implementation in BSD, as this is not directly related to limit parameter, but its related to the implementation *ECN note add
|
I did contact the original creators behind FQ_CoDeL and asked them to give potential review for explanation of FQ_CoDeL & its parameters. They gave me some feedback and tips. I changed this PR to Draft. I will include their feedback into the How-To. As I want to have this as complete and perfect as possible so the wider audience is well informed, and has at least some grasp on what to configure as baseline and tune further if needed. I am also retesting and checking further the configuration. P.S. Also I found a mistake in the config example, the queue for object pipe shouldn't have any values (it has no configuration value), this is used for dynamic queue such as FiFo etc. when we are not specifying any Queue objects. Its not needed as we will create separate queue object binded to the specific Pipe. Regards, |
|
Few observations/notes from my side (OPNsense configuration): A. CoDeL can be configured in Pipe
B. CoDeL can be configured in Queue
C. FQ_CoDel is configured in scheduler
|
* Rewrote & adjusted partially some sections (interval & target, quantum) * Did more testing on configuration using GUI as well ipfw on CLI * Removed some obsolete configuration options (MASKs, Queue value) * Explained/Specified better some configuration options * Added sources (IPFW BSD 14.1 and others) * Adjusted text formatting
* Corrected result for ping directly from CLI
Many editorial changes
docs: Richb edits on shaper_bufferbloat.rst Many editorial changes
Committing an update that includes as well Rich Brown's editorial overhaul with more specific and precise explanation FQ_CoDel: * Put all the background info for FQ_CoDel at the top * I made all the lines short. I find it easier to edit, and it's easier to see the changes. (Not so much this time, but in the future. It avoids 200-character lines in a diff where phrases are added and deleted, and you're trying to figure out which words changed...) * I moved the details of FQ_CoDel out of the tables to the top. Now the tables only contain the critical info about the particular setting (not the entire algorithm) * In each of the "B" steps, I just said, "Do it like the 'A' step, but use these values." I think it decreases clutter and shows how the A & B steps are similar. It also avoids repetitive instructions that drift apart in subsequent edits. DRY - Don't Repeat Yourself. * I dropped many of the details of FQ_CoDel machinery (new queues/old queues) since they don't change the settings people need to consider. (Of if they really want to get into it, they should work with the RFC) * Made capitalization of "FQ_CoDel" consistent with RFC 8290 * Added screenshots of the "big three" useful speed test websites * Adjusted explanation in Options for OPNsense configuration
|
Hello @AdSchellevis I think I am finished here. I was lucky enough to reach towards FQ_CoDel creators & bufferbloat community. They were kind enough to review fully the docs adjust it and provide more proper details about FQ_CoDel. Special thanks to @richb-hanover & @dtaht for giving me their time, advice and full review! P.S. last thing that is needed is just to check for grammar and spelling errors. Regards, |
I forgot to add the better explanation for FQ_C. Now all should be in place
Run the text thru a grammar checker in VS Code Fixing last grammar issues
|
@SeimusS very nice writeup, thanks for contributing it. I'll merge and push it to the docs. |
|
PS @SeimusS - you're famous(-ish)! https://randomneuronsfiring.com/opnsense-fights-bufferbloat/ Thanks again for letting me contribute. |
|
@richb-hanover hahaha bit unexpected yet lovely of you. Many thanks for mentioning OPNsense (& its official docs for FQ_C) on your blog! I had fun too. Regards, |
|
One question I didn't put the reference for the doc "shaper_bufferbloat" into source/manual/shaping.rst its only in the source/manual/how-tos/shaper.rst. Which makes it maybe a bit hidden when somebody looks at the Traffic Shaping manual as whole. Is it possible you will additionally put it there as well? Otherwise I can open another PR.
Regards, |
|
@AdSchellevis @SeimusS Google Alerts for "bufferbloat" points me to this video: https://www.youtube.com/watch?v=x05Nxw4E8fI Enjoy! |
Created a guide for FQ_CoDeL to combat Bufferbloat on OPNsense. There are several posts on forum and internet with wrong settings. This should provide proper configuration with optional tuning of FQ_C.