I don't think that we need to be as careful with framing at this layer. If we get blocked, that only prevents us from sending other frames on the stream, which boils down to trailers (which we don't send) and control frames (which don't currently exist).

So I would suggest sending everything we have as possible. Then you only have to track how much data you have left to send in the current frame (i.e., SendMessageState::SendingData now has a usize parameter). The only probing you would then need to do is to work out whether the header of the frame fits in the available space, but you can do that with the atomic write.

An application that does have other things to send than data (in the future perhaps) might withhold data from the stack to make space, or we can revise this code and cap the size of writes in some way.

That should make this a lot less complex: Encode a DATA frame header, write it atomically then send whatever data from the provided buffer you can using the non-atomic send.

That also means that an application that attempts to send X bytes and only is able to send X-Y bytes must later send at least Y bytes or bad things will happen because we will otherwise be unable to fill the frame. It's worth documenting that constraint.

Originally posted by @martinthomson in https://github.com/mozilla/neqo/diffs