[drats_users] transfer Block Sizes

[Dan Smith]

> I think I'm opening a can of worms by asking this

Whoa, I'll say.

> When a file is transferred, is the block size used that of the sender, 
> the receiver, the originating station of the transfer, or the smallest 
> block size of the stations involved?

Well, it's a little more complicated than just that, so here are some
points to keep in mind:

1. The block size is just the size of a chunk of data that the sender
   will take out of the source file being sent each time.  It may
   actually end up smaller (with compression), but has nothing to do
   with what the remote's block size is set to.
2. The block size doesn't limit the size of other packets, such as a
   long chat packet or an RPC response (like a big file list reply)
3. Some of the protocols depend on the atomicity of the packets to some
   degree.  By this I mean that the RPC layer expects a file list
   request to be a single packet and sends back the entire response in a
   single packet.  This could be resolved (or worked-around) with a
   layer in the middle, but it might be hard without changing the on-air
   packet format.

> I don't know what information gets passed when the handshake gets 
> made, so I don't know the real answer.

There's no block size information in the negotiation, so neither side
knows what the other's preferred block size is.

> What I'm thinking is several stations are in a Net, and all but one 
> can reliably transfer files, etc. with a Block Size of 4096, and one 
> poor sole can only do 128.  How many stations in that net must use a 
> block size of 128?  Is it all stations, or only any station that wants 
> to communicate with station 128?

Right, technically you'd expect anyone to set their size down to 128
when communicating with that station.  We needn't elaborate on how
inconvenient or ridiculous this would be in practice :)

> To make it worse, if there are multiple ports, i.e. radios, I suppose 
> I should set my transfer Block Size for the worst path?

Mmm, yeah, I guess that mostly falls under the same scenario as the
above "poor soul", but obviously an order of magnitude worse.

> Sorry.

Heh, yeah, good :D

So, in reality, this is really a rich-man's problem in that it's
something that I haven't really dealt with yet because there hasn't been
much of a need.  When I started D-RATS and defined some of the limiting
factors for these issues, there were no network proxies, multiple ports,
multiple transfer types (TNC, D-STAR, Net), etc.

I think that the proper solution, or at least a good way to mitigate the
problem is to have the transfer layer try to discover the appropriate
block size during a transfer.  This means picking some reasonable
starting point (perhaps 512) and then scaling that up to some maximum if
things are going well and scaling it back if not (or if things start to
degrade).  I had some code for this early in the 0.2.x days, but I never
really finished it and was distracted by all the other things that were
going on at that point.

It requires changing more of the implementation than just selecting a
new block size each time around because of the way I pipeline packets now.

Unless we're talking about changing the on-air packet format (which is a
*real* can of worms), then I think we can work our way out of this hole
at least somewhat my making the sending station smarter.  Because of
that, I'd say that we should attack the problem in 0.3.1 so as not to
delay 0.3.0 with any potentially destabilizing changes of that magnitude.

What do you think?

-- 
Dan Smith
dsmith#danplanet.com, s/#/@/
www.danplanet.com
KK7DS