From the last post there were big steps ahead.
Second variant of TCP code was good enough and I’ve even got result ‘tes’ (3 letters of word ‘test’) on server side while testing. After some testing and improvments, whole buffer was transferred well, but there were no resending queue. It appeared in third variant.
For that purposes was needed a small and cute packet allocation manager (receive buffer is still uses common cyclic buffer routines, but send data buffer is used by packet allocation manager). Effective memory allocation in case of variable size of packets is rather difficult task, so I was implementing other task – allocating of fixed size packets. Simple enough to implement it in short period of days. All packets were divided in two groups – “small” packets and “big” packets. Small packets are used for system messages (SYN, FIN, ACK without data, RST), big – are for data transmitting.
How it works? Buffer space is divided in chunks with size equal to maximum size of “small” packet. It’s now 64 bytes (this is enough for IP, TCP header, TCP options and segment description). Every chunk contains TCP related segment description (resending time and other). Sequential chunks are gathered together in blocks. Block is used for allocating data for “big” packets and it’s size 512 bytes. So, memory manager just allocates blocks or chunks in place. Block may be exclusively be owned by one big segment or by count of small segments. Amazing, that it worked from the beginning without signifcant errors. Segment resending function uses segment description, so it’s possible to send segments allocated using other manager or just statically allocated (used in sending RST packets when there are no connection, thus there are no socket and thus there are no buffer and space to allocate chunks)
So memory manager was ready and resending become pretty simple – just check resend time of segment and resend it. And I’ve started simple testing of code by sending blocks of generated sequences with identical initial generators on both sides of connection. And it worked perfectly till summar transferred data to one side was less then buffersize. Bigger amounts of data make code crazy, guard checks of memory allocator become very talkative and loved easily to fell to panic. Few days of testing located problem part of code in buffers read/write code in some cases. After standalone tesing module for buffers all become correct and data started transferring well. First test was in trabsferring 8 MBytes of data (that exceeds defsault buffer size 32K). Good new was that data is going correctly: per byte checking showed no errors. Bad new was that speed was about 5-6Kbytes/s at FastEthernet. Really annoying to wait all this 8Mb to be transferred.
The reason was simple – after filling receiving buffer and getting small window size (less then default mss) server side was waiting about second or two before asking client side, is any free space now. So, one fix in pxe_recv(), which started checking connection by sending ACK every 100 ms if receving buffer have much free space, saved the Earth, pardon, project. Speed become 500K-1000K. Well, still is not best, but better than was, next speed up may be done by sending/recv’ing bigger blocks (in test it was byte by byte in order to check sequences are correct). Now there is another problem client code is very nasty if server is not sending anything, client bombs server with ACKs about big window ready to work. I need to think how to make client behavior more correct. At this point 80Mb of data were transferred correctly and relatively fast, so test was over with verdict – passed.
It was so good, that even warned me a little bit. Time showed I was right, not all was so good. To be more accurate – connection breaking was handled incorrectly, when server initiated breaking (passive closing from client view on this situation). Some updates in tcp_disconnect() helped to solve problem and LAST_ACK, CLOSE_WAIT are now also handled in right way. At least I think so and tests prove it. This changing in disconnect function give also speed up bonus in active connection closing.
But anyway, next version of TCP implementation is already in thoughts to simplify pxe_tcp module and reduce function count. May be some other changes will be done. E.g. I’m trying to reduce sending of ACK packets by using incremental ACK (now every incoming segment is confirmed). This already is performed in checking of “same” packets: every packet that is just copy of current, placed in queue, with older sequence number to ACK is removed from resending queue. It helps a little bit, but solution is still not good enough.
Meanwhile I was doing http client, the only function of which was to fetch chosen file (I was testing on html-page) from web-server. There was not anything special, except that I was sent again to read style(9) and implemented simple snprintf() in libstand by modyfing PCHAR() macro. http receiving code was working, so next function implemented partial requesting of data. It’s rather simple, just adding of other field in header (assuming server supports such feature). Main task to do here in future – don’t establish connection for every reading of part of file. Now it established at start of reading and closed after that.
Next step, I forget about – filesystem. Well, for reasons I don’t know, I was thinking pxe.c is all that needed to change and files become getting itself from web server (by the way, www-server option was added to DHCP-client to get it ip). But it’s rather logical to separate filesystem and device. So, I was needed – make stub filesystem code. Using already available code it was simple – just call pxe_http module functions at needed time.
So, next days I’ll continue checking and correcting of existing code (also, once more style updates…), but main task will be tie together filesystem, pxenet device, http code and see what will be in result of this mix.