Comments on: Weekend hack: adfsd, a kqueue-assisted rsync tool

By: Bookmarks about Rsync

Bookmarks about Rsync — Sun, 30 Nov 2008 07:45:22 +0000

[...] – bookmarked by 2 members originally found by guevarani on 2008-10-21 Weekend hack: adfsd, a kqueue-assisted rsync tool http://blogs.freebsdish.org/ivoras/2008/06/08/weekend-hack-adfsd-a-kqueue-assisted-rsync-tool/ – [...]

By: Max

Max — Fri, 28 Nov 2008 11:56:42 +0000

i updated rsync to 3.0.4, in which command “–del” present.
revert in adfsd.c “–delete” to “–del”, recompile it

but error still happens when i try to delete file ..

I even download hash.h and recomplie ..

By: ivoras

ivoras — Fri, 28 Nov 2008 11:35:14 +0000

Your hash.h problem is very strange – it is certainly present: http://svn.freebsd.org/viewvc/base/head/sys/sys/ .

From rsync(1):

–del an alias for –delete-during
–delete delete extraneous files from dest dirs
–delete-during receiver deletes during xfer, not before

The “–delete-during” should also work, but it might also cause your error with deleting the file. It looks like rsync doesn’t wont to sync a “non-existant” file. Maybe your version of rsync doesn’t support it?

Anyway, see comment #5

By: Max

Max — Fri, 28 Nov 2008 11:09:43 +0000

Thanks, Ivoras!

I have no hash.h in /usr/include/sys but fnv_hash.h, so, i use it.
Error happens in rsync, my rsync want option “–delete” instead of “–del”, i correct that and remake.
Next error happens wneh i try to delete file:

adfsd[83300]: Change detected in: adfsd.core
adfsd[83300]: Change detected in:
adfsd[83300]: Sent to rsync: adfsd.core
rsync: link_stat “/var/ports/distfiles/adfs/adfs/source/adfsd.core” failed: No such file or directory (2)
rsync error: some files could not be transferred (code 23) at main.c(702)
adfsd[83300]: Sent 1 files to rsync
adfsd[83300]: Partial rsync terminated with error 23
adfsd[83300]: Sent to rsync:
adfsd in free(): error: chunk is already free
Abort (core dumped)

By: ivoras

ivoras — Thu, 27 Nov 2008 14:58:30 +0000

hash.h is in /usr/include/sys – it should be picked up automatically on FreeBSD 6+. Sorry about the warnings, just ignore them.

By: Max

Max — Thu, 27 Nov 2008 14:33:05 +0000

make
gcc -o adfsd -Wall -g adfsd.c
adfsd.c:33:22: sys/hash.h: No such file or directory
adfsd.c:93: warning: ‘debug_level’ defined but not used
adfsd.c:101: warning: ’stat_to_mini_stat’ defined but not used
*** Error code 1

Stop in /var/ports/distfiles/adfs/adfs.

By: ivoras

ivoras — Fri, 24 Oct 2008 20:30:04 +0000

10 MB. The whole approach of using kqueues for this appears to be flawed since kqueues coalesces requests. I don’t think adfs should be used at all – something completely different will need to be implemented.

By: Adam

Adam — Fri, 24 Oct 2008 18:16:22 +0000

What constitutes a “large” file? Are we talking 10mb, 100mb, or gigs?

By: Adam

Adam — Fri, 24 Oct 2008 18:15:37 +0000

What constitutes a large file?

By: ivoras

ivoras — Fri, 13 Jun 2008 23:20:06 +0000

I cannot replicate problems with deleting files but I can for ‘big’ files. I’ll look into it.