Incubaid Research – Rediscovering the RSync Algorithm

Don’t walk the folder and ‘rsync’ each file you encounter. A small cal­cu­la­tion will show you how bad it really is.

Sup­pose you have 20000 files, each 1KB. Sup­pose 1 rsync costs you about 0.1s (read­ing the file, send­ing over the sig­na­ture, build­ing the stream of updates, apply­ing them). This costs you about 2000s or more than half an hour.

Sys­tem admin­is­tra­tors know better:they would not hes­i­tate: “tar the tree, sync the tars, and untar the synced tar”.

Sup­pose each of the actions takes 5s (over­es­ti­mat­ing) you’re still synced in 15s.

via Incubaid Research – Redis­cov­er­ing the RSync Algo­rithm. The right way to synch two remote file systems.

Working at Loadsys as a Web Developer and learning something new every day.

Tagged with: , , , , ,
Posted in Programming, Quick Posts, Security, Software Code
Short Link: http://jty.me/zv5keL