Incubaid Research – Rediscovering the RSync Algorithm

Don’t walk the folder and ‘rsync’ each file you encounter. A small cal­cu­la­tion will show you how bad it really is.

Sup­pose you have 20000 files, each 1KB. Sup­pose 1 rsync costs you about 0.1s (read­ing the file, send­ing over the sig­na­ture, build­ing the stream of updates, apply­ing them). This costs you about 2000s or more than half an hour.

Sys­tem admin­is­tra­tors know better:they would not hes­i­tate: “tar the tree, sync the tars, and untar the synced tar”.

Sup­pose each of the actions takes 5s (over­es­ti­mat­ing) you’re still synced in 15s.

via Incubaid Research – Redis­cov­er­ing the RSync Algo­rithm. The right way to synch two remote file systems.

Web Developer, constant student, backpacker and hater of the Oxford comma.

Tagged with: , , , , ,
Posted in Programming, Quick Posts, Security, Software Code
Short Link:


Sub­scribe to this site via RSS | Twit­ter |

Fol­low me on Twit­ter | GitHub

  • No bookmarks avaliable.