Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Comments on Can rsync handle moves and renames?

Parent

Can rsync handle moves and renames?

+2
−0

Suppose I have a directory structure like

├── src
│   ├── folder_a
│   │   ├── file_w
│   │   └── file_x
│   ├── folder_b
│   │   ├── file_y
│   │   └── file_z

and I back it up using rsync:

$ rsync -r /path/to/src/ /path/to/dst/

Later, I reorganize the contents of src, for example:

├── src
│   ├── folder_c
│   │   ├── file_w
│   │   └── file_y
│   ├── folder_d
│   │   ├── file_x
│   │   └── file_z

Each file with a given name has not changed; but they don't have the same hierarchical relationship (i.e., files that used to be siblings might not be any more), and the names of their containing folders may have changed.

Can rsync detect that the dst folder contains the same files, and avoid copying files (instead only moving files within dst and renaming folders)? What options are useful to make this as smooth as possible?

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+4
−0

I'm quite certain that rsync is not able to deduce that /path/to/src/folder_a/file_w on one side of the transfer is identical to /path/to/src/folder_c/file_w on the other side of the transfer.

Therefore, rsync won't be able to handle such a move efficiently. To stock rsync, it will appear as one file having been deleted and the other having been created.

There are basically two ways of handling this:

  • Independently make the same changes on both sides of the transfer. That's easy for a single file, or if you control both sides and can access them simultaneously; but that might not always be practical.
  • Use a tool better suited for such change detection.

For example, if the (in this case) source code is in a Git repository, you can use git mv to rename the parent directory; this should allow the tooling to keep track of which file is which even after the rename. Other version control systems will likely have ways to do much the same thing.

Alternatively, you can use a tool such as unison, possibly in one-way mode, to synchronize changes. I can attest to that it generally successfully detects and handles renames without re-transferring the file, but it relies on a fairly hefty chunk of external metadata to do so efficiently, and generally wants to either do the initial transfer itself or start with two identical trees; using it to transfer such a change after it has already been made would probably be risky.

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.

2 comment threads

Files storage in git - effective, but different use case (1 comment)
Good to know (1 comment)
Good to know
Karl Knechtel‭ wrote 10 months ago

My actual use case involves mostly large binary files. I have a vague recollection of using Unison on Windows many years ago; I think I must have assumed that it only existed to replace rsync. But now I've seen that Unison is used on Linux for good reason, and also that rsync builds exist for Windows; so there's a good chance I'll end up looking in to Unison again. Thanks for the confirmation; I would hate to have wasted the time to learn the hard way.