Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Comments on Can rsync handle moves and renames?

Parent

Can rsync handle moves and renames?

+2
−0

Suppose I have a directory structure like

├── src
│   ├── folder_a
│   │   ├── file_w
│   │   └── file_x
│   ├── folder_b
│   │   ├── file_y
│   │   └── file_z

and I back it up using rsync:

$ rsync -r /path/to/src/ /path/to/dst/

Later, I reorganize the contents of src, for example:

├── src
│   ├── folder_c
│   │   ├── file_w
│   │   └── file_y
│   ├── folder_d
│   │   ├── file_x
│   │   └── file_z

Each file with a given name has not changed; but they don't have the same hierarchical relationship (i.e., files that used to be siblings might not be any more), and the names of their containing folders may have changed.

Can rsync detect that the dst folder contains the same files, and avoid copying files (instead only moving files within dst and renaming folders)? What options are useful to make this as smooth as possible?

History
Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+4
−0

I'm quite certain that rsync is not able to deduce that /path/to/src/folder_a/file_w on one side of the transfer is identical to /path/to/src/folder_c/file_w on the other side of the transfer.

Therefore, rsync won't be able to handle such a move efficiently. To stock rsync, it will appear as one file having been deleted and the other having been created.

There are basically two ways of handling this:

  • Independently make the same changes on both sides of the transfer. That's easy for a single file, or if you control both sides and can access them simultaneously; but that might not always be practical.
  • Use a tool better suited for such change detection.

For example, if the (in this case) source code is in a Git repository, you can use git mv to rename the parent directory; this should allow the tooling to keep track of which file is which even after the rename. Other version control systems will likely have ways to do much the same thing.

Alternatively, you can use a tool such as unison, possibly in one-way mode, to synchronize changes. I can attest to that it generally successfully detects and handles renames without re-transferring the file, but it relies on a fairly hefty chunk of external metadata to do so efficiently, and generally wants to either do the initial transfer itself or start with two identical trees; using it to transfer such a change after it has already been made would probably be risky.

History
Why does this post require moderator attention?
You might want to add some details to your flag.

2 comment threads

Files storage in git - effective, but different use case (1 comment)
Good to know (1 comment)
Files storage in git - effective, but different use case
Arpad Horvath‭ wrote 2 months ago

Storing more files in git with exactly the same content is quite efficient. It doesn't care what it the name of the file. It stores the file name, and the information where can you find the content. So if you move a huge file in the next commit to a different location (or you rename it) the size of the repo will be increased just a little bit. But if you change just one bit in that file, than it won't realize it if I'm correct. But I guess your use cases are different. You don't want to have the old files stored.