4

I have a cron job that uses rsync to periodically transfer all files that appear in a specific directory on server A to somewhere on server B. I use --partial and --append-verify as the files are large. I also use the remove-source-files option to deletes the files on server A when they are transferred. As per the man page, rsync only delete the source files after all file transfers have completed on a specific run.

This works fine most of the time, except if the transfer is interrupted partway through due to a network failure, after a few files have already been transferred. On the next run, the program sees some of the files already in the destination and correctly determines they don't need to be re-transferred. However, since they were not part of that transfer run, the files in the source server do not get deleted. Over time they build up.

Is there a way to tell rsync to delete the source files on the subsequent run, if they exist in the destination, or is this something I'll have to solve by writing my own scripts?

Improve this question
2
  • It may be helpful to list all the rsync options you are already using so other don't suggest ones that may already be in play. Commented Jun 12, 2017 at 21:05
  • Have you tried either the --checksum or --size-only options? Commented Jun 12, 2017 at 21:07

2 Answers 2

Reset to default
2

Honestly, its not really much of an answer, because it's kind of a hack way of getting the results you are requesting. So here goes:

If you trim out the --append-verify command and replace it with the --update command, leave all the other options and run the RSYNC without it.

This will delete all the source files while copying the files that have been modified.

Improve this answer
3
  • --update may not work. Rsync could see the partial file on the receiver as newer than the source and skip it as not needing to be copied. Commented Jun 12, 2017 at 21:10
  • I tried it exhaustively, it worked on my sandbox for partials that had same stamps on the receiver. I mean, I'm not really fond of my answer, it's truly lack-luster, but if you don't want to script something, it's a way to get by. :-/ Commented Jun 13, 2017 at 12:33
  • I will try this out the next time I get the problem, then come back to mark the answer. As luck would have it, the connection problems that have been so persistent have cleared up just when I wanted to try a solution. Commented Jun 14, 2017 at 10:44
0

I recently discovered an easy way:

rsync -rav --remove-source-files --backup-dir=<BAK> <SRC> <DST>

all files existing in DST and being detected as "changed" will be moved into BAK - now you can delete SRC folder and consider BAK as your new source folder.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.