0

I have many backups of a same file. Is there a way to transform that into incremental backup?

Those files aren't exactly the same (not same timestamps sometimes, sometimes new data appended here and there)

I can't just search for dupes files, and I can't just delete old files for the new one, because sometimes the old one have data not here anymore

I want a way to delete duplicated content from files. So there will be unique data across all the files. Ideally that would be merging, because if I just delete bunch of datas, the file would be unopenable, because sometimes theres duplicated formatting datas

The problem is idk if new datas are purely by lines, or sometimes in the same line. It's not just a story about dupe lines, sometimes it's a part of the line who is duplicated

Do you have any ideas?

Improve this question
5
  • 1
    Deduplicating backup systems like restic or borgbackup does this automatically, but it sounds like you want to adjust some pre-existing backups? How exactly have you backed up your file so far? Are you using rsync or tar or some other software, or are you just making copies of files? Commented Mar 3, 2022 at 8:36
  • It's pre existing backups. The software produce a new file every X time, so he rewrite a lot of identical data in the newly created file. I just copy the file to a path actually, and it begins to become big Commented Mar 3, 2022 at 8:58
  • Entropy only permits like 2% of compression, so yeah, I let it as is. Maybe a software who will consider the whole as 1 file so it will automatically simplify duplicates through compression? And by that achieve maybe 99+% of compression ? (in final it's a little bit the same process, but with maths involved) Commented Mar 3, 2022 at 8:59
  • Why don't you show some parts / lines of some files, and what should happen to those? Commented Mar 3, 2022 at 12:49
  • It's a proprietary format that I don't really understand, it doesn't use much ASCII characters. The file in itself is supposed to contain data of the software, the user data. So they're supposed to be opened with this software Commented Mar 3, 2022 at 13:39

0

Reset to default

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.