1

Linux Mint 20.3

lsblk -V

lsblk from util-linux 2.34

Let's find an explanation for the ambiguous behavior of the lsblk utility applied with the -E option. Here is the output without applying the -E option. You can see that there are repeated entries in the SIZE column. These are (loop0 loop2) and (loop6 loop10).

greg@mynt23:~$ lsblk -o NAME,SIZE NAME SIZE loop0 164,8M loop1 4K loop2 164,8M loop3 55,4M loop4 73,9M loop5 66,2M loop6 449M loop7 91,7M loop8 44,4M loop9 40,9M loop10 449M loop11 74,1M loop12 55,7M sda 298,1G ├─sda1 4,9G ├─sda2 104,2G └─sda3 189,1G

Now apply the -E option.

greg@mynt23:~$ lsblk -o NAME,SIZE -E SIZE NAME SIZE loop1 4K loop3 55,4M loop4 73,9M loop5 66,2M loop7 91,7M loop8 44,4M loop9 40,9M loop11 74,1M loop12 55,7M sda 298,1G ├─sda1 4,9G ├─sda2 104,2G └─sda3 189,1G

The repeated entries from the SIZE column disappeared completely, instead leaving one "original" entry. The entries left should have been loop0 and loop6.

Apply RM or TYPE as a key. You will see that all duplicates are removed, but the "original" entry is left. The manual says that duplicates are removed, not duplicates along with what the duplicates duplicate.

How do we explain it?

Improve this question
2
  • Please add lsblk -V to your question Commented Feb 5 at 7:19
  • I also get different output between lsblk -o NAME,SIZE -E SIZE and lsblk -o NAME,SIZE | awk '!h[$NF]++'. Maybe it's a bug Commented Feb 5 at 7:21

2 Answers 2

Reset to default
1

Duplicates in your first example have no inter-dependencies, but duplicates in your second example do (all of the part entries depend on some disk device). The process of building the dependency tree seemingly causes the list to be traversed multiple times – as can be seen with LSBLK_DEBUG=all – and although the deduplication works correctly the first time (e.g. sda is kept), the 2nd pass mistakenly carries on from the same state and when it encounters the same (only remaining) sda it believes that its dedup key has already been seen, thus removing it.

Sometimes the explanation really is just "that's a bug which needs to be fixed".

953439: lsblk: DEV: iterate on /sys/block -- done 953439: lsblk: DEV: [0x64b8dd060a90]: sdd: de-duplication key: disk 953439: lsblk: DEV: [0x64b8dd069d70]: sdd3: de-duplication key: part 953439: lsblk: DEV: [0x64b8dd061d30]: sdb: de-duplication key: disk 953439: lsblk: DEV: [0x64b8dd062fb0]: sdb2: de-duplication key: part 953439: lsblk: DEV: [0x64b8dd0642b0]: sdb3: de-duplication key: part 953439: lsblk: DEV: [0x64b8dd065590]: sdb1: de-duplication key: part 953439: lsblk: DEV: [0x64b8dd066870]: sdc: de-duplication key: disk 953439: lsblk: DEV: [0x64b8dd067af0]: sdc3: de-duplication key: part 953439: lsblk: DEV: [0x64b8dd067d20]: sda: de-duplication key: disk 953439: lsblk: DEV: [0x64b8dd067ef0]: sda2: de-duplication key: part 953439: lsblk: DEV: [0x64b8dd068140]: sda3: de-duplication key: part 953439: lsblk: DEV: [0x64b8dd068370]: sda1: de-duplication key: part 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: disk 953439: lsblk: DEV: [0x64b8dd060a90]: sdd: match deduplication pattern 953439: lsblk: TREE: [0x64b8dd0579a0]: remove duplicate device: 0x0x64b8dd060a90 [sdd] 953439: lsblk: DEV: [0x64b8dd061d30]: sdb: match deduplication pattern 953439: lsblk: TREE: [0x64b8dd0579a0]: remove duplicate device: 0x0x64b8dd061d30 [sdb] 953439: lsblk: DEV: [0x64b8dd066870]: sdc: match deduplication pattern 953439: lsblk: TREE: [0x64b8dd0579a0]: remove duplicate device: 0x0x64b8dd066870 [sdc] 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: part 953439: lsblk: DEV: [0x64b8dd067ef0]: sda2: match deduplication pattern 953439: lsblk: DEV: [0x64b8dd067d20]: remove duplicate dependence: 0x0x64b8dd067ef0 [sda2] 953439: lsblk: DEP: [0x64b8dd069ca0]: dealloc 953439: lsblk: DEV: [0x64b8dd068140]: sda3: match deduplication pattern 953439: lsblk: DEV: [0x64b8dd067d20]: remove duplicate dependence: 0x0x64b8dd068140 [sda3] 953439: lsblk: DEP: [0x64b8dd069d10]: dealloc 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: disk 953439: lsblk: DEV: [0x64b8dd067d20]: sda: match deduplication pattern 953439: lsblk: TREE: [0x64b8dd0579a0]: remove duplicate device: 0x0x64b8dd067d20 [sda] 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: part 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: disk 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: part 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: disk 953439: lsblk: TREE: [0x64b8dd0579a0]: de-duplicate by key: part 953439: lsblk: TREE: [0x64b8dd0579a0]: dealloc
Improve this answer
1
  • What do you mean by Duplicates in your first example have no inter-dependencies, but duplicates in your second example do (all of the part entries depend on some disk device)? If under the second example there is a result of using lsblk -o NAME,SIZE -E SIZE, then there are no duplicates there at all after using the option to remove duplicates. Commented Oct 27 at 4:53
-1

From the lsblk man page:-

-E, --dedup column

Use column as a de-duplication key to de-duplicate output tree. If the key is not available for the device, or the device is a partition and parental whole-disk device provides the same key than the device is always printed.

So a dedupe on SIZE will give the observed behaviour.

Improve this answer
3
  • 1
    Apply RM or TYPE as a key. You will see that all duplicates are removed, but the "original" entry is left. The manual says that duplicates are removed, not duplicates along with what the duplicates duplicate. Commented Feb 5 at 1:18
  • 1
    JeremyBoden that is what the documentation states but it is not the observed behaviour. Or there is a disjoint amongst people's understanding of the word "de-duplicate" Commented Feb 5 at 7:22
  • Yes - my mistake. its a bug. Commented Feb 5 at 12:56

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.