5

I have one input file and run some command but want the output to be saved with the same name as the input file has.

I tried below command but it makes the output file blank:

cat file1 | grep "YISHA" > file1
Improve this question
3
  • 2
    Simplest way is use temp-file grep "YISHA" file1 > file1.tmp ; mv file1{tmp,} Commented Mar 11, 2015 at 8:49
  • 1
    The answers cover the solutions nicely. Just to comment on why this happens: the order of execution in a pipeline is not sequential, bash has to set up "plumbing" before it runs cat and grep: it needs to connect the output of cat to the input of grep and the output of grep to the output file before the pipeline is run, so file1 is already empty and ready for writing into, when cat opens it. Commented Mar 11, 2015 at 11:22
  • Related: What are the shell's control and redirection operators? — Warning regarding ‘>’. Commented Aug 19, 2015 at 19:46

3 Answers 3

Reset to default
8

On GNU system, you could use sed (the GNU implementation):

sed -i -n '/YISHA/p' file1

The FreeBSD or OS/X equivalent:

sed -i '' -n '/YISHA/p' file1

or using sponge from moreutils:

grep "YISHA" file1 | sponge file1
Improve this answer
2
  • for sponge it is showing command not found at terminal Commented Mar 11, 2015 at 9:21
  • 2
    @yisha: you need to install moreutils. Commented Mar 11, 2015 at 9:23
4

When removing data, you can write the file over itself and truncate it afterwards:

{ grep YISHA perl -e 'truncate STDOUT, tell STDOUT' } < file 1<> file

Of course here, you can do everything in perl:

perl -ne 'print if /YISHA/; END{truncate STDOUT, tell STDOUT}' < file 1<> file

perl also has a -i option for in-place editing (the one that GNU sed copied):

perl -ni -e 'print if /YISHA/' file

But note that like sed, it creates a new file with the same name, it doesn't really rewrite the file in-place, which means inode numbers and other attributes of the file could be affected in the process. It will also break symlinks.

Improve this answer
1
  • It will break hard links to this file, and will break if the file itself is a symlink pointing elsewhere. If it's a regular file and there are symlinks pointing to it, no problem. Commented Mar 11, 2015 at 18:29
4

Your shell will likely get you a secure temp file on request:

grep "YISHA" <<IN > file $(cat file) IN

That will drop blank lines from tail of file though (which shouldn't be relevant unless you're grepping for blank lines). If that should matter, then just echo . after cat in the command substitution and drop the last line.

Another option available to you is dd. For example:

seq 5000000 >/tmp/temp -rw-r--r-- 1 mikeserv mikeserv 38888896 Mar 11 04:20 /tmp/temp

Just a dummy file large enough to outweigh any pipe buffer.

</tmp/temp grep 5\$ | dd bs=4k of=/tmp/temp conv=notrunc,sync

You can see that I exceeded the size of any possible pipe buffer:

949+1 records in 950+0 records out 3891200 bytes (3.9 MB) copied, 0.164471 s, 23.7 MB/s

When the notrunc conversion is specified, dd doesn't touch the output file except to write over what it reads in. With seek= you could even put that input data at some other offset in the file if you liked. But... the file still needs truncating. You can see that dd flushed its last input buffer as well: 949+1 records were read in, but 950 were written out - dd synced it's last input block to the full 4k size w/ nulls (which is a generally reasonable block size to choose when accepting piped input from tools that use stdio - such as grep).

So...

ls -l /tmp/temp; tail /tmp/temp -rw-r--r-- 1 mikeserv mikeserv 38888896 Mar 11 04:22 /tmp/temp 4999991 4999992 4999993 4999994 4999995 4999996 4999997 4999998 4999999 5000000

It's still the same file for everything beyond what dd wrote.

But...

dd if=/dev/null bs=4k seek=950 of=/tmp/temp

...we can truncate it to the point that dd wrote to it, and...

0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.000153783 s, 0.0 kB/s

...it looks like nothing happened, except that...

ls -l /tmp/temp; tail /tmp/temp -rw-r--r-- 1 mikeserv mikeserv 3891200 Mar 11 04:25 /tmp/temp 4999915 4999925 4999935 4999945 4999955 4999965 4999975 4999985 4999995

dd cuts it short that time. In truth, though, there is that last synced partial block at the tail of the file, so...

tail /tmp/temp | wc -c 2383

...there are a bunch of nulls at the end.

Improve this answer
2
  • Except with zsh, that won't work with binary files (files containing NUL bytes). Like sponge, that stores the whole file in memory (and in a temp files with most shells). Commented Mar 11, 2015 at 11:06
  • @StéphaneChazelas - won't work w/ binary files as written in most any shell - that's true. Some tricks can be done with /dev/fd/num links though. I sometimes do cmd 3<<! 3<<!\n!\n$(cmd >/dev/fd/3)\n!\n but different shells handle that differently. Thanks for the edit. I'm really sleepy, I guess. Commented Mar 11, 2015 at 11:16

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.