0

I am using AIX 7.1.

I made some shell scripts8 and called /bin/bsh.

Inside the script, I connect to a server via FTP and have many get commands because I would like to avoid the wildcard problem. I just declare every filename and get them one by one, for example:

get PDF_02378230_20110609.PDF get PDF_02432482_20110609.PDF get PDF_02432565_20110609.PDF get PDF_02432573_20110609.PDF get PDF_02432581_20110609.PDF get PDF_02432599_20110609.PDF get PDF_02432607_20110609.PDF get PDF_02432615_20110609.PDF get PDF_02432623_20110609.PDF get PDF_02432649_20110609.PDF get PDF_02432656_20110609.PDF get PDF_02432672_20110609.PDF get PDF_02432755_20110609.PDF get PDF_02432763_20110609.PDF get PDF_02432821_20110609.PDF get PDF_02432920_20110609.PDF get PDF_02433175_20110609.PDF get PDF_02433266_20110609.PDF get PDF_02433290_20110609.PDF get PDF_02433308_20110609.PDF get PDF_02433373_20110609.PDF get PDF_02433399_20110609.PDF

There could be up to 100000 files to be transfer in 1 shell scripts. I would run these shell scripts and let them ftp the files down. But in some shell script there are 1 or 2 missing files not been "get". For example, when I count the number of files, there might be 1 or 2 less files than the number of "get" command in the shell script. I cannot found an exception or error inside the log. The truth is, I cannot go through the whole log line by line because it is too long. But I have found which file was missing, and I tried to search the log file with the missing filename, and I cannot found any trace of that filename.

Improve this question
6
  • Illuminate me. What "wildcard problem"? Commented Aug 22, 2011 at 3:02
  • forget about it, I don't want to start it here. Commented Aug 22, 2011 at 3:08
  • I'm genuinely curious, as I don't have much AIX experience. Is there a link I can read? Commented Aug 22, 2011 at 3:10
  • That's nothing specific to AIX. It's just a common "argument too long" problem arosed by wildcard in unix syste. Commented Aug 22, 2011 at 3:34
  • 1
    Huh. I wouldn't have thought that it would affect a ftp script... Commented Aug 22, 2011 at 3:36

2 Answers 2

Reset to default
2

Some FTP clients like lftp and ncftp have a mirror command that can be used to avoid problems like this by figuring out what needs to be done to sync a local and remote folder for you.

You might also check if your client supports mget or a way to do a recursive download and work on the parent folder instead of everything in it.

However the best solution would be to switch to a different protocol. This sounds like a great time to start using rsync.

Improve this answer
1
  • AIX's default ftp client definitely supports mget. Commented Aug 22, 2011 at 9:33
2

I would switch to wget if the other end has to be an FTP server. Even then you'll still need to handle exceptions where individual files don't transfer properly. With wget however you can at least tell it to retry a few times per file, and you can also run it again but tell it to ignore any files that are the same size locally (so it'll just re-get files that it didn't get first time).

Not sure if AIX 7.1 comes with wget pre-installed, but it'll be on the Linux Toolbox CD, or you can get it from the IBM website.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.