Hi i have written a shell script which has to do below,

1. capture session commands into one file.
2. Every individual command into separate file.
3. mail every individual command file content based on certain criteria..

As for my observation, the loop has to iterate minimum of 25,000 times. Now my problem is that it is taking more than 6 hours to complete all iterations.

Below is the main part of the script which is taking long time to process.

 if [ -s "$LOC/check.txt" ]; then while read line; do echo -e " started processing $line at `date` " >> "$SCRIPT_LOC/running_status.txt" TST=`grep -w $line $PERM_LOC/id_processing.txt` USER=`echo $TST | grep -w $line | awk -F '"' '{print $10}'` HOST=`echo $TST | grep -w $line | awk -F '"' '{print $18}'` ID=`echo $TST | echo $line | tr -d '\"'` IP=`echo $TST | grep -w $line | awk -F '"' '{print $20}'` DB=`echo $TST | grep -w $line | awk -F '"' '{print $22}'` CONN_TSMP=`echo $TST | grep -w $line | awk -F '"' '{print $2}'` if [ -z "$IP" ]; then IP=`echo "$HOST"` fi if [ "$USER" == "root" ] && [ -z $DB ]; then TARGET=/data1/sessions/root_sec CMD_TARGET=/data1/commands/root_commands FILE=`echo "$ID-$CONN_TSMP-$USER@$IP.txt"` else TARGET=/data1/sessions/user_sec CMD_TARGET=/data1/commands/user_commands FILE=`echo "$ID-$CONN_TSMP-$USER@$IP.txt"` fi ls $TARGET/$FILE If [ $? -ne 0 ]; then echo $TST | awk -F 'STATUS="0"' '{print $2}'| sed "s/[</>]//g" >> "$TARGET/$FILE" echo -e "\n" >> "$TARGET/$FILE" fi grep $line $LOC/out.txt > "$LOC/temp.txt" while read val; do TSMP=`echo "$val" | awk -F '"' '{print $2}'` QUERY=`echo "$val" | awk -F 'SQLTEXT=' '{print $2}' | sed "s/[/]//g"` echo " TIMESTAMP=$TSMP " >> "$TARGET/$FILE" echo " QUERY=$QUERY " >> "$TARGET/$FILE" RES=`echo "$QUERY" | awk {'print $1'} | sed 's/["]//g' ` TEXT=`grep "$RES" "$PERM_LOC/commands.txt"` if [ -n "$TEXT" ]; then NUM=`expr $NUM + 1` SUB_FILE=`echo "$ID-$command-$NUM-$TSMP-$USER@$IP.txt"` echo -e "===============\n" > "$CMD_TARGET/$SUB_FILE" echo "FILE = \"$SUB_FILE\"" >> "$CMD_TARGET/$SUB_FILE" ### same way append 6 more lines to $SUB_FILE SUB=`echo "$WARN_ME" | grep "$command"` if [ "$command" == "$VC" ]; then STATE=`echo " very critical "` elif [ -z "$SUB" ]; then STATE=CRITICAL else STATE=WARNING fi if [ "$USER" != "root" -a "$command" != "$VC" ]; then mail command & elif [ "$USER" == "root" -a -z "$HOST" ]; then mail command & elif [ "$USER" == "root" -a "$command" == "$VC" ]; then mail command & else echo -e "some message \n" >> $LOC/operations.txt fi fi done < "$LOC/temp.txt" done < "$LOC/check.txt" fi 

Can any one help me how to optimize this code either by dividing or by changing logic or by using functions or by anything else?

Here i have use shell script only and Server on which the script will be executed should not take more than 3GB of RAM to process it.

Any help is very very useful..