Return to Answer

Since the file is not of any of the types of executable recognised by the system, and assuming you've got the permission to execute that file, the execve() system call will typically fail with a ENOEXE (not an executable) error.

execlp()/execvp(), upon execve() returning ENOEXE will typically invoke sh on it. For systems that have more than one sh because they can conform to more than one standard, which sh it is will be typically determined at compilation time (of the application using execvp()/execlp() by linking a different blob of code which refers to a different path to sh). For instance, on Solaris, that will be either /usr/xpg4/bin/sh (a standard, POSIX sh) or /bin/sh (the Bourne shell (an antiquated shell) on Solaris 10 and older, ksh93 in Solaris 11).

Also note that if execve() fails with ENOEXE but the file does have a shebang line, some shells do try to interpret that shebang line themselves.

Since the file is not of any of the types of executable recognised by the system, and assuming you've got the permission to execute that file, the execve() system call will typically fail with a ENOEXEC (not an executable) error.

execlp()/execvp(), upon execve() returning ENOEXEC will typically invoke sh on it. For systems that have more than one sh because they can conform to more than one standard, which sh it is will be typically determined at compilation time (of the application using execvp()/execlp() by linking a different blob of code which refers to a different path to sh). For instance, on Solaris, that will be either /usr/xpg4/bin/sh (a standard, POSIX sh) or /bin/sh (the Bourne shell (an antiquated shell) on Solaris 10 and older, ksh93 in Solaris 11).

Also note that if execve() fails with ENOEXEC but the file does have a shebang line, some shells do try to interpret that shebang line themselves.

Most other applications will use either of those when they run a command. They will invoke a shell for instance by way of the system("command line") libc function which will typically invoke sh to parse that command line (the path of which can be determined at compile time (like /bin/sh vs /usr/xpg4/bin/sh on Solaris)), or invoke the shell stored in $SHELL by themselves like vi with it's ! command, or xterm -e 'command line' and many other commands (su user -c will invoke the user's login shell instead of $SHELL).

execlp()/execvp(), upon execve() returning ENOEXE will typically invoke sh on it. For systems that have more than one sh because they can conform to more than one standard, which sh it is will be typically determined at compilation time. For instance, on Solaris, that will be either /usr/xpg4/bin/sh (a standard, POSIX sh) or /bin/sh (the Bourne shell (an antiquated shell) on Solaris 10 and older, ksh93 in Solaris 11).

When it comes to shells, there's a lot of variation. bash, AT&T ksh, the Bourne shell will typically interpret the script themselves (in a child process unless exec is used) after having simulated a execve(), that is unset all the unexported variables, closed all the close-on-exec fds, removed all the custom traps, aliases, functions... (bash will interpret the script in sh mode). yash will execute itself (with sh as argv[0] so in sh mode) to parse it.

Most other applications will use either of those when they run a command. They will invoke a shell for instance by way of the system("command line") libc function which will typically invoke sh to parse that command line (the path of which can be determined at compile time (like /bin/sh vs /usr/xpg4/bin/sh on Solaris)), or invoke the shell stored in $SHELL by themselves like vi with its ! command, or xterm -e 'command line' and many other commands (su user -c will invoke the user's login shell instead of $SHELL).

execlp()/execvp(), upon execve() returning ENOEXE will typically invoke sh on it. For systems that have more than one sh because they can conform to more than one standard, which sh it is will be typically determined at compilation time (of the application using execvp()/execlp() by linking a different blob of code which refers to a different path to sh). For instance, on Solaris, that will be either /usr/xpg4/bin/sh (a standard, POSIX sh) or /bin/sh (the Bourne shell (an antiquated shell) on Solaris 10 and older, ksh93 in Solaris 11).

When it comes to shells, there's a lot of variation. bash, AT&T ksh, the Bourne shell will typically interpret the script themselves (in a child process unless exec is used) after having simulated a execve(), that is unset all the unexported variables, closed all the close-on-exec fds, removed all the custom traps, aliases, functions... (bash will interpret the script in sh mode). yash will execute itself (with sh as argv[0] so in sh mode) to interpret it.

The system("./myscript") libc function will typically invoke a shell (sh, the path of which can be determined at compile time (like /bin/sh vs /usr/xpg4/bin/sh on Solaris) to parse that ./myscript command line (and that's that shell that will perform the execve() and decide what to do when it fails with ENOEXE).

vi's !, or xterm -e and many other commands will also invoke a shell to parse a command line, but that time, that will be the one whose path is stored in $SHELL (initialised from the login shell). su user -c will invoke the user's login shell...

• When $SHELL is /bin/bash, xterm -e myscript will have myscript interpreted by bash in sh mode.
• su -c myscript on Solaris 10 where root's login shell is /bin/sh and /bin/sh is the Bourne shell will have myscript interpreted by the Bourne shell.
• /usr/xpg4/bin/awk 'BEGIN{system("myscript")' on Solaris 10 will have it interpreted by /usr/xpg4/bin/sh (same for /usr/xpg4/bin/env myscript).
• find . -prune -exec myscript {} \; on Solaris 10 (using execvp()) will have it interpreted by /bin/sh even with /usr/xpg4/bin/find, even in a POSIX environment (a conformance bug).
• csh -c myscript will have it interpreted by csh if it starts with #, with sh otherwise.

Most other applications will use either of those when they run a command. They will invoke a shell for instance by way of the system("command line") libc function which will typically invoke sh to parse that command line (the path of which can be determined at compile time (like /bin/sh vs /usr/xpg4/bin/sh on Solaris)), or invoke the shell stored in $SHELL by themselves like vi with it's ! command, or xterm -e 'command line' and many other commands (su user -c will invoke the user's login shell instead of $SHELL).

• When $SHELL is /bin/bash, xterm -e 'myscript with args' will have myscript interpreted by bash in sh mode. While with xterm -e myscript with args, xterm will use execvp() so the script will be interpreted by sh.
• su -c myscript on Solaris 10 where root's login shell is /bin/sh and /bin/sh is the Bourne shell will have myscript interpreted by the Bourne shell.
• /usr/xpg4/bin/awk 'BEGIN{system("myscript")' on Solaris 10 will have it interpreted by /usr/xpg4/bin/sh (same for /usr/xpg4/bin/env myscript).
• find . -prune -exec myscript {} \; on Solaris 10 (using execvp()) will have it interpreted by /bin/sh even with /usr/xpg4/bin/find, even in a POSIX environment (a conformance bug).
• csh -c myscript will have it interpreted by csh if it starts with #, with sh otherwise.