I'm trying to make it so when I create a new user either by using useradd or adduser, an empty directory named "whatever" is automatically created under that user's home directory.
I've tried playing around with the /etc/default/useradd config file but no luck.
I'm not trying to change every new user's default home directory but rather just have an empty directory automatically be created the same time the new user is created.
sudo mkdir -p /etc/skel/whatever /etc/skel is the "skeleton" of a new user's home directory; anything located therein is replicated as the starting point for new users' homes.
From, for instance, the manual page for useradd:
-k, --skel SKEL_DIR The skeleton directory, which contains files and directories to be copied in the user's home directory, when the home directory is created by useradd. This option is only valid if the -m (or --create-home) option is specified. If this option is not set, the skeleton directory is defined by the SKEL variable in /etc/default/useradd or, by default, /etc/skel. If possible, the ACLs and extended attributes are copied. Or from adduser's:
adduser will copy files from SKEL into the home directory and prompt for finger (gecos) information and a password. The gecos may also be set with the --gecos option. With the --disabled-login option, the account will be created but will be disabled until a password is set. The --disabled-password option will not set a password, but login is still possible (for example with SSH RSA keys). To set up an encrypted home directory for the new user, add the --encrypt-home option. For more information, refer to the -b option of ecryptfs-setup-private(1). 
mkdir /etc/skel/whatever, maybe? Can't test it right now so not an answer as such.