Remove just exim4 package itself on Debian 7 with following command
#apt-get remove exim4
Remove exim4 with dependent packages on Debian 7 with following command
#apt-get remove --auto-remove exim4
Remove exim4 and delete configuration, data files on Debian 7 with following command
#apt-get purge exim4
Remove and delete exim4 configuration, data files, dependencies on Debian 7 with following command
#apt-get purge --auto-remove exim4