Always had a thing for
XAMPP. This is a German site by default, but an English version is there as well.
Windows and Linux version available. In 35 to 45 Mbyte you will get an Apache webserver, MySQL database server, Mail server, PHP, Perl, OpenSSL, PHPMyAdmin etc. All ready to run after unpacking and executing 1 small batch script.
Having said that the default configuration is very open, making it ideal for developing but not to put in a (semi-) public place. There is a link in the opening page that takes care of that so you should not have to lose too much sleep over it.