Before we begin
This tutorial is intended to help people who wish to set up a web server on Amazon EC2 cloud. It took me a lot of time to come up with a nice Open Source based solution and I hope it will be helpful for someone else.
I am not very familiar with Linux, my background is programming, not system administration. The idea was to set up a web server with an easy-to-use control panel where someone like me, who is much more comfortable with graphical interface, could easily host multiple client web sites running on PHP+MySQL, create FTP and e-mail accounts for the users etc.
I only started documenting what I’m doing after I ran into a lot of problems and had to terminate/re-launch the instance many times, so this tutorial doesn’t really begin at the beginning. So what you’ll need is an instance in the cloud, with an Elastic IP assigned to it and an EBS attached to it as well – the EBS is going to be used for MySQL data, 10Gb is a good start for my needs, it may be different for you.
You will need a Linux with EBS root device type – I’m using a Debian Lenny distro ami-209e7349, not sure how this stuff would work on other configurations.
I assume you already figured out how to create a Key Pair and you can SSH into your instance using ppk-based authentication (if you’re using Putty, the private key ppk file goes to Connection/SSH/Auth).
joe and mc
I tried several times to figure out the standard Unix vi editor, and I hated it every time. So let’s start with installing joe – it reminds me of the great old times when Turbo Pascal was cool 😉
apt-get install joe
And Norton Commander has always been cool, too 😉
apt-get install mc
Initial update
Since we’re installing from a canned AMI, first thing is to update the system – but even before that let’s update the list of sources for APT:
joe /etc/apt/sources.list
~~~~~~~~~~~~~~~~~~~~~
deb http://ftp.us.debian.org/debian/ lenny main
deb http://security.debian.org/debian-security lenny/updates main
deb http://volatile.debian.org/debian-volatile lenny/volatile main
~~~~~~~~~~~~~~~~~~~~~
apt-get update
Mount EBS as /vol
The following steps are taken from this great tutorial: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1663. The idea is to have EBS volume mounted to the system – it becomes available as /vol and it’s indistinguishable from any other directory. This is really awesome, because we now can create snapshots of the entire /vol directory within literally seconds. Note that EBS must be attached as /dev/shd.
# Install XFS (file system) and MySQL
apt-get install -y xfsprogs mysql-server
grep -q xfs /proc/filesystems || sudo modprobe xfs
sudo mkfs.xfs /dev/sdh # Then mount it as /vol
echo "/dev/sdh /vol xfs noatime 0 0" | tee -a /etc/fstab
mkdir -m 000 /vol
mount /vol
Install MySQL
I haven’t started documenting when I installed MySQL, but I’m pretty sure it’s done like this:
apt-get install mysql
It’s important to leave root password in MySQL blank – otherwise Virtualmin will complain later (you will set the root password for MySQL from Virtualmin later).
Move and bind MySQL directories to /vol
Now we move all MySQL stuff to EBS – and MySQL doesn’t even know about it. This is pretty cool, if you ask me 😉
# Stop MySQL
/etc/init.d/mysql stop
mkdir /vol/mysql # Move /var/lib/mysql to /vol/mysql/lib
mv /var/lib/mysql /vol/mysql/lib/
# Then create directory again
mkdir /var/lib/mysql
# And mount –bind it to /vol – this way MySQL is now looking at EBS
mount –bind /vol/mysql/lib /var/lib/mysql # Repeat for /etc/mysql:
mv /etc/mysql /vol/mysql/etc
mkdir /etc/mysql
mount –bind /vol/mysql/etc /etc/mysql # Repeat for /var/log/mysql
mv /var/log/mysql /vol/mysql/log
mkdir /var/log/mysql
mount –bind /vol/mysql/log /var/log/mysql # Restart MySQL
/etc/init.d/mysql start
I’m actually considering doing the same thing with /home directory, so that it’s easier to create backups of user-uploaded content.
"Autoexec.bat"
Some things need to be run on every system boot, so let’s place them in "Autoexec.bat" of Debian as explained here.
joe /etc/init.d/local
~~~~~~~~~~~~~~~~~~~~~
#!/bin/sh
mount –bind /vol/mysql/lib /var/lib/mysql
mount –bind /vol/mysql/etc /etc/mysql
mount –bind /vol/mysql/log /var/log/mysql
sleep 30 # boy, was I happy when I figured this out! 😉
hostname god.domain.com
/etc/init.d/mysql start
/etc/webmin/start
~~~~~~~~~~~~~~~~~~~~~
chmod 755 /etc/init.d/local
update-rc.d local start 98 2 3 4 5 .
mkdir /etc/rc.d/
ln -s /etc/init.d/local /etc/rc.d/rc.local
We don’t yet have Webmin installed, but that’s ok. Replace god.domain.com with your server fully qualified domain name of choice – in my practice people choose a particular god (like a Greek god Zeus or an Indian god Indra) and add it as a subdomain to your company name.
Notice the sleep 30 command – I’m not sure why it’s so, but if you give the server extra 30 seconds to load the rest of the stuff, it works perfectly, otherwise it just doesn’t work at all. I don’t know why Webmin and MySQL are not starting on their own – feel free to enlighten me. What’s important for me is that I can reboot my server and it’s up and serving within 1 minute. Thanks to Amazon, if the server dies, I can launch a new instance from the server AMI within another like 3 minutes – this is really powerful in my opinion.
FQDN
I don’t really understand how this works, but your server has to have a fully qualified domain name. I did the following to make it happen:
joe /etc/hosts
~~~~~~~~~~~~~~~~~~~~~
127.0.0.1 localhost.localdomain localhost
127.0.0.1 god.domain.com # << added these
127.0.0.1 mx.domain.com mx # << two lines
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
~~~~~~~~~~~~~~~~~~~~~
I’m not really sure if mx line is needed, I just copied it from somewhere and forgot about it… doesn’t seem to hurt anyone.
Download Virtualmin installer
cd /usr/local
mkdir virtualmin
cd virtualmin
wget http://software.virtualmin.com/gpl/scripts/install.sh
chmod +x install.sh
Set root password
passwd root
This is required to log in to Virtualmin later – it won’t work with a blank root password.
Create a temporary AMI
Now this is a great opportunity to create an AMI – if something goes wrong later, we won’t have to repeat all the steps above!
IMPORTANT: do not try to create an AMI from a running instance! Hopefully Amazon will fix this soon, but for me it just crashes the server and the image hangs in Failed state. I had to do it all over again when this happened – and this is when I started to actually document stuff, so there’s no bad without a drop of good in it 😉
I’m using Elasticfox to create AMIs – just stop your instance (wait until it shows "stopped"!), then right-click and choose "Create Image (EBS AMI)" – this is why we needed an image with root device type EBS from the beginning, otherwise I don’t really know how to create an AMI.
Install Virtualmin
/usr/local/virtualmin/install.sh
This takes a while, like 3-4 minutes.
php5-mysql
I have no idea why this module is not installed automatically, but you have to do this manually if you’re going to host PHP+MySQL web sites:
apt-get install php5-mysql
A few important settings in Virtualmin
At this point we can open our browser and navigate to https://god.domain.name:10000/ – almost forgot, make sure you add port 10000 and 21 to the security group assigned to your instance (I use the default All Incoming security group with these two ports added), otherwise you won’t be able to access Virtualmin and FTP.
Log in as root with the password you set before.
Once you’re there, Virtualmin will ask you a few questions – default answers work for me. I guess you should update all software packages as Virtualmin suggests right away – why not.
Now, a few final tweaks that I find necessary in my work:
1) Virtualmin by default assigns specific IP addresses to all new Virtual Servers, which is not cool, because every time you launch a new instance, your IP changes. To go around that we need to add NameVirtualHost directive to the main Apache config file. Go to Webmin -> Servers -> Apache -> Global -> Edit Config Files -> /etc/apache2/apache2.conf, the last two lines should read:
~~~~~~~~~~~~~~~~~~~~~
NameVirtualHost *:80
Include /etc/apache2/sites-enabled/
~~~~~~~~~~~~~~~~~~~~~
2) By default, FTP users can browse the entire directories tree – we totally should limit this if we don’t want our users to wander around the server. Go to Virtualmin -> Limits and Validation -> FTP Directory Restrictions and check Active box in the only row available (All virtual servers, Users’ home directories).
3) To treat .html files as PHP go to Webmin -> Servers -> Apache Webserver -> Global configuration -> Edit config files -> …/php5.conf:
~~~~~~~~~~~~~~~~~~~~~
<IfModule mod_php5.c>
AddType application/x-httpd-php .php .html
AddType application/x-httpd-php-source .phps
</IfModule>
~~~~~~~~~~~~~~~~~~~~~
5) MySQL Dumps from Windows have all table names in lowercase – to make MySQL the same (retarded) way here you can do this:
joe /etc/mysql/my.cnf~~~~~~~~~~~~~~~~~~~~~
lower_case_table_names = 1 # add this line in [mysqld] section
~~~~~~~~~~~~~~~~~~~~~
/etc/init.d/mysql restart
Conclusion
That’s it folks. Time to create another AMI and we are ready to host hundreds of web sites on this nice little server. With a reserved AMI instance the price should be around $300 for the first month and then something like $60-70/month, which is cheaper than a dedicated server machine you would get in any "normal" web hosting company.
Happy hosting in the cloud! 🙂
Leave a Reply