May 29, 2016 | Author: Luca Sciarratta | Category: N/A
Download Hosting Ruby on Rails on Amazon AWS With Ruby...
Tutorial on hosting a web application (Ruby on Rails - RoR) on Amazon AWS with EC2, EBS, Ruby Enterprise Edition (REE) and Phusion Passenger (mod_rails). Author: K. C. Ramakrishna (www.rknowsys.com) License: GNU-FDL - http://www.gnu.org/licenses/fdl.html Date: 18-Nov-08 Please mail comments/corrections to kcr AT rknowsys DOT com, kcr AT members DOT fsf DOT org,
Note: This is a follow up to the tutorial on hosting the site with mongrel. This tutorial can be found here: http://docs.google.com/ Doc?id=dcn2ckbh_20hk4kc4d4 You may have to read it for some optimisations of RoR app to use EBS. Credits and Disclaimer: Quite a bit of stuff exists in various tutorials - especially the part where we configure mysql to use EBS - This excellent tutorial is on AWS resources page (I don't have the exact URL). We just made an effort to consolidate a lot of relevant stuff into one document with a consistent form. This is the general outline you need to follow for any app on AWS. Background: We built a tourism portal in RoR for one of our clients. You have a look at it - www.tripladder.com After building it, we were requested to host and manage it for them. Initially we went with knownhost which is OK but a production RoR application needs more RAM than what we get on most VPS plans - especially if we have image processing. We did consider AWS but at that time it did not have EBS and the client did not initially expect enough traffic to justify a 'scalr' managed cluster. We were looking for a replacement to a dedicated server. Once EBS was launched, we immediately decided to move the site to AWS. The Cost-benefit analysis is compelling. Update: tripladder.com is not live anymore, please look at www.vitalizeu.com but the below process is still good. The following tutorial starts off after signing up with AWS and configuring your desktop/ laptop to be able to connect to AWS and launch instances i.e. we assume that you have completed the 'Getting Started' section of AWS. We have started with the stock Fedora image and modified it to our requirements. We could have used CentOS but Fedora-8 appeared at the top of the list and we went ahead with it. The application hosting has the following steps. 1. Launching an instance. 2. Installing RoR, gems, plugins...We used rmagick, hence we had to install Imagemagick too. 3. Installing REE and Phusion (mod_rails) 4. Installing mysql. 5. Intalling the application (checkout from subversion). 6. Creating and attaching a EBS volume. Mysql with data on EBS 7. Modifying the RoR app to save user upload files to EBS.(http://docs.google.com/ Doc?id=dcn2ckbh_20hk4kc4d4) 8. Installing and configuring a production level ferret server 9. Configuring Apache to serve the application, caching optimisations for performance.
10. Configuring permanent public IP (covered) and DNS (we have the domain parked with go daddy but this is not covered in this article) 11. Configuring smtp (email) support for RoR application. 12. Once we have the perfect server setup, save it to S3. 13. Periodic automated backups - Using Amazon snapshots.
1. Launching an instance and logging in: guest@kc-laptop:~$ ec2-run-instances ami-2b5fba42 -k gsg-keypair -z us-east-1a RESERVATION r-a864bec1 526262918289 default INSTANCE i-xxxxxxxx ami-2b5fba42 pending gsg-keypair 0 m1.small 2008-11-09T10:19:29+0000 us-east-1a akia71cf9ce ari-a51cf9cc guest@kc-laptop:~$ ec2-describe-instances i-xxxxxxxx RESERVATION r-a864bec1 526262918289 default INSTANCE i-xxxxxxxx ami-2b5fba42 ec2-xxxxx-49-xxx.compute-1.amazonaws.com domU-12-31-39-00-68-93.compute-1.internal running gsg-keypair 0 m1.small 2008-11-09T10:19:29+0000 us-east-1a aki-a71cf9ce a51cf9cc
ari-
# Authorise access on port 22 for ssh guest@kc-laptop:~$ ec2-authorize default -p 22 GROUP default PERMISSION default ALLOWS tcp 22
22
FROM
CIDR
0.0.0.0/0
# Authorise access on port 80 for http guest@kc-laptop:~$ ec2-authorize default -p 80 GROUP default PERMISSION default ALLOWS tcp 80
80
FROM
CIDR
0.0.0.0/0
# Authorise access on port 3306 for mysql admin. guest@kc-laptop:~$ ec2-authorize default -p 3306 GROUP default PERMISSION default ALLOWS tcp 3306
3306
FROM
CIDR
0.0.0.0/0
1.1 Associating a permanent IP address with this instance. guest@kc-laptop:~$ ec2-allocate-address ADDRESS 75.101.158.167 guest@kc-laptop:~$ ec2-associate-address -i i-xxxxxxxx 75.101.158.167 ADDRESS 75.101.158.167 i-xxxxxxxx 1.2 Check by logging in guest@kc-laptop:~$ ssh -i ~/AWS-do-not-delete/id_rsa-gsg-keypair
[email protected] OR guest@kc-laptop:~$ ssh -i ~/AWS-do-not-delete/id_rsa-gsg-keypair
[email protected] Server is accessible;
Went to godaddy.com (or your registrar) and set the DNS records accordingly: tripladder.com 75.101.158.167
2. Updating the system and installing required software 2.1 Updating the system [root@domU-12-31-39-00-68-93 ~]# yum update Updated: fedora-release.noarch 0:8-6.transition Complete! 2.2 Installing mysql [root@domU-12-31-39-00-68-93 ~]# yum install mysql mysql-server mysql-devel Installed: mysql-devel.i386 0:5.0.45-6.fc8 mysql-server.i386 0:5.0.45-6.fc8 Dependency Installed: device-mapper-devel.i386 0:1.02.22-1.fc8 e2fsprogs-devel.i386 0:1.40.4-2.fc8 keyutils-libs-devel.i386 0:1.2-2.fc6 krb5-devel.i386 0:1.6.2-14.fc8 libselinux-devel.i386 0:2.0.43-1.fc8 libsepol-devel.i386 0:2.0.15-1.fc8 mysql.i386 0:5.0.45-6.fc8 mysql-libs.i386 0:5.0.45-6.fc8 openssl-devel.i386 0:0.9.8b-17.fc8 perlDBD-MySQL.i386 0:4.005-2.fc8.1 perl-DBI.i386 0:1.58-2.fc8 zlib-devel.i386 0:1.2.3-14.fc8 Complete! 2.3 Installing httpd server, tools required for installing REE and subversion (for checking out the code). [root@domU-12-31-39-00-68-93 ~]# yum install gcc-c++ httpd httpd-devel subversion Installed: gcc-c++.i386 0:4.1.2-33 httpd-devel.i386 0:2.2.9-1.fc8 subversion.i386 0:1.4.4-7 Dependency Installed: apr.i386 0:1.2.11-2 apr-devel.i386 0:1.2.11-2 apr-util.i386 0:1.2.10-2.fc8 apr-util-devel.i386 0:1.2.10-2.fc8 cyrus-sasl-devel.i386 0:2.1.22-8.fc8 db4-cxx.i386 0:4.6.21-2.fc8 db4-devel.i386 0:4.6.21-2.fc8 expat-devel.i386 0:2.0.1-2 httpd.i386 0:2.2.9-1.fc8 httpd-tools.i386 0:2.2.9-1.fc8 libstdc++-devel.i386 0:4.1.2-33 neon.i386 0:0.27.2-2 openldap-devel.i386 0:2.3.39-4.fc8 perl-URI.noarch 0:1.35-3.1 Updated: db4.i386 0:4.6.21-2.fc8 openldap.i386 0:2.3.39-4.fc8 Complete! 2.4 Installing REE [root@domU-12-31-39-00-68-93 ~]# mkdir software [root@domU-12-31-39-00-68-93 ~]# cd software/ [root@domU-12-31-39-00-68-93 software]# wget http://rubyforge.org/frs/ download.php/41040/ruby-enterprise-1.8.6-20080810.tar.gz 05:28:36 (3.66 MB/s) - `ruby-enterprise-1.8.6-20080810.tar.gz' saved [6431918/ 6431918] [root@domU-12-31-39-00-68-93 software]# tar xzf rubyenterprise-1.8.6-20080810.tar.gz [root@domU-12-31-39-00-68-93 software]# ./ruby-enterprise-1.8.6-20080810/installer [/opt/ruby-enterprise-1.8.6-20080810] : ## kc - Choose a path - I chose the default as mentioned above. Type 'yes' to all questions. ## kc - You will get a confirmation after REE has been successfully installed.
2.4.1 Uploading the keys to EC2 instance - this will enable you to run AWS commands right from the instance. [root@domU-12-31-39-00-68-93 ~]# mkdir .ec2 # Now go back to a terminal on your laptop/desktop: guest@kc-laptop:~$ scp -i ~/AWS-do-not-delete/id_rsa-gsg-keypair /home/guest/AWSdo-not-delete/EC2-keys/pk-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem
[email protected]: pk-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem 100% 922 0.9KB/s 00:00 guest@kc-laptop:~$ scp -i ~/AWS-do-not-delete/id_rsa-gsg-keypair /home/guest/AWSdo-not-delete/EC2-keys/cert-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem
[email protected]: cert-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem 100% 916 0.9KB/s 00:00 [root@domU-12-31-39-00-68-93 ~]# mv certIFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem .ec2/ [root@domU-12-31-39-00-68-93 ~]# mv pkIFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem .ec2/ 2.4.2 Install latest and full version of AWS tools on the instance We need Java for the AWS APIs: [root@domU-12-31-39-00-68-93 software]# wget [root@domU-12-31-39-00-68-93 software]# bash jdk-6u10-linux-i586.bin # kc - Java has been installed to /root/software/jdk1.6.0_10/ # - We need to put it into .bash_profile [root@domU-12-31-39-00-68-93 ~]# cd software/ [root@domU-12-31-39-00-68-93 software]# wget http://s3.amazonaws.com/ ec2-downloads/ec2-api-tools.zip [root@domU-12-31-39-00-68-93 software]# unzip -q ec2-api-tools.zip 2.4.3 Set some environment variables for convenience # kc - Note: Because we installed REE and not the native (yum install ruby) we need to make a small modification for ruby-lib-paths # We need to put in some stuff in the bash_profile for some things - these are indicated in the comments below: [root@domU-12-31-39-00-68-93 ~]# vi .bash_profile # kc start - 09Nov08 - for enterprise ruby PATH=/opt/ruby-enterprise-1.8.6-20080810/bin:$PATH #http://developer.amazonwebservices.com/connect/thread.jspa?threadID=12491 # Because we have REE instead of 'yum install ruby' export RUBYLIB=/usr/lib/site_ruby # kc end - 09Nov08 - for enterprise ruby # kc start - 09Nov08 - for EC2 tools export EC2_HOME=/root/software/ec2-api-tools-1.3-26369 export EC2_PRIVATE_KEY=~/.ec2/pk-5YPLCLZG2ZBEKNWCTWESOHEOMACLCZTN.pem export EC2_CERT=~/.ec2/cert-5YPLCLZG2ZBEKNWCTWESOHEOMACLCZTN.pem PATH=$PATH:$EC2_HOME/bin
export JAVA_HOME=/root/software/jdk1.6.0_10/jre # kc start - 09Nov08 - for EC2 tools [root@domU-12-31-39-00-68-93 ~]# source .bash_profile [root@domU-12-31-39-00-68-93 software]# source ~/.bash_profile 2.4.4 Installing Passenger and configuring it for the previously installed http server [root@domU-12-31-39-00-68-93 ~]# gem install --no-ri --no-rdoc passenger Building native extensions. This could take a while... Successfully installed passenger-2.0.3 1 gem installed [root@domU-12-31-39-00-68-93 ~]# passenger-install-apache2-module # kc - accept default values and let the installation finish - It will take some time so be patient. 2.4.5 Installing some ruby gems and related software # - strictly speaking this is not necessary but some gems check for availablity of rails-2.0.2 before installing so we might as well get it installed. Especially ferret checks for rails 2.0.2 before it starts. [root@domU-12-31-39-00-68-93 tripladder]# gem install --no-ri --no-rdoc rails v=2.0.2 Bulk updating Gem source index for: http://gems.rubyforge.org Successfully installed rails-2.0.2 Successfully installed rake-0.8.2 Successfully installed activesupport-2.0.2 Successfully installed activerecord-2.0.2 Successfully installed actionpack-2.0.2 Successfully installed actionmailer-2.0.2 Successfully installed activeresource-2.0.2 All below gems are required for our app - You may install based on your application: [root@domU-12-31-39-00-68-93 ~]# gem install --no-rdoc --no-ri actionwebservice v=1.2.6 Successfully installed activesupport-1.4.4 Successfully installed actionpack-1.13.6 Successfully installed activerecord-1.15.6 Successfully installed actionwebservice-1.2.6 4 gems installed [root@domU-12-31-39-00-68-93 ~]# gem install --no-rdoc --no-ri ferret -v=0.11.4 Building native extensions. This could take a while... Successfully installed ferret-0.11.4 1 gem installed [root@domU-12-31-39-00-68-93 tripladder]# gem install --no-rdoc --no-ri acts_as_ferret -v=0.4.3 Successfully installed acts_as_ferret-0.4.3 1 gem installed 2.5.6 Installing Imagemagick and rmagick - This step gave me a lot of trouble - do these with a lot of care # See this tutorial: http://rmagick.rubyforge.org/install-faq.html [root@domU-12-31-39-00-68-93 software]# yum install ghostscript-devel ghostscript
ghostscript-font Installed: ghostscript-devel.i386 0:8.63-1.fc8 Dependency Installed: ghostscript.i386 0:8.63-1.fc8 ghostscript-fonts.noarch 0:5.50-18.fc8 jasper-libs.i386 0:1.900.1-7.fc8 libXfont.i386 0:1.3.1-2.fc8 libfontenc.i386 0:1.0.4-4.fc8 urw-fonts.noarch 0:2.4-3.fc8 xorg-x11-font-utils.i386 1:7.2-2.fc8 Complete! [root@domU-12-31-39-00-68-93 ~]# yum install perl perl-devel *ghostscript* *freetype* *jpeg* *png* *wmf* *tiff* *ghostscript* *lcms* *libexif* * libxml* *zlib* *bzip* # - There will be lot of activity - just relax and accept all options. # - Now get the exact version of Imagemagick and copy it to your 'software' directory. [root@domU-12-31-39-00-68-93 ImageMagick-6.4.1-0.tar.gz ./
[email protected]'s password: ImageMagick-6.4.1-0.tar.gz [root@domU-12-31-39-00-68-93 [root@domU-12-31-39-00-68-93 [root@domU-12-31-39-00-68-93
software]# scp root@:software/ 100% 11MB 1.2MB/s 00:09 software]# tar xzf ImageMagick-6.4.1-0.tar.gz software]# cd ImageMagick-6.4.1 ImageMagick-6.4.1]# ./configure
# kc - *Lots* of activity..... [root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# make install # kc - *Lots* of activity..... [root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# ln -s /usr/local/lib/* /usr/lib/ ln: creating symbolic link `/usr/lib/pkgconfig': File exists [root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# mv /usr/lib/pkgconfig/ /root/ software/original_pkgcongi_09Nov08 [root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# ln -s /usr/local/lib/pkgconfig/ /usr/lib/ [root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# gem install --no-rdoc --no-ri rmagick -v=2.3.0 Building native extensions. This could take a while... Successfully installed rmagick-2.3.0 1 gem installed
3. Creating and attaching external EBS volume and making it usable 3.1 Creating a volume guest@kc-laptop:~$ ec2-create-volume -s 20 -z us-east-1a VOLUME vol- 20 us-east-1a creating 2008-09-22T10:29:36+0000 guest@kc-laptop:~$ ec2-describe-volumes vol- VOLUME vol- 20 us-east-1a available 2008-09-22T10:29:36+0000 3.2 Attaching the volume to our instance guest@kc-laptop:~$ ec2-attach-volume vol- -i -d /dev/ sdh ATTACHMENT vol- /dev/sdh attaching 2008-09-22T10:32:38+0000 guest@kc-laptop:~$ ec2-describe-volumes
VOLUME vol-8105e0e8 20 2008-09-22T10:29:36+0000 ATTACHMENT vol- 2008-09-22T10:32:38+0000
us-east-1a
in-use
/dev/sdh
attached
3.3Making it usable [root@domU-12-31-39-00-65-E3 ~]# yes | mkfs -t ext3 /dev/sdh mke2fs 1.40.4 (31-Dec-2007) /dev/sdh is entire device, not just one partition! Proceed anyway? (y,n) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 2621440 inodes, 5242880 blocks 262144 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=0 160 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000 Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 26 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. [root@domU-12-31-39-00-65-E3 ~]# mkdir -p /mnt/data-store [root@domU-12-31-39-00-65-E3 ~]# mount /dev/sdh /mnt/data-store/
4. Ensuring that mysql uses EBS [root@domU-12-31-39-00-68-93 tripladder]# rmdir /var/lib/mysql/ [root@domU-12-31-39-00-68-93 tripladder]# mkdir -p /mnt/data-store/ new_mysql_store/ [root@domU-12-31-39-00-68-93 tripladder]# ln -s /mnt/data-store/new_mysql_store/ /var/lib/mysql/ [root@domU-12-31-39-02-68-78 tripladder]# chown -R mysql:mysql /mnt/data-store/ new_mysql_store/ [root@domU-12-31-39-00-65-E3 ~]# service mysqld start Initializing MySQL database: Installing MySQL system tables... OK ... [root@domU-12-31-39-00-68-93 tripladder]# ls /mnt/data-store/new_mysql_store/ ibdata1 ib_logfile0 ib_logfile1 mysql mysql.sock ......... 4.1 secure your mysql installation [root@domU-12-31-39-00-65-E3 ~]# mysql_secure_installation 4.2 Create your DB
# - Now create your DB, Db-user and access privileges to the db-user. # - You will have to do something like this: [root@domU-12-31-39-00-68-93 tripladder]# mysql -u root -p mysql> create database ; Query OK, 1 row affected (0.01 sec) mysql> grant all on .* to ''@'localhost' identified by ''; Query OK, 0 rows affected (0.00 sec) 4.3 Creating a schema or restoring from backup. (Optional) You can even do this later. # after creating the database, you will have to create the schema and seed data You can do this: (Take care to edit the config/database.yml file as shown in the next section before you run rake) [root@domU-12-31-39-00-68-93 tripladder]# rake db:migrate RAILS_ENV=production OR [root@domU-12-31-39-00-68-93 tripladder]# mysql -u -p < /backup.sql 4.4 Stop the mysqld [root@domU-12-31-39-00-68-93 tripladder]# service mysqld stop
5. Installing the application [root@domU-12-31-39-00-68-93 ~]# cd /var/www/html/ [root@domU-12-31-39-00-68-93 html]# svn co https:///svn/tripladder/ 5.1 Configuring the database access credentials [root@domU-12-31-39-00-68-93 html]# cd tripladder/ [root@domU-12-31-39-00-68-93 tripladder]# vi config/database.yml production: adapter: mysql database: username: password: host: localhost 5.2 Configuring some stuff in environment.rb # kc - Note - these configs are spread across the whole file. I am listing out the sections that need changing. # - kc Note - Tripladder uses google apps for Intranet and email. [root@domU-12-31-39-00-68-93 tripladder]# vi config/environment.rb ENV['RAILS_ENV'] ||= 'production' RAILS_GEM_VERSION = '2.0.2' unless defined? RAILS_GEM_VERSION
# kc start - use google to send our email - 23Sep08 ActionMailer::Base.smtp_settings = { :tls => true, :address => "smtp.gmail.com", :port => "587", :domain => "tripladder.com", :authentication => :plain, :user_name => "@tripladder.com", :password => "" } # kc end - use google to send our email - 23Sep08 5.3 Configuring stuff for ferret # kc - Ferret will give some trouble unless all related directories have the right ownerships/permissions. You may need to iterate here a bit. 5.3.1 Ferret configurations [root@domU-12-31-39-00-68-93 tripladder]# vi config/ferret_server.yml production: host: localhost port: 9010 pid_file: log/ferret.pid log_file: log/ferret_server.log log_level: warn # kc - Note - Also comment out rest of the file. [root@domU-12-31-39-00-68-93 tripladder]# vi config/environments/ ferret_environment.rb INDEX = Index::Index.new(:path => '/var/www/html/tripladder/index') # - kc - Now we will commit all these files to subversion so that we don't have to keep worrying about this during svn updates (Developers will have to take care to change relevant sections. [root@domU-12-31-39-00-68-93 tripladder]# svn commit config -m "From production server - kc" Sending config/database.yml Sending config/environment.rb Sending config/environments/ferret_environment.rb Sending config/ferret_server.yml Transmitting file data .... Committed revision 997. 5.3.2 Configuring ferret init scripts # kc - Create this file and paste the contents (make changes as per your setup) [root@domU-12-31-39-00-68-93 tripladder]# vi script/ferret_init_production # kc - end of content for the file - Do not copy this line #!/bin/bash # # This script starts and stops the ferret DRb server # chkconfig: 2345 89 36 # description: Ferret search engine for ruby apps. #
# save the current directory CURDIR=`pwd` PATH=/usr/local/bin:/opt/ruby-enterprise-1.8.6-20080810/bin:$PATH RORPATH="/var/www/html/tripladder" case "$1" in start) cd $RORPATH echo "Starting ferret DRb server." FERRET_USE_LOCAL_INDEX=1 \ script/ferret_server -e production start chmod a+rwx /var/www/html/tripladder/log/ferret.pid # we need this for to allow some manipulations ;; stop) cd $RORPATH echo "Stopping ferret DRb server." FERRET_USE_LOCAL_INDEX=1 \ script/ferret_server -e production stop ;; *) echo $"Usage: $0 {start, stop}" exit 1 ;; esac cd $CURDIR # kc - end of content for the file- Do not copy this line [root@domU-12-31-39-00-68-93 tripladder]# chmod a+x script/ferret_init_production [root@domU-12-31-39-00-68-93 tripladder]# svn add script/ferret_init_production A script/ferret_init_production [root@domU-12-31-39-00-68-93 tripladder]# svn commit script -m "From production server - kc" Adding script/ferret_init_production Transmitting file data . Committed revision 998. [root@domU-12-31-39-00-68-93 tripladder]# ln -s /var/www/html/tripladder/script/ ferret_init_production /etc/init.d/ [root@domU-12-31-39-00-68-93 tripladder]# chmod -R a+rwx log/ [root@domU-12-31-39-00-68-93 tripladder]# chmod -R a+rwx tmp/ # -Start - section is optional - you will need to check if your dir structure already has 'index' directory required by ferret (ferret_environment.rb) . [root@domU-12-31-39-00-68-93 tripladder]# mkdir index [root@domU-12-31-39-00-68-93 tripladder]# chmod -R a+rwx index/ # - End - section is optional - you will need to check if your dir structure already has 'index' directory. [root@domU-12-31-39-00-68-93 software]# service ferret_init_production start Starting ferret DRb server. Install the ruby-openid gem to enable OpenID support starting ferret server... [root@domU-12-31-39-00-68-93 software]# service ferret_init_production stop
Stopping ferret DRb server. Install the ruby-openid gem to enable OpenID support stopping ferret server... process 1723 has stopped # kc - If you have problems here you will need to check for /log/ ferret.log(or ferret.out) files for errors and fix them accordingly. # - Usually we will have problems with ownership or permission for ferret log files - 'log' directory and 'index' directory in ROR-Home.
6. Configuring Apache and getting it to serve the application 6.1 Getting Apache introduced to REE and Passenger [root@domU-12-31-39-00-68-93 tripladder]# vi /etc/httpd/conf/httpd.conf # kc start For Phusion and enterprise Ruby - 09Nov08 LoadModule passenger_module /opt/ruby-enterprise-1.8.6-20080810/lib/ruby/gems/ 1.8/gems/passenger-2.0.3/ext/apache2/mod_passenger.so PassengerRoot /opt/ruby-enterprise-1.8.6-20080810/lib/ruby/gems/1.8/gems/ passenger-2.0.3 PassengerRuby /opt/ruby-enterprise-1.8.6-20080810/bin/ruby # kc end For Phusion and enterprise Ruby - 09Nov08 6.2 Getting Apache to recognise our application as well as some REE/Passenger config parameters # kc start - Deploying tripladder app RailsEnv production PassengerMaxPoolSize 26 PassengerLogLevel 0 PassengerMaxInstancesPerApp 0 PassengerPoolIdleTime 86400 RailsSpawnMethod smart ServerName www.tripladder.com DocumentRoot /var/www/html/tripladder/public # kc - to redirect tripladder.com to www.tripladder.com RewriteEngine on RewriteCond %{HTTP_HOST} ^tripladder\.com RewriteRule ^(.*)$ http://www.tripladder.com/$1 [R=permanent,L] # kc - to redirect tripladder.com to www.tripladder.com # kc end - Deploying tripladder app 6.3 Deploying the app and checking that its working [root@domU-12-31-39-00-68-93 tripladder]# service mysqld start Starting MySQL: [ OK ] [root@domU-12-31-39-00-68-93 tripladder]# service httpd restart Stopping httpd: [FAILED] Starting httpd: [ OK ] [root@domU-12-31-39-00-68-93 software]# service ferret_init_production start Starting ferret DRb server. Install the ruby-openid gem to enable OpenID support
starting ferret server... # - Check that the application is accessible from the URL or the IP address.
7. Setting up periodic backups, cleaning up and miscellaneous activities 7.1 Ensure that you have full version of AWS tools (Section 2.4.2) 7.2 Set a cron task to take periodic backups of mysql # - Create a directory on EBS for your mysql dumps. [root@domU-12-31-39-00-68-93 ~]# mkdir -p /mnt/data-store/mysql_backups/ [root@domU-12-31-39-00-68-93 ~]# chmod -R a+rw /mnt/data-store/mysql_backups/ # Check that mysqldump works manually before you put it in cron. [root@domU-12-31-39-00-68-93 ~]# crontab -e 2 0 * * * mysqldump --user=root --password= -single-transaction > /mnt/data-store/mysql_backups/ database_name.sql 7.3 Use logrotate to clean up older mysqld dumps as well as RoR log files. # kc - Note - I have elected to have full backups everyday and rotate them every 30 days. i.e. I can go back-data upto 30 days if I want to restore mysql. [root@domU-12-31-39-00-68-93 ~]# vi /etc/logrotate.conf # kc - 09Nov08 - for rotating rails logs /var/www/html/tripladder/log/*.log { size=10M missingok rotate 4 compress delaycompress notifempty copytruncate } # kc - 09Nov08 - for rotating rails logs # kc - 09Nov08 - for rotating mysqldumps /mnt/data-store/mysql_backups/tripladder_prod.sql { daily rotate 30 compress delaycompress notifempty notifempty copytruncate } # kc - 09Nov08 - for rotating mysql dumps 7.4 Use EBS snapshot facility to automate backups. [root@domU-12-31-39-02-68-78 ~]# vi software/backup_script.sh #!/bin/bash
export EC2_PRIVATE_KEY=$(echo /root/.ec2/pk-xxxxxxxxxx.pem) export EC2_CERT=$(echo /root/.ec2/cert-xxxxxxxxxxx.pem) export EC2_HOME=/root/software/ec2-api-tools-1.3-26369 export PATH=$PATH:$EC2_HOME/bin export JAVA_HOME=/root/software/jdk1.6.0_10/jre /root/software/ec2-api-tools-1.3-26369/bin/ec2-create-snapshot vol- # - test that this is working!!! [root@domU-12-31-39-00-68-93 ~]# chmod a+x software/backup_script.sh [root@domU-12-31-39-00-68-93 ~]# ./software/backup_script.sh SNAPSHOT snap-d55cbdbc vol-8105e0e8 pending 2008-11-11T10:56:10+0000 # - kc - now we put this in the crontab. [root@domU-12-31-39-00-68-93 ~]# crontab -e 3 0 * * * /root/software/backup_script.sh crontab: installing new crontab # kc - start - Important # We save all user uploaded data to 2 location - 1st to a directory in the RoR dir structure and 2nd to a directory on the EBS. # The rails app access this data from EC2 local storage. # If ever the instance dies, we copy the data from EBS to associated directory on directory structure. Now why don't we link the RoR pubic directory to EBS is because EBS has some # price attached to disk access. I could not calculate this with any conviction and hence decided not to use the disk for reads. The EBS is only used for writes for user data. # Having user data like this reduces your disk access costs. Please look at this tutorial to see how we achieved this in rails - http://docs.google.com/ Doc?id=dckr2rnx_5ghz54qcj # Since Mysql can't have 2 data dirs, we host the mysql data directory on EBS. # kc - end - Important 7.5 Some apache optimisations [root@domU-12-31-39-00-68-93 tripladder]# vi public/.htaccess # kc start - enable browser caching - 24Sep08 ExpiresActive On ExpiresDefault "access plus 1 seconds" ExpiresByType text/html "access plus 1 seconds" ExpiresByType image/gif "access plus 120 minutes" ExpiresByType image/jpeg "access plus 120 minutes" ExpiresByType image/png "access plus 120 minutes" ExpiresByType text/css "access plus 60 minutes" ExpiresByType text/javascript "access plus 60 minutes" ExpiresByType application/x-javascript "access plus 60 minutes" ExpiresByType text/xml "access plus 60 minutes" [root@domU-12-31-39-00-68-93 tripladder]# svn commit public/.htaccess -m "From production server" Sending public/.htaccess Transmitting file data . Committed revision 1000. # - Get Apache to read this by restarting the application:
[root@domU-12-31-39-00-68-93 tripladder]# touch tmp/restart.txt -- OR -[root@domU-12-31-39-00-68-93 tripladder]# service httpd restart 7.6 We need to be able to use rake to clear caches else this will become a major headache when we update to new versions. [root@domU-12-31-39-00-68-93 Stopping httpd: [root@domU-12-31-39-00-68-93 [root@domU-12-31-39-00-68-93 [root@domU-12-31-39-00-68-93 [root@domU-12-31-39-00-68-93 [root@domU-12-31-39-00-68-93 [root@domU-12-31-39-00-68-93 D cache [root@domU-12-31-39-00-68-93 public/cache" Deleting public/cache
tripladder]# service httpd stop [ OK ] tripladder]# rm -f public/cache/*cache tripladder]# ls public/cache/ tripladder]# tripladder]# cd public/cache/ cache]# cd .. public]# svn del cache public]# svn commit -m "For symlinking tmp/cache to
Committed revision 1041. [root@domU-12-31-39-00-68-93 public]# ln -s /var/www/html/tripladder/tmp/cache/ /var/www/html/tripladder/public/ [root@domU-12-31-39-00-68-93 public]# chmod a+rwx /var/www/html/tripladder/tmp/ cache [root@domU-12-31-39-00-68-93 public]# service httpd start Starting httpd: [ OK ] # - KC now to test the rake task to clear cache [root@domU-12-31-39-00-68-93 tripladder]# ls public/cache/ application_layout_header.cache homepage_meta.cache homepage.cache send_review_to_freind.cache homepage_header.cache [root@domU-12-31-39-00-68-93 tripladder]# ls tmp/cache/ application_layout_header.cache homepage_meta.cache homepage.cache send_review_to_freind.cache homepage_header.cache [root@domU-12-31-39-00-68-93 tripladder]# rake tmp:clear (in /var/www/html/tripladder) [root@domU-12-31-39-00-68-93 tripladder]# ls tmp/cache/ [root@domU-12-31-39-00-68-93 tripladder]# ls public/cache/ # kc - Confirmed that rake works.
8. Bundling the image 8.1 First stop all running services Now stop all services: [root@domU-12-31-39-00-68-93 tripladder]# service httpd stop Stopping httpd: [ OK ] [root@domU-12-31-39-00-68-93 software]# service ferret_init_production stop Stopping ferret DRb server. Install the ruby-openid gem to enable OpenID support stopping ferret server... process 1741 has stopped
[root@domU-12-31-39-00-68-93 software]# service mysqld stop Stopping MySQL: [ OK ] 8.2 Ensure that services do not start when system reboots -
have to attach EBS before we start the services.
This is important as we
[root@domU-12-31-39-00-68-93 software]# chkconfig --list httpd httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off [root@domU-12-31-39-00-68-93 software]# chkconfig --list mysqld mysqld 0:off 1:off 2:off 3:off 4:off 5:off 6:off [root@domU-12-31-39-00-68-93 software]# chkconfig --list ferret_init_production service ferret_init_production supports chkconfig, but is not referenced in any runlevel (run 'chkconfig --add ferret_init_production') [root@domU-12-31-39-00-68-93 software]# chkconfig --add ferret_init_production [root@domU-12-31-39-00-68-93 software]# chkconfig --list ferret_init_production ferret_init_production 0:off 1:off 2:on 3:on 4:on 5:on 6:off [root@domU-12-31-39-00-68-93 software]# chkconfig ferret_init_production off [root@domU-12-31-39-00-68-93 software]# chkconfig --list ferret_init_production ferret_init_production 0:off 1:off 2:off 3:off 4:off 5:off 6:off 8.3 Bundle the volume and save it to S3. [root@domU-12-31-39-00-68-93 ~]# ec2-bundle-vol -d /mnt/ -k .ec2/pk.pem -c .ec2/cert-.pem -u Please specify a value for arch [i386]: ....# Lots of activity [root@domU-12-31-39-00-68-93 ~]# ec2-upload-bundle -b tripladder-09Nov08 -m /mnt/image.manifest.xml -a -s guest@kc-laptop:~$ ec2-register tripladder-09Nov08/image.manifest.xml IMAGE ami-3c15f155 8.4 Cleaning up # kc - The AWS bundle tools seem to have memory leak - The used memory never comes back to normal and seems to be using up ~ 1 GB of RAM [root@domU-12-31-39-00-68-93 ~]# umount /mnt/data-store/ [root@domU-12-31-39-00-68-93 ~]# reboot
9. Bringing up the application after reboot. [root@domU-12-31-39-00-68-93 ~]# mount /dev/sdh /mnt/data-store/ [root@domU-12-31-39-00-68-93 ~]# service mysqld start Starting MySQL: [ OK ] [root@domU-12-31-39-00-68-93 ~]# service ferret_init_production start Starting ferret DRb server. Install the ruby-openid gem to enable OpenID support starting ferret server... [root@domU-12-31-39-00-68-93 ~]# service httpd start Starting httpd: [ OK ] # kc - Site should be running perfectly.
10. Bringing up the application from AMI
If ever your instance is killed, just do the following: 1. Launch your bundled instance wih the AMI id you got when you registered your bundle, 2. Attach the EBS volume, 3. Associate the IP address 4. Start services - mysql, ferret and httpd. 5. Presto!!! # - Important - NEVER put the mount option for EBS in fstab. When the system is brought up from AMI, the EBS volume is still not attached and hence will not be mounted. This will not allow the system to boot. You AMI will essentially be useless. WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG !!!!!!!!!!!!!!!! [root@domU-12-31-39-00-65-E3 ~]# vi /etc/fstab /dev/sdh /mnt/data-store ext3 defaults,noatime 0 2 # - Amazon is working on attaching a volume to EC2 instance during launch - when this feature is available, we can use fstab.... WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG !!!!!!!!!!!!!!!
