Adam

Tweetfactor – An experiment using Ruby, Sinatra, Twitter and Redis

by Adam on December 8, 2009

in Code

I stumbled across a blog post by digitalhobbit about building a little Sinatra App on top of the Twitter Streaming API.

I followed that guide and made the core of Tweetfactor a couple of weeks ago. Actually getting the thing deployed, however, was pretty involved for me, having never used Rack/Passenger/God etc. etc.

So the following are hopefully all the steps I took, and the appropriate config files to get stuff up and running:

  1. So first off, I didn’t even have anywhere to host this app so I signed up to a 256 VPS from webbynode and then created an Ubuntu Rails Ready Stack (basically a one click install for an Rails Ready Ubuntu system).
  2. Then I added a user and setup SSH and pub/priv keys from my laptop to the VPS.
  3. Then I installed Capistrano – sudo gem install capistrano – for one click deploys of the app to the server. My code is on github in a private repo, so you’ll probably have to look through the Capistrano docs to edit this to your needs:


    #========================
    #CONFIG
    #========================
    default_run_options[:pty] = true
    set :application, "tweetfactor.co.uk"
    set :scm, :git
    set :git_enable_submodules, 1
    set :repository, "git@github.com:adamtaylor/Tweetfactor.git"
    set :branch, "master"
    set :ssh_options, { :forward_agent => true }
    set :stage, :production
    set :user, "xxxx"
    set :scm_passphrase, "xxxxxxx"
    set :use_sudo, false
    set :runner, "deploy"
    set :deploy_to, "/var/www/#{application}"
    set :app_server, :passenger
    set :domain, "www.tweetfactor.co.uk"
    #========================
    #ROLES
    #========================
    role :app, domain
    role :web, domain
    role :db, domain, :primary => true
    #========================
    #CUSTOM
    #========================
    namespace :deploy do
    task :start, :roles => :app do
    run "touch #{current_release}/tmp/restart.txt"
    end
    task :stop, :roles => :app do
    # Do nothing.
    end
    desc "Restart Application"
    task :restart, :roles => :app do
    run "touch #{current_release}/tmp/restart.txt"
    end
    end

  4. Then I had to download god, for process monitoring: sudo gem install god.
  5. When I setup my Rails Ready Stack I told it to use Ruby Enterprise Edition, which means that my gems were installed in a funny place which I had to add to my $PATH.
  6. With the help of this post I wrote a god init.d script as follows:


    #!/bin/sh

    ### BEGIN INIT INFO
    # Provides: god
    # Required-Start: $all
    # Required-Stop: $all
    # Default-Start: 2 3 4 5
    # Default-Stop: 0 1 6
    # Short-Description: God
    ### END INIT INFO

    NAME=god
    DESC=god

    set -e

    # Make sure the binary and the config file are present before proceeding
    test -x /opt/ruby-enterprise/bin/god || exit 0

    # Create this file and put in a variable called GOD_CONFIG, pointing to
    # your God configuration file
    # test -f /etc/default/god && . /etc/default/god
    # [ $GOD_CONFIG ] || exit 0

    . /lib/lsb/init-functions

    RETVAL=0

    case "$1" in
    start)
    echo -n "Starting $DESC: "
    #/opt/ruby-enterprise/bin/god -c $GOD_CONFIG -P /var/run/god.pid -l /var/log/god.log
    /opt/ruby-enterprise/bin/god -c /etc/default/god -P /var/run/god.pid -l /var/log/god.log
    RETVAL=$?
    echo "$NAME."
    ;;
    stop)
    echo -n "Stopping $DESC: "
    kill `cat /var/run/god.pid`
    RETVAL=$?
    echo "$NAME."
    ;;
    restart)
    echo -n "Restarting $DESC: "
    kill `cat /var/run/god.pid`
    #/opt/ruby-enterprise/bin/god -c $GOD_CONFIG -P /var/run/god.pid -l /var/log/god.log
    /opt/ruby-enterprise/bin/god -c /etc/default/god -P /var/run/god.pid -l /var/log/god.log
    RETVAL=$?
    echo "$NAME."
    ;;
    status)
    /opt/ruby-enterprise/bin/god status
    RETVAL=$?
    ;;
    *)
    echo "Usage: god {start|stop|restart|status}"
    exit 1
    ;;
    esac

    exit $RETVAL

  7. Then I setup a god config file in /etc/default/god – it has some code duplication so could probably be improved but it seems to work none-the-less:

    # run with: god -c /path/to/redis.god -D

    God.watch do |w|
    w.name = "redis-server"
    w.interval = 30.seconds # default
    w.start = "redis-server /var/www/tweetfactor.co.uk/current/redis.conf"
    w.stop = "kill `cat /var/run/redis.pid`"
    w.restart = "kill `cat /var/run/redis.pid`; redis-server /var/www/tweetfactor.co.uk/current/redis.conf"
    w.start_grace = 10.seconds
    w.restart_grace = 10.seconds
    w.pid_file = "/var/run/redis.pid"

    w.behavior(:clean_pid_file)

    w.start_if do |start|
    start.condition(:process_running) do |c|
    c.interval = 5.seconds
    c.running = false
    end
    end

    w.restart_if do |restart|
    restart.condition(:memory_usage) do |c|
    c.above = 30.megabytes
    c.times = [3, 5] # 3 out of 5 intervals
    end

    restart.condition(:cpu_usage) do |c|
    c.above = 30.percent
    c.times = 5
    end
    end

    # lifecycle
    w.lifecycle do |on|
    on.condition(:flapping) do |c|
    c.to_state = [:start, :restart]
    c.times = 5
    c.within = 5.minute
    c.transition = :unmonitored
    c.retry_in = 10.minutes
    c.retry_times = 5
    c.retry_within = 2.hours
    end
    end
    end

    God.watch do |w|
    w.name = "twitter_filter"
    w.interval = 30.seconds
    w.start = "cd /var/www/tweetfactor.co.uk/current/; ruby twitter_filter.rb start"
    w.stop = "cd /var/www/tweetfactor.co.uk/current/; ruby twitter_filter.rb stop"
    w.start_grace = 10.seconds
    w.restart_grace = 10.seconds
    w.pid_file = "/var/www/tweetfactor/releases/log/twitter_filter.pid"

    w.behavior(:clean_pid_file)

    w.start_if do |start|
    start.condition(:process_running) do |c|
    c.interval = 5.seconds
    c.running = false
    end
    end

    w.restart_if do |restart|
    restart.condition(:memory_usage) do |c|
    c.above = 30.megabytes
    c.times = [3, 5] # 3 out of 5 intervals
    end

    restart.condition(:cpu_usage) do |c|
    c.above = 30.percent
    c.times = 5
    end
    end

    # lifecycle
    w.lifecycle do |on|
    on.condition(:flapping) do |c|
    c.to_state = [:start, :restart]
    c.times = 5
    c.within = 5.minute
    c.transition = :unmonitored
    c.retry_in = 10.minutes
    c.retry_times = 5
    c.retry_within = 2.hours
    end
    end
    end

  8. A quick chmod +x god /etc/init.d/god
  9. And a sudo update-rc.d god defaults to have it added to startup.
  10. I need to add a rackup config file in the root called config.ru so that Nginx/Passenger could handle the Sinatra app:

    require 'rubygems'
    require 'sinatra'

    require 'tweetfactor'

    set :environment, :production

    run Sinatra::Application

  11. The last thing I had to do was modify my twitter_filter.rb from the original blog post so it could be daemonized, I messaged digitalhobbit for some help on this and he sent me a modified version which worked a charm. I also had to sudo gem install daemons

I had a fun few hours learning about a tonne of new stuff and I hope this may help you if you wanna deploy it too.

Any questions, shoot in the comments!

Related posts:

  1. Kill A Process Using It’s PID File

Tagged as: ruby

{ 1 trackback }

Tweets that mention Tweetfactor – An experiment using Ruby, Sinatra, Twitter and Redis -- Topsy.com
December 8, 2009 at 11:21 pm

{ 0 comments… add one now }

Leave a Comment