Autor: talin.senner
Original em plone-relstorage-postgresql-bucardo

Plone && Relstorage && Postgresql && Async Replication

Deploying Plone & Relstorage with Postgresql with replication done by rubyrep, in master/master configuration on Ubuntu 11.10

 So you don't have the cash to pay for ZRS? RubyRep does Master/Master replication. Well it will also do  A ->B  A->C A->D etc...

This will have 1 plone server and two database servers, both having a full copy of the plone data

Plone server

build a Ubuntu machine and deploy out a Plone site however you are comfortable. Now modify it as follows.

Relstorage

make sure

sudo apt-get install libpq-dev

Filesystem to DB BLOBS

To convert from Blobs on filestorage to Postgres:

it requires the blob information from your zope.conf

zodbconvert.cfg

<filestorage source>
 path /opt/plone/zeocluster/var/filestorage.bkp/Data.fs
 blob-dir /opt/plone/zeocluster/var/blobstorage.bkp
</filestorage>

 <relstorage destination>
 blob-dir /tmp/blobcache
 shared-blob-dir false
 <postgresql>
 dsn dbname='plonedb1' user='plone' host='10.0.x.x' password='passwordhere'
 </postgresql>
</relstorage>

Postgresql

build two ubuntu machines then upgrade ubuntu to 11.10 via do-release-upgrade

 sudo su

 apt-get install postgresql-9.1 postgresql-plperl postgresql-client ssl-cert libsys-hostname-long-perl libsys-syslog-perl libboolean-perl libdbix-safe-perl libdbd-pg-perl postgresql-contrib

make sure postgresql.conf is allowing TCP/IP connections by modifying

vim /etc/postgresql/9.1/main/pg_hba.conf

add to bottom:

 host plonedb1 all 0.0.0.0 0.0.0.0 md5

RubyRep Installation 

sudo apt-get install gawk unzip wget

wget http://rubyforge.org/frs/download.php/74408/rubyrep-1.2.0.zip

Put the following into file /opt/rubyrep/rubyrep.conf

PROXY=0 
CONFIG="plonedbsync.conf" 
RUBYREP="/opt/rubyrep/rubyrep"

 
mkdir /var/log/rubyrep
touch /var/log/rubyrep/replication.log

Put the following into the file /opt/rubyrep/plondbsync.conf

RR::Initializer::run do |config|
 config.left = {
 :adapter  => 'postgresql', 
 :database => 'plonedb1',
 :username => 'replicationuser',
 :password => 'xxxxx',
 :host     => 'x.x.x.b',
 :pg_params => '?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory',


 }

 config.right = {
 :adapter  => 'postgresql',
 :database => 'plonedb1',
 :username => 'replicationuser',
 :password => 'xxxxx',
 :host     => 'x.x.x.a',
 :pg_params => '?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory',


 }

 config.options[:sync_conflict_handling] = :left_wins
 config.options[:replication_conflict_handling] = :left_wins
 config.include_tables /./ # regexp matching all tables in the database
 config.exclude_tables /commit_lock$/

 

Create the file /etc/init.d/rubyrep and add the following:

#!/bin/bash
#
# /etc/init.d/rubyrep
#
# Written by David Post
# Functions provided by this file
# * Start
# * Stop
# * Restart
# * Sync
# * Status
# Configuration for use with this script should be kept in /etc/rubyrep/rubyrep.conf
# The only configuration that I am writing right now is proxy, conf and rubyrep.
if [ -f /opt/rubyrep/rubyrep.conf ]
then
 . /opt/rubyrep/rubyrep.conf
fi
if [ ! -n "$CONFIG" ]
then
 echo "No config file specified in rubyrep.conf"
 exit 1
fi
[ -r /opt/rubyrep/"$CONFIG" ] || { echo "File not found: $CONFIG"; exit 1; }
CONFIG=/opt/rubyrep/"$CONFIG"
# These are some functions to make this check if RubyRep is running before starting it.
# This checks if it is running or not
# RUN = is it running proxy or replicate, this may be different than the configuration file says. e.g.
# The just changed the config file
function test_running(){
 PID=""
 if (("$PROXY"))
 then
 PID=`ps -Af | grep rubyrep| grep proxy | grep java |gawk '{print $2}'`
 if [ -n "$PID" ]
 then
 RUN=P
 else
 PID=`ps -Af | grep rubyrep| grep replicate | grep java |gawk '{print $2}'`
 [ ! -n "$PID" ] || RUN=R
 fi
 else
 PID=`ps -Af | grep rubyrep| grep replicate | grep java |gawk '{print $2}'`
 if [ -n "$PID" ]
 then
 RUN=R
 else
 PID=`ps -Af | grep rubyrep| grep proxy | grep java |gawk '{print $2}'`
 [ ! -n "$PID" ] || RUN=P
 fi
 fi

}

case "$1" in
 start)
 test_running
 if [ -n "$PID" ]
 then
 echo "RubyRep already running."
 exit 0
 fi
 if (("$PROXY"))
 then
 echo "Starting RubyRep proxy"
 $RUBYREP proxy 2>&1 | gawk '{print strftime("%Y-%m-% %T",systime()), $0; fflush();}' >> /var/log/rubyrep/proxy.log &
 else
 echo "Starting rubyrep replication"
 $RUBYREP replicate -c $CONFIG 2>&1 | gawk '{print strftime("%Y-%m-%d %T",systime()), $0; fflush();}' >> /var/log/rubyrep/replication.log &
 fi
 ;;
 stop)
 test_running
 case "$RUN" in
 P)
 echo "Stopping RubyRep proxy"
 kill `ps -Af | grep rubyrep | grep proxy | grep java | gawk '{print $2}'` > /dev/null
 ;;
 R)
 echo "Stopping RubyRep replication"
 kill `ps -Af | grep rubyrep | grep replicate | grep java | gawk '{print $2}'` > /dev/null
 ;;
 *)
 echo "RubyRep not running"
 ;;
 esac
 ;;
 restart)
 test_running
 if [ -n "$PID" ]
 then
 $0 stop
 sleep 1
 fi
 $0 start
 ;;
 sync)
 echo "Syncing databases..."
 /opt/rubyrep/rubyrep "sync -c $CONFIG"
 ;;
 status)
 if (("$PROXY"))
 then
 echo -e "RubyRep Proxy is \\c"
 ps -Af | grep rubyrep| grep proxy | grep java >/dev/null || echo -e not \\c
 else
 echo -e "RubyRep Replication is \\c"
 ps -Af | grep rubyrep| grep replicate | grep java >/dev/null || echo -e not \\c
 fi
 echo running
 ;;
 test) # This ones sort of a secret that I used for testing if the function test_running() was working or not
 test_running
 if [ -n "$PID" ]
 then
 echo "RubyRep is running on PID $PID"
 else
 echo "RubyRep is not running"
 fi
 ;;

 *)
 echo "Usage: $0 {start|stop|restart|sync|status}"
 ;;
esac
exit 0 

---

java -Djruby.memory.max=1280m -Djruby.stack.max=2048k -Xmx1280m -Xms256m -Xss2048k -Djffi.boot.library.path=/opt/rubyrep/jruby/lib/native/i386-Linux:/opt/rubyrep/jruby/lib/native/ppc-Linux:/opt/rubyrep/jruby/lib/native/x86_64-Linux -Djava.awt.headless=true -Xbootclasspath/a:/opt/rubyrep/jruby/lib/jruby.jar -classpath : -Djruby.home=/opt/rubyrep/jruby -Djruby.lib=/opt/rubyrep/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /opt/rubyrep/jruby/bin/jgem install activerecord activerecord-jdbc-adapter

Make sure both databases that you are syncing have the schema loaded.  This is easiest with a zodbconvert to both databases at the same time, but a schema export from one database without data will work as well.

Referências

https://www.wildcardcorp.com/blog/plone-relstorage-postgresql-bucardo