1. Install log rotate:
#>yum install logrotate

2. Create the configuration file (refer below for the complete list of options):
#>vim /etc/logrotate.d/apache

/usr/local/apache2/logs/*log {
  size 70M
  compress
  dateext
  notifempty
  rotate 7
  missingok
  sharedscripts
  postrotate
    /etc/init.d/apachectl restart
  endscript
}

so basically here we are specifying the next:

• /usr/local/apache2/logs/*log :
•  any of the logs in “/usr/local/apache2/logs/”
• size 70M :
• The files should be rotated when goes bigger than 70M (we can specify based on time, daily, weekly, monthly)
• compress :
• Compress older files with gzip. Opposite: nocompress
• dateext :
•  the date will be append in the new filename
• notifempty :
• Don’t do any rotation if the logfile is empty. Opposite: ifempty
• rotate 7 :
• We should keep no more than nn files
• missingok :
• Do not generate an error if the log is missing
• sharedscripts:
• Run any given prerotate or postrotate script for each logfile individually. Opposite: nosharedscripts.
• postrotate:
• Anything between these is executed after the rotation process. Opposite : prerotate

cd /usr/local/apache2/logs/

logrotate -s logstatus /etc/logrotate.d/apache
ls -lah

-rw-r--r-- 1 root root 4.8K Jul 20 2011 access_log
-rw-r--r-- 1 root root 8.2M May 14 22:16 access_log-20120514.gz
-rw-r--r-- 1 root root 0 Jul 20 2011 jsr-access_log
-rw-r--r-- 1 root root 0 Jul 20 2011 jsr-error_log
-rw-r--r-- 1 root root 647 May 14 22:16 logstatus
-rw-r--r-- 1 root root 0 May 14 22:16 members-access_log
-rw-r--r-- 1 root root 4.8M May 14 22:16 members-access_log-20120514.gz
-rw-r--r-- 1 root root 0 May 14 22:16 members-error_log
-rw-r--r-- 1 root root 1.2M May 14 22:16 members-error_log-20120514.gz


Pretty simple… isn’t it? now we’re all set!

List of available options:

compress: Old versions of log files are compressed with gzip by default. See also nocompress.

compresscmd: Specifies which command to use to compress log files. The default is gzip. See also compress.

uncompresscmd: Specifies which command to use to uncompress log files. The default is gunzip.

compressext: Specifies which extension to use on compressed logfiles, if compression is enabled. The default follows that of the configured compression command.

compressoptions: Command line options may be passed to the compression program, if one is in use. The default, for gzip, is “-9″ (maximum compression).

copy: Make a copy of the log file, but don’t change the original at all. This option can be used, for instance, to make a snapshot of the current log file, or when some other utility needs to truncate or pare the file. When this option is used, the createoption will have no effect, as the old log file stays in place.

copytruncate: Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one, It can be used when some program can not be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the createoption will have no effect, as the old log file stays in place.

create: mode owner groupImmediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated). mode specifies the mode for the log file in octal (the same as chmod(2)), owner specifies the user name who will own the log file, and group specifies the group the log file will belong to. Any of the log file attributes may be omitted, in which case those attributes for the new file will use the same values as the original log file for the omitted attributes. This option can be disabled using the nocreateoption.

daily: Log files are rotated every day.

delaycompress: Postpone compression of the previous log file to the next rotation cycle. This has only effect when used in combination with compress. It can be used when some program can not be told to close its logfile and thus might continue writing to the previous log file for some time.

extension:  extLog files are given the final extension ext after rotation. If compression is used, the compression extension (normally .gz) appears after ext.

ifempty: Rotate the log file even if it is empty, overiding the notifemptyoption (ifempty is the default).

include : file_or_directoryReads the file given as an argument as if it was included inline where the include directive appears. If a directory is given, most of the files in that directory are read in alphabetic order before processing of the including file continues. The only files which are ignored are files which are not regular files (such as directories and named pipes) and files whose names end with one of the taboo extensions, as specified by the tabooext directive. The includedirective may not appear inside of a log file definition.

mail addressWhen a log is rotated out-of-existence, it is mailed to address. If no mail should be generated by a particular log, the nomaildirective may be used.

mailfirstWhen using the mailcommand, mail the just-rotated file, instead of the about-to-expire file.

maillastWhen using the mailcommand, mail the about-to-expire file, instead of the just-rotated file (this is the default).

missingokIf the log file is missing, go on to the next one without issuing an error message. See also nomissingok.

monthlyLog files are rotated the first time logrotateis run in a month (this is normally on the first day of the month).

nocompressOld versions of log files are not compressed with gzip. See also compress.

nocopyDo not copy the original log file and leave it in place. (this overrides the copyoption).

nocopytruncateDo not truncate the original log file in place after creating a copy (this overrides the copytruncateoption).

nocreateNew log files are not created (this overrides the createoption).

nodelaycompressDo not postpone compression of the previous log file to the next rotation cycle (this overrides the delaycompressoption).

nomailDon’t mail old log files to any address.

nomissingokIf a log file does not exist, issue an error. This is the default.

noolddirLogs are rotated in the same directory the log normally resides in (this overrides the olddiroption).

nosharedscriptsRun prerotate and postrotate scripts for every script which is rotated (this is the default, and overrides the sharedscriptsoption).

notifemptyDo not rotate the log if it is empty (this overrides the ifemptyoption).

olddir directoryLogs are moved into directory for rotation. The directory must be on the same physical device as the log file being rotated. When this option is used all old versions of the log end up in directory. This option may be overriden by the noolddiroption.

postrotate/endscriptThe lines between postrotate and endscript (both of which must appear on lines by themselves) are executed after the log file is rotated. These directives may only appear inside of a log file definition. See prerotateas well.

prerotate/endscriptThe lines between prerotate and endscript (both of which must appear on lines by themselves) are executed before the log file is rotated and only if the log will actually be rotated. These directives may only appear inside of a log file definition. See postrotateas well.

firstaction/endscriptThe lines between firstaction and endscript (both of which must appear on lines by themselves) are executed once before all log files that match the wildcarded pattern are rotated, before prerotate script is run and only if at least one log will actually be rotated. These directives may only appear inside of a log file definition. See lastactionas well.

lastaction/endscriptThe lines between lastaction and endscript (both of which must appear on lines by themselves) are executed once after all log files that match the wildcarded pattern are rotated, after postrotate script is run and only if at least one log is rotated. These directives may only appear inside of a log file definition. See lastactionas well.

rotate countLog files are rotated times before being removed or mailed to the address specified in a mail directive. If countis 0, old versions are removed rather then rotated.

size sizeLog files are rotated when they grow bigger then size bytes. If size is followed by M, the size if assumed to be in megabytes. If the k is used, the size is in kilobytes. So size 100, size 100k, and size 100Mare all valid.

sharedscriptsNormally, prescript and postscript scripts are run for each log which is rotated, meaning that a single script may be run multiple times for log file entries which match multiple files (such as the /var/log/news/* example). If sharedscriptis specified, the scripts are only run once, no matter how many logs match the wildcarded pattern. However, if none of the logs in the pattern require rotating, the scripts will not be run at all. This option overrides the nosharedscripts option.

start countThis is the number to use as the base for rotation. For example, if you specify 0, the logs will be created with a .0 extension as they are rotated from the original log files. If you specify 9, log files will be created with a .9, skipping 0-8. Files will still be rotated the number of times specified with the countdirective.

tabooext [+] listThe current taboo extension list is changed (see the includedirective for information on the taboo extensions). If a + precedes the list of extensions, the current taboo extension list is augmented, otherwise it is replaced. At startup, the taboo extension list contains .rpmorig, .rpmsave, ,v, .swp, .rpmnew, and ~.

weeklyLog files are rotated if the current weekday is less then the weekday of the last rotation or if more then a week has passed since the last rotation. This is normally the same as rotating logs on the first day of the week, but it works better if logrotate is not run every night.

Adding volumes to your AMI instances :

Connect the volume to the specific AMI that is running via the console

Then login via SSH into the and create a file system on the newly created volume and then make it available to the AMI.

sudo mkfs.ext4 /dev/xvdf

Then add this to the /etc/fstab

/var/log/btmp

The btmp log keeps track of failed login attempts. I have seen on a default linux setup with logrotate configured where the btmp log is left out of rotation and eventually grows out of hand. So first you want to make sure that the btmp log is rotated using logrotate with the below information.

How to Read btmp Log

last -f /var/log/btmp

/var/log/wtmp

The wtmp file records all logins and logouts history

How to Read wtmp Log

last -f /var/log/wtmp

root pts/1 233-14-150-49-ne Tue Feb 14 17:54 still logged in
root pts/0 233-14-150-49-ne Tue Feb 14 16:00 still logged in
root pts/0 233-14-150-49-ne Mon Feb 13 19:58 – 23:33 (03:34)
root pts/0 233-14-150-49-ne Wed Feb 8 22:21 – 02:23 (04:02)
root pts/0 233-14-150-49-ne Wed Feb 8 22:19 – 22:19 (00:00)
root pts/0 233-14-150-49-ne Wed Jan 25 17:05 – 23:24 (06:19)
root pts/0 233-14-150-49-ne Thu Jan 19 20:23 – 23:22 (02:58)

 

To review recent logins. If the login is from a remote location, it will be associated with a specific IPaddress

utmpdump /var/log/wtmp | less

-Para calzarse la capitanía hay que lisonjear un tanto a los patrones.

-¡Borracho!, eso me gusta; ¡ay de los que no gustan del vino!, tienen algún mal pensamiento, y temen que el vino se lo haga revelar.

-Los franceses tienen sobre los españoles la ventaja de que los españoles piensan y los franceses improvisan.

-El corazón de la mujer es de tal naturaleza que aunque árido y endurecido por las exigencias sociales, siempre guarda un rincón fértil y amable, el que Dios ha consagrado al amor de madre.

-hablad y, sobre todo, comenzad por el principio, porque me gusta el orden en todas las cosas. -Señor

-los reyes de ahora, encerrados en los límites de lo probable, no tienen la audacia de la voluntad, temen el oído que escucha las órdenes que ellos mismos dan, el ojo que ve sus acciones; no sienten en sí lo superior de la esencia divina, son hombres coronados, en una palabra.

-En otro tiempo se creían o a lo menos se decían hijos de Júpiter, y conservaban algo del ser de su padre; que no se plagian fácilmente las cosas de ultra-nubes. Ahora los reyes se hacen muy a menudo vulgares.

-Dantés no conocía más que su pasado, tan breve; su presente, tan sombrío, y su futuro tan dudoso.

-devorándolo como el implacable Ugolino devora el cráneo del arzobispo Roger en el Infierno del Dante.

-Aprender no es saber, de aquí nacen los eruditos y los sabios, la memoria forma a los unos, y la filosofía a los otros.

-La filosofía no se aprende. La filosofía es el matrimonio entre las ciencias y el genio que las aplica. La filosofía es la nube resplandeciente en que puso Dios el pie para subir a la gloria.

-Desengañaos…, sufro menos porque tengo menos fuerzas para sufrir.

-Sed sabio como Néstor, y astuto como Ulises.

-Cucumetto había violado a tu hija -dijo el bandido-, y como yo la amaba más que a mí mismo, la he matado, porque después de él iba a servir de juguete a toda la compañía. »Los labios del anciano no se entreabrieron para murmurar la más mínima palabra, pero su rostro volvióse tan pálido como el de un cadáver. »-Ahora -prosiguió Carlini-, si he hecho mal, véngala. »Y arrancó el cuchillo del seno de la joven, que presentó con una mano al anciano, mientras que con la otra apartaba su camisa y le presentaba su pecho desnudo. »-Has hecho bien -le dijo el anciano con voz sorda-. ¡Abrázame, hijo mío!

-tavolette son unas tabletas de madera que se cuelgan en todas las esquinas de las calles la víspera de las ejecuciones, y en las cuales están escritos los nombres de los condenados, la causa de su condenación y la clase de suplicio. Tienen por objeto invitar a los fieles a que rueguen a Dios para que dé a los culpables un sincero arrepentimiento.

-saliera por la puerta del Popolo, que diese la vuelta por el lado exterior de las murallas y que entrase por la puerta de San Juan

-todo su ser parecía obedecer a un movimiento maquinal en el cual no entraba ya para nada su voluntad.

-no os detengo, caballero -dijo la condesa-, porque no quiero que mi reconocimiento sea indiscreción.

-le miraría como a uno de los personajes de Byron, a quienes la desgracia ha marcado con un sello fatal. Algún Manfredo, algún Lara, algún Werner, como uno de esos restos, en fin, de alguna familia antigua que, desheredados de su fortuna paterna, han encontrado una por la fuerza de su genio aventurero, que les ha hecho superiores a las leyes de la sociedad.

-los malos no mueren así, porque Dios parece protegerlos para hacerlos instrumentos de sus venganzas.

-mis peones de México habrán descubierto alguna mina.

-Patros men aten, ma de onoma prodotu kai prodosiam, eipe emin.

-La vida es tan incierta, que la felicidad debe aprovecharse en el momento en que se presenta.

-Cuando vendiste a tu amigo, empezó Dios, no por castigarte, sino por advertirte. Caíste en la miseria y tuviste hambre, pasaste la mitad de tu vida codiciando lo que hubieras podido adquirir, y ya pensabas en el crimen, dándote a ti mismo la disculpa de la necesidad, cuando Dios obró un milagro, cuando Dios te envió por mi mano, cuando más miserable estabas, una fortuna inmensa para ti, que nada habías poseído. Pero esta fortuna inesperada e inaudita te parece insuficiente desde el momento en que empiezas a poseerla. Quieres doblarla. ¿Y por qué medio? Por el del asesinato. La doblas, pero Dios te la arranca, conduciéndote ante la justicia humana.

-¡Ah!, no crees en Dios, y Dios, que sólo exige una súplica, una palabra, una lágrima para perdonar… Dios, que podía dirigir el puñal del asesino de modo que expirases en el acto…, te concedió un cuarto de hora para arrepentirte… ¡Vuelve en ti, desventurado, y arrepiéntete!

-¡habré trabajado para ser mañana un poco de polvo! No siendo la muerte del cuerpo, esta destrucción del principio vital ¿no es el reposo al cual todos los desgraciados aspiran? Esa tranquilidad de la materia tras la que he suspirado tanto tiempo y a la que me encaminaba por medio del hambre

-¿Qué es la muerte para mí? Uno o dos grados más en el silencio. No, no es la existencia la que lamento perder, es la ruina

-Montecristo imprimió en aquella frente pura y hermosa un beso que hizo latir dos corazones a la vez; el uno con violencia, y el otro sordamente

-Obedeceré si me mandáis que levante la losa que cubre a la hija de Jairo. Caminaré sobre las ondas como el apóstol, si me hacéis señal con la mano de caminar sobre ellas, obedeceré en todo…

-Un hombre del temple del conde de Montecristo no podía estar mucho tiempo sumergido en la melancolía que suele reinar en las almas vulgares, dándoles una originalidad aparente, pero que aniquila las almas superiores.

-¡Cómo!, ¡una hora bastaría para probar al arquitecto que la obra de todas sus esperanzas era, si no imposible, al menos sacrílega!

-hasta el día en que Dios se digne descifrar el porvenir al hombre, toda la sabiduría humana estará resumida en dos palabras: ¡Confiar y esperar!

Q:But how APC does that magic?
A:well the workflow of any php page is:
Whenever a client requests a PHP page, the server will read in the source code of the page, compile it into bytecode and then execute it.

Q:Haaaa so where APC takes action there?
A: APC caches the compiled output of each PHP script run and reuses it for subsequent requests. This reduces the time and processing cycles needed to fully satisfy each request, leading to better performance and lower response times.

Q:Awesome!!! I want it… How can I install it?

A:Installation (with PECL):
APC is a PECL extension and is not bundled with PHP

pecl install apc-3.1.9

Q:But wait… pecl?
A:really? Do I need to explain this?
yum install php-pear
apt-get install php5-pear

Q:sorry!

Installation like real Macho Men:

wget http://pecl.php.net/get/APC-3.1.9.tgz
tar -xvzf APC-3.1.9.tgz
cd APC-3.1.9
phpize
locate php-config
./configure --with-php-config=/usr/local/php/bin/php-config --enable-apc
make
make install

Tthis is the Output:
Installing shared extensions: /usr/local/lib/php/extensions/no-debug-non-zts-20050922/
Installing header files: /usr/local/include/php/
Libraries have been installed in: /opt/apc/APC-3.1.9/modules

Probably you will need to copy the module where php can read it:
cp modules/apc.so /usr/lib/php/modules/

Ok now edit the “php.ini” file and add the module

vim /etc/php.ini

Suggested Configuration (in your php.ini file)
———————————————-
extension=apc.so
apc.enabled=1
apc.shm_size=128M
apc.ttl=7200
apc.user_ttl=7200
apc.enable_cli=1

/etc/init.d/apachectl restart

Q: got it… ok it’s done… but how can I know it worked?
A: create a page in your document root with this:
and name it: phpinfo.php

then open it in a web browser (Please not IE) and look for APC you should have something like this:

Q: Ok is installed now what?
A: Well that’s it APC will be catching the compiled output of php, there is a web page you can use (apc.php) in the source code just copy and past it in your www directory and you can see some metrics there, if you need more info you can check this site: http://devzone.zend.com/article/12618

Q: AWESOME YOU ROCK!!!

ab -n 1000 -c 5 http://example.com/test.php

-n requests: Number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.

-c concurrency: Number of multiple requests to perform at a time. Default is one request at a time.

Here is the output of the command:

This is ApacheBench, Version 2.3
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking example.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests

Server Software: Apache/2.2.19
Server Hostname: example.com
Server Port: 80

Document Path: /test.php
Document Length: 2978 bytes

Concurrency Level: 5
Time taken for tests: 222.898 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 3485000 bytes
HTML transferred: 2978000 bytes
Requests per second: 4.49 [#/sec] (mean)
Time per request: 1114.488 [ms] (mean)
Time per request: 222.898 [ms] (mean, across all concurrent requests)
Transfer rate: 15.27 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 30 912 538.0 889 6864
Processing: 0 198 240.2 108 1008
Waiting: 0 191 240.9 100 1008
Total: 664 1110 456.0 1018 6864

Percentage of the requests served within a certain time (ms)
50% 1018
66% 1082
75% 1125
80% 1156
90% 1303
95% 2067
98% 2424
99% 2899
100% 6864 (longest request)

The main numbers to look at here are the requests per second and the average time per request. The lower the average time per request, the better the performance. Similarly, the greater the number of requests served, the better the performance.

If you want to know more about ab follow this link: http://httpd.apache.org/docs/2.0/programs/ab.html

Saramago, Jose (2011). Cain (Kindle Location 1625). Houghton Mifflin Harcourt. Kindle Edition.

zip -r archivefile2 papers
This copies the directory “papers”, located in the current directory, into “archivefile2.zip”.

zip -r archivefile3 /home/joe/papers
This copies the directory “/home/joe/papers” into “archivefile3.zip”. Since in this case the absolute path is given, it doesn’t matter what the current directory is, except that the zip file will be created there.

Unzip Files
unzip archivefile1.zip
This writes the files extracted from “archivefile1.zip” to the current directory.

TAR Files 

The GNU tar is archiving utility but it can be use to compressing large file(s). GNU tar supports both archive compressing through gzip and bzip2. If you have more than 2 files then it is recommended to use tar instead of gzip or bzip2.
-z: use gzip compress
-j: use bzip2 compress

.tar basically merges multiple files as a single file – there is no compression, but it captures all linux permission settings, etc.

.gz/bz2 provides compression. It can take single or multiple files, but it does not capture permission settings.

To compress a diectory or files with tar.gz
tar -czfv file.tar.gz files/

To extract archives from a file.tar.gz
tar -xvzf file.tar.gz

To compress diectory or files with bz2
tar -jcvf file.tar.bz2 files/

To extract files compressed with bz2
tar -jxvf file.tar.bz2

If you dont want to be asked for your password:

echo 'loginname ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers


So…

[birkoff@hector ~]$ su
Password:

[root@hector birkoff]# chmod +w /etc/sudoers
[root@hector birkoff]# echo 'birkoff ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
[root@hector birkoff]# chmod -w /etc/sudoers
[root@hector birkoff]# exit


Amazon S3 is a service allowing to store and access files at very low cost. It is a viable option as a remote backup server and using the extremely handy s3sync the heavy lifting work has already been done for you. First you’ll need an Amazon S3 account. Once you have that in place to set things up you’ll need your access key ID and your secret access key. To find these go to Your Web Services Account and choose AWS Access Identifiers.
On your server

To use s3sync you need ruby to be installed. I found openssl was already installed on my server but you may need to get that too if you want to use ssl connections (you can use yum for this too).

To get ruby use yum (oh yeah baby red-had based distro centos or fedora):

yum install ruby

Once has installed check the version using

ruby -v

You should see something like

ruby 1.8.7 (2011-06-30 patchlevel 352) [i386-linux]

Download and extract s3sync and then removing the download:

wget http://s3.amazonaws.com/ServEdge_pub/s3sync/s3sync.tar.gz
tar xzf s3sync.tar.gz
rm s3sync.tar.gz

Now you’ll need to set up the configuration with the access keys you have from s3

cd s3sync

# Copy the default configuration to the right location in /etc
# You may need to be root for this

mkdir /etc/s3conf
cp s3config.yml.example /etc/s3conf/s3config.yml

# Edit the file

vi /etc/s3conf/s3config.yml

# Edit the file with the following lines

aws_access_key_id: ------Your Access Key here ------
aws_secret_access_key: ---- Your Secret Access Key here ------
ssl_cert_dir: /home/your-user/s3sync/certs

# Now we need to set up the SSL certificates so we can connect on a secure connection.

mkdir /home/your-user/s3sync/certs
cd /home/your-user/s3sync/certs
wget http://mirbsd.mirsolutions.de/cvs.cgi/~checkout~/src/etc/ssl.certs.shar

# Run the script

sh ssl.certs.shar

Connecting to S3
You should be set up now to access S3. There are two scripts you can use to administer and set up your backups. Both s3sync and s3cmd are well documented at s3sync but I will take you through a basic setup.

First we are going to set up a bucket for this server (as we may wish to back up others in the future).

cd /home/your-user/s3sync

# Create the bucket (add -s to use ssl)

ruby s3cmd.rb createbucket birkoff_content

For this backup I’m going to backup my media directory. Here’s the command I use (I’m still in /home/your-user/s3sync).

ruby s3sync.rb -r -s -v --exclude="cache$|captchas$" --delete /data/media/ birkoff_content:media > /var/log/s3sync

Let’s go through the options

-r
    This tells the script to act recursively including everything in the folder
-s
    This tells the script to use SSL. We certainly want to do this if there is any sensitive information being transmitted and I'd recommend doing this by default anyway.
-v
    This tells the script to be verbose, meaning it should show output all messages to the terminal.
--exclude="cache$|captchas$"
    This tells the script to exclude certain folders or files based on a regular expression. In this example I want to exclude any folders called cache or captchas.
/data/media
    This is path to the folder that you want to back up. Bear in mind that this backs up everything in the folder.
--delete
    This tells the script to delete any obsolete files. So it will remove files you have deleted on your local server from the mirror.
birkoff_content:media
    This is first the bucket that you want to use (this is the one we created earlier), and then the prefix you would like. I'm backing up my media so media is a good one for me (in other manuals they use birkoff_content:/media with a / before the name of the folder, for me this create a no-name dir in the bucket and then inside that folder a "media" directory, so if we remove the / will be good and create a media dir inside the bucket).
> /var/log/s3sync
    This tells the script to log the output into a log file. This is optoinal but I like to keep an eye on things. You'll need
to make sure your user has permissions to write to the file or the script will error. This is crude logging as it will only log the last sync.

You can run the script with a dry run by using the additional –dryrun flag and this will show you everything the script will do without it actually doing it. You can also use the -d flag to debug the script. Depending on the size of your folder syncing can take some time so be patient. That’s it – you now have a remote backup of your files that is likely to cost cents rather than dollars per month. If any files or folers are subsequently removed from or added to /data/media/ when you run the script again your remote copy will be updated to mirror your folder.
Automating the task

To take all the administration out of this task you can automate the backup using cron. First we need to put the command into file so cron can use it.

mkdir /home/your-user/shell_scripts
cd /home/your-user/shell_scripts

# Create and edit the file

vi s3backup.sh

Copy the script you want to run as a cron job into this file, ensuring you specify the full path to your ruby script. Remember to add #!/bin/bash or whichever shell you use at the top of the script.

ruby /home/your-user/s3sync/s3sync.rb -r -s -v --exclude="cache$|captchas$" --delete /data/media/ birkoff_content:media > /var/log/s3sync

Save this file and then set up the cron job

crontab -e

# Add the following line. This runs the backup every Sunday at 6am

0 6 * * 0 /home/your-user/shell_scripts/s3backup.sh

The backup will now run at 6am every Sunday without any further input from you. You can check the script is running ok by checking /var/log/s3sync (if you have created it). If you want to do it more frequently just change the cron timings.

And wooohoooo all our data in S3 now!!!