Mark's Blog

Following is my bash script for copying Wordpress blog files, such as downloads, images, attachments in general to CloudFront. Written with efficiency in mind it will actually synchronize files thus sparing you massive uploads and lets you use CloudFront as CDN.

CSS and JS files will be compressed before uploading, if necessary, and the appropriate headers will be set correctly.

I’ve been inspired by the script on Mudy’s blog, which I found being inefficient as it yields many requests agains S3 (resulting in a slightly higher monthly bill, I guess) and which utilizes a Ruby script. But mine depends on s3cmd, which is written in Python, as I believe more people have Python installed than Ruby. Plus s3cmd has a native ’sync’ command.
I believe you can easily alter my script to fit other blogging systems or even arbitrary folder structures. You’re welcome to let the public know whether you succeeded in this.

So, here is the script:

#!/bin/bash
# Synchronizer script of static Wordpress files to Amazon S3.
#    by W-Mark Kubacki; wmark@hurrikane.de
#    http://mark.ossdl.de/2009/09/how-to-copy-your-wordpress-files-to-cloudfront-efficiently
#
# Utilizes s3cmd, which can be found here:
#    http://s3tools.org/s3cmd
#
# Licensed under RPL for private and academia use,
# and only so for commercial use and blogs with advertisements if
# a paypal donation is made to the author.
#
# version 2009-09-11-r2
#

# the bucket files will be synced with; e.g. s3://my-blog/
BUCKET=s3://your-blog/
# temporary directory for CSS and JS file structure, will be created
TMPDIR=/tmp/your-blog

# all files except JS and CSS
for DIR in ./; do
        s3cmd sync --exclude 'wp-admin/**' --exclude 'wp-content/cache/**' --exclude '**.svn**' \
        --exclude '*.php' --exclude '*.js' --exclude '*.css' --exclude '.htaccess' \
        --exclude '*.orig' \
        --acl-public --guess-mime-type --add-header=Cache-Control:max-age=604800 \
        --no-preserve --recursive "${DIR}" "${BUCKET}"
done

# JS and CSS files; being compressed if necessary, then synchronized
test -e $TMPDIR && rm -r $TMPDIR
for F in $(find -type f -readable -name '*.js' -o -name '*.css' ! -path '**.svn**' ! -path 'wp-content/cache/**' | grep -v 'wp-admin/' | cut -b 3-); do
        mkdir -p "${TMPDIR}/${F%/*}"
        file "${F}" | grep -q gzip \
        && cp -a "${F}" "${TMPDIR}/${F}" \
        || gzip -9 -c "${F}" > "${TMPDIR}/${F}"
done
cd $TMPDIR
        s3cmd sync --exclude '*.*' --include '*.js' --include '*.css' \
        --add-header=Content-Encoding:gzip \
        --acl-public --guess-mime-type --add-header="Cache-Control:max-age=604800, public" \
        --no-preserve --recursive ./ "${BUCKET}"
cd -
test -e $TMPDIR && rm -r $TMPDIR

You can also download my own latest version here.

Bookmark to:

Tags: CDN, cloud, know-how