I am creating a number of websites based on wordpress. Up till now I was just hacking the server and hoping that a backup was in place. At a certain moment I accidentally broke the .htaccess file. I enabled nice permalinks for an image library and my .htaccess file got overwritten. I wanted to have a look at the backup to see the rewrite rules I had in place. The hosting party wanted me to pay around 40 euros to provide the backup. That made me think about my hacking around on the server. There should be a better option than this.
I started thinking about putting the sources in a version control system. Since I like Git and have a Github account I wanted to store the files in Git. But I had some other requirements.
- I do not want to manually copy the changed files to the server.
- I am limited to ftp access, the hosted does not provide ssh access.
- I want to be able to make quick changes from multiple laptops.
- I want to have the changes taken from Git but also changes that have not yet been committed if I wanted to.
- I have no requirements for removing files, this does not happen a lot.
Since I am on a Mac and have some very basic experience with Bash, I decided to use Bash. I did have some issues with FTP, but more on that later on.
If you are curious about the websites I have created:
Yvonne van der Mey ~ Wildlife Photographer
Nico Bulder ~ Wildlife Painter
Repository layout
I want to support multiple sites in one repository. I give each site it’s own folder. The root of the project contains the deploy.lib file. This file contains the reusable code, which is almost everything. Each website folder contains a deploy.sh file. In the website folder we also have a file ftpconfig. This file is discussed later on when discussing the NcFtp client.
I do not store the complete wordpress installation in git. I most of store the template that you can find under wp-content/themes/yvonne. But I also add files that I added to the wordpress install like the favicon.ico. The idea is that I copy all changes over the existing wordpress installation. Like I stated before, I am not going to remove files. Using this mechanism I can also add custom plugins in the same way.
Explaining Git commands
The sources are stored in Git. This makes it easier to track changes and it helps me doing stuff the right way. So what is the right way? I want to deploy files that have changes in commits. Therefore I store the sha1 key of the last commit that I have deployed in a file. The next time I deploy changes, I read this file and deploy all changes after this commit. Sometimes however, you are debugging things and you do not want to continuously do commits. In that scenario you want to be able to deploy all changes in your local copy. This is also possible with the script.
Changes based on commits
Git has good support for finding more information on commits. There are multiple commands that can do the trick. For the most clean output came from the following command:
git show --pretty="format:" --name-only $startingfrom..HEAD
The $startingfrom is the sha1 key obtained through a provided parameter or from the file that was written the last time we did a deploy.
SHA1 of the last commit
We want to keep track of the changes that have already been deployed to the server. Since we are basing our deploys git commits, we want to obtain the sha1 key of the last commit. Git provides are daily easy way to obtain this key.
git log -1 --pretty=format:%H > $shaversion
With these commands we know enough to obtain changes after the last deploy. But what if we are trying something and want to make it possible to deploy files that have not yet been committed.
Changes in the local workspace
Changes that are not yet committed are called changes in the workspace of git. Again provides a good way to obtain the names of the files than have changes in the local workspace. You can check them against he HEAD.
git diff HEAD --name-only
That is it. Now we have the power to create a list of files that have changed. How can we copy files to the server using ftp.
NcFtp
The bash script creates a folder with a nested folder structure. The structure contains files on multiple levels. We want to use one command to recursively ftp all files to the server. Well, that is a hassle. First you have to overcome issue with ftp and logging in. Using the .netrc file this is easy to overcome on the mac. Still this does not resolve all our issues. Ftp does not support recursive ftp in a folder. You can use mput, but this only works for files in a folder. Not in a nested folder.
There are other options for ftp. I decided to use ncftp client software. Easy to install using Homebrew or Macports. I store the ftp connection data in a file that I configure in the ncftpput command. THe following command recursively copies the sources in the provided local folder to the provided remote folder.
The ftpconfig file must contain three properties: host, user, pass. The following command shows how you can copy the content of a directory to the server.
ncftpput -f ftpconfig -R mainwebsite_html deploy/yvonne/*
Doing bash
Time to put the pieces together. The glue is provided by bash.
Site specific configuration
Most of the bash code is shared between the different websites. That code is in the deploy.lib file. Of course you can place all code in one file. I have just one parameter and the included of the library file in the specific site deploy.sh file.
sitename="yvonne" . ../deploy.lib exit 1
Handling input
You can provide a few options to the script to influence what it does. It is good practice to show the options in a usage or help method. The following code block shows this method as well as the handling of the provided arguments.
function usage() {
echo "Usage: `basename $0` [--nocommit] {GIT REVISION SHA}"
echo " optional flags: "
echo " --help, --usage # show this file"
echo " --nocommit #By default only commits are used, with this flag we use changes in your workspace."
echo " --sha1 <value> #Uses the commits from the provided value to the HEAD. Default reads the sha1.txt file."
}
commitonly=true
while true; do
# case $# in 0) usage; exit 2 ;; esac
case $1 in
--help) usage; exit 1 ;; esac
case $1 in
--usage) usage; exit 1 ;; esac
case $1 in
--nocommit) shift; commitonly=false ;; esac
case $1 in
--sha1) shift; startingfrom=$1; shift ;;
*) break ;;
esac
done
The default is to gather all changes since the last commit that got deployed using the sha1 key in the sha1.tx file. By providing the –nocommit flag you can tell the script to switch to gathering changes in the local workspace. By providing the argument –sha1, you can pass another sha1 key to obtain changes for.
Obtaining the right changes
To find the files that have changed, we create a file containing the path to a file that is changed per line. By looping through these lines we can copy all files that we want to deploy. The next code block creates this file, but it has two side effects. It also creates a version.php that contains the date of the deploy as well as all the files that have been deployed. The second site effect is creating the sha1.txt file that contains the sha1 key of the last commit that has been deployed.
filelist="files-$(date +"%s").txt"
mkdir -p "deploy/$sitename"
versionphp="deploy/$sitename/version.php"
shaversion="sha1.txt"
if $commitonly ; then
if [[ ${#startingfrom} -lt 1 ]]; then
echo "Need to reed the sha1.txt file"
while read line; do
startingfrom=$(echo $line);
done < $shaversion
fi
echo "Using commits for changes: $startingfrom"
git show --pretty="format:" --name-only $startingfrom..HEAD > $filelist
echo "Committed changes: $(date)" > $versionphp
echo "<br>Last commit: $startingfrom" >> $versionphp
git log -1 --pretty=format:%H > $shaversion
echo "" >> $shaversion
else
echo "We only use changes in the workspace"
git diff HEAD --name-only > $filelist
echo "Local changes: $(date)" > $versionphp
fi
Line 6 determines if the starting from parameter has been provided by an argument to the script. We determine if the length of the value is smaller than 1. If so we obtain it from the file sha1.txt. I think this was the hardest trick.
Ftp the changed files
Now we have a file that contains all the names of the files that have changed. The next block loops over the files and filters a few of them based on the website we are working for at the moment. If we have files left at the end we use the ncftpput command to actually ftp them.
ftpfiles=false for file in $(<$filelist); do if [[ $file == $sitename* && $file != $sitename/deploy* && $file != $sitename/sha1.txt ]]; then echo "$file" echo "<br>$file" >> $versionphp mkdir -p "deploy/$(dirname $file)" cp "../$file" "deploy/$file" ftpfiles=true fi done if $ftpfiles ; then ncftpput -f ftpconfig -R mainwebsite_html deploy/yvonne/* else echo "Nothing to ftp." fi
Concluding
I am now able to adjust my workflow. I can make the changed locally, commit them into git before I actually automatically deploy them to the server. It is nice to see what you can do with a little bit of bash. If you have improvements please let me know.
The post Deployments with Git and Bash on a Mac appeared first on Gridshore.