Deploy to Gandi Simple Hosting

July 31 2015

At the end of the article, a step by step guide to continuous deployment on Gandi Simple Hosting.

----

As Silex is free and open source software, and used by many people for free, Gandi.net is kindly providing me with free hosting for Silex.me - see their supported projects page. So I use Gandi Simple Hosting for a while now, I have 2 Node.js instances, for preprod and prod.

Compared to the previous hosting we had for Silex, which is Heroku, the performance have increased a lot, as stated in this tweet.

Gandi Simple Hosting allows me to deploy with sftp or git and connect with ssh but it has several quite annoying limitations though, I believe this is because Node.js hosting is quite new at Gandi.net.

Here is an explanation of what I wanted to deploy my code, what problems I had and the solution I have found.

From heroku (2 dynos) to @gandi_net (1 instance) #node.js hosting, for #silexv2 open source website builder :) pic.twitter.com/rVmEOKA3mo
— Alex Hoyau (@lexoyo) 21 Juin 2014

The situation

On Simple Hosting, it is possible to deploy with git out of the box, but it does not support submodules. And there is no way to trigger a build after a push, just checkout the files, so you would have to commit prebuild files in git, which is bad practice. SSH access can be used to trigger a build (run a script) after a push on github (with web hooks), but it needs to be activated by hand first in Gandi's web interface. And it resets every hour.

And finally when you find a way to upload your build files to the server, you have to restart your Node.js app, which is only possible from within the web interface.

The web interface of Gandi Simple Hosting

The journey

So my goal for tonight was to be able to

deploy with almost no interruption of service
easily deploy with git push
run test and do not deploy if it fails

I have tried many tools and techniques to solve

build and test (and possibly abort)
transfer the production files to Gandi
reload Node.js app or reboot if needed (which takes around one minute)

Here are a few links

rsync and sftp man pages
codeship article Deploy via FTP, SFTP, SCP, RSYNC and SSH
forever.js and PM2 which aim at managing sub processes. They both can be used programmatically in Node.js, not only with the command line, which is a requirement since Simple Hosting ssh access needs to be activated via the web interface by hand every hour... Also note that I could not make PM2 run on Gandi Simple Hosting because of access rights issues, and forever did not let me communicate with my child process easily.
a bug of the sftp put command which makes it impossible to use `sftp put -r` on Simple Hosting

So I have finally come to a solution, combining

build and test (and possibly abort): use codeship
transfer the production files to Gandi: upload with sftp (custom shell script in codeship)
reload Node.js app: a child process to run Silex, kill and reload it at will + a web hook triggerd by codeship calls Silex Node.js API which plays with processes in order to reload

Please note that I have tested the same process with Circle CI and it works. But Travis CI do NOT work with Simple Hosting because of the ssh key which I could not get working properly (maybe if I had a paid account it would though)

Step by step solution

Build and test (and possibly abort)

I linked codeship to Silex github repository so that github triggers a build for each push. This is really easy, simply create an account on codeship and follow the instructions (click, click, click)

Transfer the production files to Gandi

Then I have created a "Deployment pipeline" as they say in codeship, selecting the "custom script" option. In the script section I have written this to zip and then transfer the files to Gandi when a build succeeds on codeship's servers:

zip -r silex.zip dist/ node_modules/
echo "put ${HOME}/silex.zip" | sftp 1234567@sftp.dc0.gpaas.net:vhosts/default

Her you will want to replace `1234567` and `sftp.dc0.gpaas.net` by the ID and host you find in Gandi's web interface, hovering the "?" at the right of "SFTP". Also in Gandi web interface, click on "SSH key - add" and add the ssh key which is provided by codeship in the "General" section of the settings.

Reload Node.js app

A web hook in Silex Node.js API to trigger reload

In the same "Deployment pipeline" section on codeship, I have added this line to call the web hook which I have created in Silex as explain bellow

wget http://preprod.silex.me/tasks/restart-xxxxxx

This web hook is a route in Silex Node.js API used to notify the parent process that it should kill and reload the Silex child process now. This is where it is defined in Silex code. The important parts:

process.env.RESTART_ROUTE is a way to store the route in an env var, to avoid having this "secret" route in the repository
process.send('restart'); is how we send a message to the parent process in order to reload Silex - see bellow

A child process to run Silex, kill and reload it at will

Almost there: here is how I handle the process which runs Silex in Node.js.

When Gandi Simple Hosting boots, it launches a `node server.js`. So this is what I have now in my `server.js` file:

// require the Node.js process management module
var childProcess = require("child_process");

// define the env var to use in Silex Node.js code

process.env.RESTART_ROUTE = 'restart-lexo1000';

// path to Silex server side script
var silexServerScript = __dirname + '/dist/server/server.js';

// start the process

start();

/**

 * recursive function wich loads, kill and then reload Silex process

 */

function start() {

  // create the process
  var child = childProcess.fork(silexServerScript, [], {
      env: process.ENV,
      silent: false
    }
  );

  // wait for Silex message to kill and then start again the process


  child.on('message', function(message){
    console.log('message', message);
    if(message === 'restart') {
      childProcess.exec(__dirname + '/update.sh', function (error, stdout, stderr) {
        console.log('exec update.sh done', error);
        console.log('kill process');
        child.kill();
        console.log('restart process');
        start();
        console.log('done');
      });
    }
  });


  child.on('message', function(message){
     if(message === 'restart') {
      child.kill();
      start();
    }
  });
  return child;
}

This script is run after each successful build, because I have this `wget` call in codeship. And this script itself calls `update.sh` which is a script at the root of my Simple Hosting which unzips and replace the previous Silex folder. Then it removes the old Silex folder in the background:

#!/bin/sh

mkdir -p Silex
unzip -o silex.zip -d clone && mv Silex old && mv clone Silex && rm silex.zip
rm -r old &

The full build process in codeship

Drawbacks and limitations

This is a bit raw, I will probably fine tune it, but clearly it has these problems

it requires specific code in my app, and a hidden route to trigger the reload
it takes quite a long time to build, zip, upload, unzip and reload all the files needed from codeship to gandi (around 3 min) - the down time is very short though, only a few seconds