Before We Get Started
For this example we want to cover just one specific real-world workflow. When continuously developing, changes need to be pushed out regularly to a staging server where the quality assurance team can review the features. This is just one part of the overall process but an important one to get working well. So in the end, we want Buddy to solve two things:
- Run our unit test suite against the application on every push to
- Deploy our code to a server whenever code is pushed to
To begin with, we first need to make some assumptions about our starting point:
You Have a Testable Application
We are going to assume you have a functioning Laravel application, with unit tests written using PHPUnit. If you have done nothing more than installed a fresh copy of the Laravel application then this prerequisite is met. Even if you do not have the actual unit tests written, just having the unit test runner ready to run tests is sufficient. If you use another testing framework, our examples can be easily adapted for your favorite.
You Have a Working Server
Additionally we will assume you have proper hosting infrastructure setup and configured on a public server running as a production environment. Any hosting will do but our recommendation is always Digital Ocean or Amazon Web Services. If you need to get up and running quickly sign up for Laravel Forge to provision a solid production server in your Digital Ocean account.
You Have a Buddy Account
Next you should have signed up with Buddy. Buddy offers a variety of Buddy Cloud (SaaS) plans starting with a Free tier, while also providing a self-hosted, on-premise version called Buddy GO. For the purposes of this demo, any of the free offerings will suffice, but if you need to add additional users, or need multiple projects to build concurrently, you should look into some of the paid plans.
You Have a Hosted Repository
Finally, your code must be in a Git repository. Something that makes Buddy unique is that it can host repositories itself, or integrate with Github, Gitlab, and Bitbucket – the three most popular repository hosts available. Either solution works fine, although for our demo we have elected to let Buddy host the repository.
Create the Project
To start things off, you will first need to create a project, select your repository hosting service, and do your first push. We chose Buddy to host our code base and named our app "My Laravel Application". You can name it what you like.
Pipelines and Actions
The foundation of Buddy's utility – and continuous development in general – is a workflow driven by pipelines. Pipelines are a series of steps (or "actions") which run against your repository in sequence. What makes Buddy dramatically faster than many of its competitors is that:
- Actions are run in Docker containers, so launching the environment for an action can be done very quickly, especially on subsequent runs.
- Actions share a filesystem common to all actions within a pipeline and is persisted even across subsequent runs of the pipeline. This means, for example, dependencies usually only need to be downloaded once, unless they are updated. You can manually create and edit files in a pipeline. This makes the filesystem useful for storing environment configurations that can be safely stored within Buddy, without having to commit them to source control.
Create a PHP Container
For our first pipeline, let's run our unit tests. For this, we will need a Docker container with PHP installed. This container will use the filesystem that has our repository's code on it as the basis for the test runner – PHPUnit. Buddy of course provides the PHP container option as a build type action.
Buddy's default PHP container can support just about any version of PHP 5.6 or 7.
If you needed something like HHVM then use the Ubuntu container and install the
dependencies yourself or choose a publically hosted HHVM docker container image.
We chose PHP 7.0. Buddy automatically installs Composer
composer install as a default command to be executed. While this
works for basic setups, a production pipeline needs a bit of fine-tuning. We are
going to make an adjustment to the Run Command settings.
Customize the Run Command
vendor/bin/phpunit to the command being run. We use the version of
vendor so that the repository can control which version of PHPUnit
to install and we do not have to worry about global dependencies. Some developers
choose to configure
composer test in their
composer.json file to indicate the
testing commands that should be ran. This is a good alternative and one that is
adopted by other languages such as Node.js which uses
Tweak the Container for Performance
Switching to the
Packages & Caches tab, there are a few extra settings we should change.
These settings are provided below for copy-pasting:
apt-get update && apt-get install git zip unzip -y curl -sS https://getcomposer.org/installer | php mv composer.phar /usr/local/bin/composer
We have done two things in this panel to help improve the performance of Composer.
- We added the
unzippackages to the container. These will allow Composer to download packages much more quickly than the fallback of using Git.
- We have added a
.composerdirectory to the action's cache. This is the directory where Composer will cache the zip files, so that on subsequent runs it will not have to do download them again. The path here can be tricky and must be exact. The
.composerdirectory is created within the pipeline's mounted fileystem path and must be absolute. If you change the mountpoint path or your project name is different than ours you'll need to change the
/my-laravel-app/.composer/path appropriately. Also remember that we want to cache the entire
.composerdirectory so the trailing slash is required per the filesystem's pattern matching rules.
Give the Action a Name
Now's as good a time as any to provide a familiar name for the Action. We chose "Run PHPUnit" as that's what it does. If your testing commands are different then perhaps a more generic "Run Composer Test" is more accurate.
When creating your first action for a new project, the action is added to a pipeline that is also created for you. When adding new actions to an existing pipeline, they are automatically added for you. So after clicking "Add this action" you should be presented with options to add more actions or proceed to viewing your pipeline.
Verify the Pipeline Settings
Before we try things out, we are going to verify a few more things.
- Go to the
Pipeline Settingspage to change your pipeline's name. By default it's something silly like "My First Pipeline". In our case we named it after the primary action, "Run Unit Tests".
- Also verify the Trigger Mode is set to "On every push" and for the
masterbranch. For our testing pipeline this is perfect but for other pipelines you'll want to configure these settings appropriately.
Setup the Environment
We are going to use Buddy's ability to manipulate the pipeline filesystem to
.env file for our unit testing environment. From the
page, click in the right sidebar "Browse the filesystem" to do just that. If you
.env file on your local machine you can upload it directly, or you can
use Buddy's interface to create one directly based on the
We used the
.env.example file that ships with Laravel, but with an
already generated. Depending on the complexity of your unit tests, you may need
to tweak database settings or modify other parameters. Also remember to verify
your test runner's configs (in the case of PHPUnit that would be
Run the Pipeline Manually
At this point, you can run the pipeline manually. Assuming everything went to plan, your dependencies should download, PHPunit should run your tests, and you should get a happy green build icon.
The first time you run the pipeline, the container is setup. This can take a bit
of time so be patient. If the pipeline executed without a problem, try executing
it again and you'll find it to be much faster. Remember that Composer outputs a
composer.lock file in your directory which will skip installation of dependencies
on the second build. For this reason, you would want to either delete the lock file
manually through the filesystem browser should you want to install again, or better
yet, commit the
composer.lock file with your repository by removing the line
.gitignore file in the root of the Laravel code base. This will ensure
the right version of all dependencies are installed and consistently across builds.
Build and Deploy the Application
Now that we have a functioning unit test pipeline, let's use Buddy to automate our deployment process. The basic building blocks are going to remain the same, but there are going to be more actions involved, and we are going to use connect two pipelines together to accomplish the over all workflow.
Create a Staging Pipeline
Our first deployment pipeline is going to use a manual trigger to build the
application, including all its dependencies, and upload them to our previously
setup staging server. We named the pipeline "Deploy to Staging" and setup the
Trigger Mode to be
Manual on a
Specific branch which in our case is
Build PHP Application Action
Our pipeline is going to be made up of five actions. First, we need a PHP container-based action to install all the application's dependencies. We named it "Build PHP application". The Run Commands we used are as follows:
composer install --no-dev npm install npm run prod
Notice that we are skipping the dev dependencies for this pipeline using the
--no-dev switch. Since we are uploading to a staging server, they should not
be necessary as the dev dependencies are only need to build the app, not run it.
You will also see that we are installing and compiling all our frontend assets using NPM and Gulp. These services are not available in the default PHP container and will require some additions to the container's build steps and caches:
curl -sL https://deb.nodesource.com/setup_6.x | bash - apt-get update && apt-get install git build-essential nodejs zip unzip -y npm install --global gulp marked node-gyp curl -sS https://getcomposer.org/installer | php mv composer.phar /usr/local/bin/composer
In addition to installing Composer, this will install the latest version of Node.js and Laravel's frontend build tools. Much like we previously did with Composer, we are going to add NPM's cache directory to Buddy's list of cached directories to speed up build time:
Switch to Maintenance Mode Action
If the previous action completes, we have a successful build of the application. Next, we should connect to the staging server and put it into maintenance mode. For this, we will use Buddy's SSH action. The interface for this action includes some handy instructions for installing Buddy's public key on your server. This is necessary so that the pipeline can connect without having to use a password. If you're using Laravel Forge then you can easily upload the key provided directly through the Forge web interface and let Forge do the heavy lifting.
We named the action "Bring Application Down". The Run Command itself is a one-liner:
touch artisan && php artisan down
When we first run the pipeline,
artisan will not have been uploaded to the server,
touch will give PHP an (empty) file to try executing. Under the
More Options section, make sure your working directory on the server is set
correctly, and the action is set to ignore errors. Normally you would want a
pipeline action to stop if any errors are encountered, but for this we can
assume that an error means the application is already down (albeit, less gracefully)
or has not yet been deployed.
Upload Files Action
Now that the application is disabled, we can copy the files to the server. If you are using a hosting provider such as DigitalOcean there are several convenient actions to lookup your available servers which all use the SFTP action under the hood. We are going to be using SFTP action to copy our application files en masse to the staging server. Select the SFTP action, setup your staging server's IP address, port, and credentials just like before, and make sure the remote path is correct. Do not forget to give the actino a good name like "Upload to Staging" which represents exactly what the action does.
There are also several directories within our pipeline we can and you should ignore as uploading them to the staging server is completely unnecessary and will slow down your builds:
/.npm/ /.composer/ /node_modules/
The cache directories for NPM and Composer can be safely ignored. In addition
though, we also exclude the
node_modules directory. Since our assets will have
all been compiled in the first step of this pipeline, we no longer need to have
the contents of
node_modules, and skipping it can dramatically improve upload
times. You could also add
npm run prod should have used
gulp behind the scenes to build your assets and put them in the
The same could be said for directories like
/tests/ which are not needed on a
It should be noted that this is generally the most time consuming action in the
pipeline and while subsequent runs are generally faster than the first run, the
entire time you are uploading files, your application is down. This makes this
pipeline create some graceful downtime for you application. In practice, deployments
encounter no more than one minute of total maintenance mode down time. That's
pretty neglible for most apps but if it counts, more advanced zero-downtime
strategies can be used with Buddy and faster deployment mechanisms like using
rsync are possible.
Run Migrations Action
After the files have been uploaded, we can migrate the database. Just like with the "Switch to Maintenance Mode" action, this is an SSH action. You might name the action "Run Migrations" and use the following Run Commands:
php artisan migrate --force
We add the
--force flag so that migrations will be run even though the
application is in a non-local (a staging or production) environment.
Restart Application Action
Now that the application code is deployed, the database is migrated, it's time to bring the application back up out of maintenance mode. You can run Laravel's optimizations and restart queue workers if you have them running. While this could have been done at the same time as the previous step, the best practice approach says that when you have two separate concepts you have two separate processes and therefore separate pipeline actions. The added benefit of following best practices is that Buddy lets you resume any failed action from that point which might lead to resolving failed actions sooner.
We named our action simply "Bring Application Up" with the following Run Commands:
php artisan cache:clear php artisan config:cache php artisan route:cache php artisan optimize php artisan queue:restart php artisan up
This should ensure optimal performance from your Laravel application in a production environment.
Setting Up the Environment
Just like with our unit testing pipeline, we need to upload or create a
file for our staging environment. It is important to note, however, that this is
.env file the staging server will be using, not necessarily what the
actions in the pipeline will use. Make sure your environment variables are set
accordingly and add the
.env file to the pipeline's filesystem.
The resulting pipeline should look something like this:
Buddy can have one pipeline run another pipeline. The pipeline we just created
will not run automatically. That is because we want it to be triggered by a
second pipeline that will be automatically triggered. Even though we run our
unit tests on every push to
master it is a good idea to always run your tests
on staging or production code before deploying it. Thankfully, our unit tests
pipeline is already configured to do this for us, we just need to adjust its
settings and add an additional step. Create another pipeline but from the
Add a new pipeline form, click
Clone existing pipeline and select your unit
Tweak the new pipeline's settings to point to your
staging branch. In this new
pipeline, after the unit testing action has been run, we want to run all the actions
in our previous deployment pipeline. Buddy makes this easy for us: we can run the
deployment pipeline as an action from this testing pipeline – inception style.
This way, once the unit tests have all passed on the
staging branch, your code
will be copied to the server. Create your
.env file as we did with the previous
two pipelines and you are ready to deploy some code – simply by pushing to the
staging branch. Rinse and repeat these steps to add as many deployment
environments as you like including one for a
The initial upload will probably be quite lengthy, however on subsequent commits Buddy is pretty smart about determining what files have changed and need to be copied to the server.
With these tools in hand, web artisans can construct powerful and flexible integration and deployment pipelines. We have just begun to scratch the surface of all that you can do with Buddy. There is a wealth of additional features and actions that can be used, such as sending messages to Slack on successful completion or errors, deploying AWS Lambda functions, or building completely custom actions using a generic Ubuntu container. As the tools continue to mature, we look forward to putting them into our own production processes. Ultimately, Buddy makes it easy for the app developers to continuously develop.