How to Use Buddy to Continuously Develop a Laravel PHP Application

Before We Get Started

For this example we want to cover just one specific real-world workflow. When continuously developing, changes need to be pushed out regularly to a staging server where the quality assurance team can review the features. This is just one part of the overall process but an important one to get working well. So in the end, we want Buddy to solve two things:

1. Run our unit test suite against the application on every push to master branch.
2. Deploy our code to a server whenever code is pushed to staging branch.

To begin with, we first need to make some assumptions about our starting point:

You Have a Testable Application

We are going to assume you have a functioning Laravel application, with unit tests written using PHPUnit. If you have done nothing more than installed a fresh copy of the Laravel application then this prerequisite is met. Even if you do not have the actual unit tests written, just having the unit test runner ready to run tests is sufficient. If you use another testing framework, our examples can be easily adapted for your favorite.

You Have a Working Server

Additionally we will assume you have proper hosting infrastructure setup and configured on a public server running as a production environment. Any hosting will do but our recommendation is always Digital Ocean or Amazon Web Services. If you need to get up and running quickly sign up for Laravel Forge to provision a solid production server in your Digital Ocean account.

You Have a Buddy Account

Next you should have signed up with Buddy. Buddy offers a variety of Buddy Cloud (SaaS) plans starting with a Free tier, while also providing a self-hosted, on-premise version called Buddy GO. For the purposes of this demo, any of the free offerings will suffice, but if you need to add additional users, or need multiple projects to build concurrently, you should look into some of the paid plans.

You Have a Hosted Repository

Finally, your code must be in a Git repository. Something that makes Buddy unique is that it can host repositories itself, or integrate with Github, Gitlab, and Bitbucket – the three most popular repository hosts available. Either solution works fine, although for our demo we have elected to let Buddy host the repository.

---

Create the Project

To start things off, you will first need to create a project, select your repository hosting service, and do your first push. We chose Buddy to host our code base and named our app "My Laravel Application". You can name it what you like.

Pipelines and Actions

The foundation of Buddy's utility – and continuous development in general – is a workflow driven by pipelines. Pipelines are a series of steps (or "actions") which run against your repository in sequence. What makes Buddy dramatically faster than many of its competitors is that:

1. Actions are run in Docker containers, so launching the environment for an action can be done very quickly, especially on subsequent runs.
2. Actions share a filesystem common to all actions within a pipeline and is persisted even across subsequent runs of the pipeline. This means, for example, dependencies usually only need to be downloaded once, unless they are updated. You can manually create and edit files in a pipeline. This makes the filesystem useful for storing environment configurations that can be safely stored within Buddy, without having to commit them to source control.

Create a PHP Container

For our first pipeline, let's run our unit tests. For this, we will need a Docker container with PHP installed. This container will use the filesystem that has our repository's code on it as the basis for the test runner – PHPUnit. Buddy of course provides the PHP container option as a build type action.

Buddy's default PHP container can support just about any version of PHP 5.6 or 7. If you needed something like HHVM then use the Ubuntu container and install the dependencies yourself or choose a publically hosted HHVM docker container image. We chose PHP 7.0. Buddy automatically installs Composer and provides composer install as a default command to be executed. While this works for basic setups, a production pipeline needs a bit of fine-tuning. We are going to make an adjustment to the Run Command settings.

Customize the Run Command

We added vendor/bin/phpunit to the command being run. We use the version of phpunit installed in vendor so that the repository can control which version of PHPUnit to install and we do not have to worry about global dependencies. Some developers choose to configure composer test in their composer.json file to indicate the testing commands that should be ran. This is a good alternative and one that is adopted by other languages such as Node.js which uses npm test.

Tweak the Container for Performance

Switching to the Packages & Caches tab, there are a few extra settings we should change.

These settings are provided below for copy-pasting:

apt-get update && apt-get install git zip unzip -y
curl -sS https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer

We have done two things in this panel to help improve the performance of Composer.

1. We added the zip and unzip packages to the container. These will allow Composer to download packages much more quickly than the fallback of using Git.
2. We have added a .composer directory to the action's cache. This is the directory where Composer will cache the zip files, so that on subsequent runs it will not have to do download them again. The path here can be tricky and must be exact. The .composer directory is created within the pipeline's mounted fileystem path and must be absolute. If you change the mountpoint path or your project name is different than ours you'll need to change the /my-laravel-app/.composer/ path appropriately. Also remember that we want to cache the entire .composer directory so the trailing slash is required per the filesystem's pattern matching rules.

Give the Action a Name

Now's as good a time as any to provide a familiar name for the Action. We chose "Run PHPUnit" as that's what it does. If your testing commands are different then perhaps a more generic "Run Composer Test" is more accurate.

When creating your first action for a new project, the action is added to a pipeline that is also created for you. When adding new actions to an existing pipeline, they are automatically added for you. So after clicking "Add this action" you should be presented with options to add more actions or proceed to viewing your pipeline.

Verify the Pipeline Settings

Before we try things out, we are going to verify a few more things.

1. Go to the Pipeline Settings page to change your pipeline's name. By default it's something silly like "My First Pipeline". In our case we named it after the primary action, "Run Unit Tests".
2. Also verify the Trigger Mode is set to "On every push" and for the master branch. For our testing pipeline this is perfect but for other pipelines you'll want to configure these settings appropriately.

Setup the Environment

We are going to use Buddy's ability to manipulate the pipeline filesystem to create a .env file for our unit testing environment. From the Pipeline Settings page, click in the right sidebar "Browse the filesystem" to do just that. If you have a .env file on your local machine you can upload it directly, or you can use Buddy's interface to create one directly based on the .env.example default. We used the .env.example file that ships with Laravel, but with an APP_KEY already generated. Depending on the complexity of your unit tests, you may need to tweak database settings or modify other parameters. Also remember to verify your test runner's configs (in the case of PHPUnit that would be phpunit.xml).

Run the Pipeline Manually

At this point, you can run the pipeline manually. Assuming everything went to plan, your dependencies should download, PHPunit should run your tests, and you should get a happy green build icon.

The first time you run the pipeline, the container is setup. This can take a bit of time so be patient. If the pipeline executed without a problem, try executing it again and you'll find it to be much faster. Remember that Composer outputs a composer.lock file in your directory which will skip installation of dependencies on the second build. For this reason, you would want to either delete the lock file manually through the filesystem browser should you want to install again, or better yet, commit the composer.lock file with your repository by removing the line from the .gitignore file in the root of the Laravel code base. This will ensure the right version of all dependencies are installed and consistently across builds.

---

Build and Deploy the Application

Now that we have a functioning unit test pipeline, let's use Buddy to automate our deployment process. The basic building blocks are going to remain the same, but there are going to be more actions involved, and we are going to use connect two pipelines together to accomplish the over all workflow.

Create a Staging Pipeline

Our first deployment pipeline is going to use a manual trigger to build the application, including all its dependencies, and upload them to our previously setup staging server. We named the pipeline "Deploy to Staging" and setup the Trigger Mode to be Manual on a Specific branch which in our case is stage.

Build PHP Application Action

Our pipeline is going to be made up of five actions. First, we need a PHP container-based action to install all the application's dependencies. We named it "Build PHP application". The Run Commands we used are as follows:

composer install --no-dev
npm install
npm run prod

Notice that we are skipping the dev dependencies for this pipeline using the --no-dev switch. Since we are uploading to a staging server, they should not be necessary as the dev dependencies are only need to build the app, not run it.

You will also see that we are installing and compiling all our frontend assets using NPM and Gulp. These services are not available in the default PHP container and will require some additions to the container's build steps and caches:

curl -sL https://deb.nodesource.com/setup_6.x | bash -
apt-get update && apt-get install git build-essential nodejs zip unzip -y
npm install --global gulp marked node-gyp
curl -sS https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer

In addition to installing Composer, this will install the latest version of Node.js and Laravel's frontend build tools. Much like we previously did with Composer, we are going to add NPM's cache directory to Buddy's list of cached directories to speed up build time:

/my-laravel-app/.composer/
/my-laravel-app/.npm/

Switch to Maintenance Mode Action

If the previous action completes, we have a successful build of the application. Next, we should connect to the staging server and put it into maintenance mode. For this, we will use Buddy's SSH action. The interface for this action includes some handy instructions for installing Buddy's public key on your server. This is necessary so that the pipeline can connect without having to use a password. If you're using Laravel Forge then you can easily upload the key provided directly through the Forge web interface and let Forge do the heavy lifting.

We named the action "Bring Application Down". The Run Command itself is a one-liner:

touch artisan && php artisan down

When we first run the pipeline, artisan will not have been uploaded to the server, running touch will give PHP an (empty) file to try executing. Under the More Options section, make sure your working directory on the server is set correctly, and the action is set to ignore errors. Normally you would want a pipeline action to stop if any errors are encountered, but for this we can assume that an error means the application is already down (albeit, less gracefully) or has not yet been deployed.

Upload Files Action

Now that the application is disabled, we can copy the files to the server. If you are using a hosting provider such as DigitalOcean there are several convenient actions to lookup your available servers which all use the SFTP action under the hood. We are going to be using SFTP action to copy our application files en masse to the staging server. Select the SFTP action, setup your staging server's IP address, port, and credentials just like before, and make sure the remote path is correct. Do not forget to give the actino a good name like "Upload to Staging" which represents exactly what the action does.

There are also several directories within our pipeline we can and you should ignore as uploading them to the staging server is completely unnecessary and will slow down your builds:

/.npm/
/.composer/
/node_modules/

The cache directories for NPM and Composer can be safely ignored. In addition though, we also exclude the node_modules directory. Since our assets will have all been compiled in the first step of this pipeline, we no longer need to have the contents of node_modules, and skipping it can dramatically improve upload times. You could also add /resources/assets/ as npm run prod should have used gulp behind the scenes to build your assets and put them in the public directory. The same could be said for directories like /tests/ which are not needed on a staging server.

It should be noted that this is generally the most time consuming action in the pipeline and while subsequent runs are generally faster than the first run, the entire time you are uploading files, your application is down. This makes this pipeline create some graceful downtime for you application. In practice, deployments encounter no more than one minute of total maintenance mode down time. That's pretty neglible for most apps but if it counts, more advanced zero-downtime strategies can be used with Buddy and faster deployment mechanisms like using rsync are possible.

Run Migrations Action

After the files have been uploaded, we can migrate the database. Just like with the "Switch to Maintenance Mode" action, this is an SSH action. You might name the action "Run Migrations" and use the following Run Commands:

php artisan migrate --force

We add the --force flag so that migrations will be run even though the application is in a non-local (a staging or production) environment.

Restart Application Action

Now that the application code is deployed, the database is migrated, it's time to bring the application back up out of maintenance mode. You can run Laravel's optimizations and restart queue workers if you have them running. While this could have been done at the same time as the previous step, the best practice approach says that when you have two separate concepts you have two separate processes and therefore separate pipeline actions. The added benefit of following best practices is that Buddy lets you resume any failed action from that point which might lead to resolving failed actions sooner.

We named our action simply "Bring Application Up" with the following Run Commands:

php artisan cache:clear
php artisan config:cache
php artisan route:cache
php artisan optimize
php artisan queue:restart
php artisan up

This should ensure optimal performance from your Laravel application in a production environment.

Setting Up the Environment

Just like with our unit testing pipeline, we need to upload or create a .env file for our staging environment. It is important to note, however, that this is the .env file the staging server will be using, not necessarily what the actions in the pipeline will use. Make sure your environment variables are set accordingly and add the .env file to the pipeline's filesystem.

The resulting pipeline should look something like this:

---

Pipeline Inception

Buddy can have one pipeline run another pipeline. The pipeline we just created will not run automatically. That is because we want it to be triggered by a second pipeline that will be automatically triggered. Even though we run our unit tests on every push to master it is a good idea to always run your tests on staging or production code before deploying it. Thankfully, our unit tests pipeline is already configured to do this for us, we just need to adjust its settings and add an additional step. Create another pipeline but from the Add a new pipeline form, click Clone existing pipeline and select your unit testing pipeline.

Tweak the new pipeline's settings to point to your staging branch. In this new pipeline, after the unit testing action has been run, we want to run all the actions in our previous deployment pipeline. Buddy makes this easy for us: we can run the deployment pipeline as an action from this testing pipeline – inception style.

This way, once the unit tests have all passed on the staging branch, your code will be copied to the server. Create your .env file as we did with the previous two pipelines and you are ready to deploy some code – simply by pushing to the staging branch. Rinse and repeat these steps to add as many deployment environments as you like including one for a production branch.

The initial upload will probably be quite lengthy, however on subsequent commits Buddy is pretty smart about determining what files have changed and need to be copied to the server.

---

Going Further

With these tools in hand, web artisans can construct powerful and flexible integration and deployment pipelines. We have just begun to scratch the surface of all that you can do with Buddy. There is a wealth of additional features and actions that can be used, such as sending messages to Slack on successful completion or errors, deploying AWS Lambda functions, or building completely custom actions using a generic Ubuntu container. As the tools continue to mature, we look forward to putting them into our own production processes. Ultimately, Buddy makes it easy for the app developers to continuously develop.