For those wanting to automate your CI and CD scripts, here are some general tips regarding networking, security, and Docker when using Bitbucket Pipelines.
With Bitbucket Pipelines, you can quickly adopt a continuous integration or continuous delivery workflow for your repositories. An essential part of this process is to turn manual processes into scripts that can be run automated by a machine without the need for human intervention. But sometimes it can be tricky to automate tasks as you might have some issues with authentication, installing dependencies or reporting issues. This guide will help you with some tips for writing your scripts!
Before moving any further into the world of automation, you need to review your logs and make sure that you do not output sensitive data such as API keys, credentials or any information that can compromise your system. As soon as you start using Bitbucket Pipelines to run your scripts the logs will be stored and readable by anyone who has access to your repository.
Authentication is often one of the most troublesome parts of automation. SSH keys have the double advantage of making a connection to remote servers easy to manage, and being very secure. With Bitbucket Pipelines, you can easily generate a new key pair that can be used on every pipeline run to connect to remote servers.
You’ll simply need to copy the public key in your remote server to be able to connect to it from your running pipeline. For example, once the SSH keys are set up for on the server (you can use a URL or IP address) , the script below would list files under the /var/www directory without the need to provide a password.
bitbucket-pipelines.yml:
If you need to use a remote API as part of your scripts, the chances are that your API provider lets you use their protected resources with an API key. You can safely add credentials to Bitbucket Pipelines using secured environment variables. Once saved you can invoke them in your scripts, and they’ll stay masked in the log output.
If you need to install dependencies as part of your script, ensure that it does not prompt the user asking for validation or input. Look into the documentation of the commands you are using to see if there is a flag that allows you to run them in a non-interactive way.
For instance, the flag -y in the command below will install PostgreSQL on a Debian server.
And the flag -q allows you to run Google Cloud SDK commands in a non-interactive way.
gcloud -q app deploy app.yaml
Installing the dependencies necessary for your pipeline to run can be time-consuming. You could save a lot of running time by creating your own Docker image with the basic tools and packages required to build and test your application.
For instance, in the following Pipelines configuration we install AWS CLI at the beginning to use it later to deploy the application to AWS Elastic Beanstalk.
bitbucket-pipelines.yml:
The issue here is that the AWS CLI does not change with every commit, which means that we’re wasting some time installing a dependency that could be bundled by default.
The following Dockerfile could be used to create a custom Docker image ready for Elastic Beanstalk deployments.
Dockerfile:
bitbucket-pipelines.yml:
These tips should help you turn manual tasks into automated processes that can be run repeatedly and reliably by a service like Bitbucket Pipelines. Eventually, they’ll be the guardians of your releases and will be powerful tools able to trigger the deployment of your entire production environments across multiple servers and platforms.
This is why you need to treat your automation scripts as code and take them through the same review and quality process that you have for your code. Thankfully, this is something that can easily be done with Bitbucket as your pipeline configuration will be checked in with your code, allowing you to created pull requests in the right context.
And finally, don’t forget to run scripts in a test environment before applying them to production – these extra minutes might save you from wiping out production data by mistake.
