Applied Gitflow

From charlesreid1

About

This article describes how we apply the gitflow model to keep track of our code, and to keep track of what code we have deployed to the cloud in a multi-stage cloud environment.

The Tools

The main tool for this writeup is git, used for version control of the repository. GitHub and Gitea provide an interface to manage the git repository and to collaborate/review code. We use the gitflow model for managing branches and code, so we'll summarize it, and provide a link with the details.

We are also using a cloud service where we upload the code to the cloud, and the code is run there. That code is separate from the git repository, so we need a way to track what code is in what environment. We'll cover how we do that, too.

Finally, we'll present the centerpiece of this article: the Makefile, release, and deploy scripts that we drop into brand-new repositories as a starting point.

Gitflow

Link: https://nvie.com/posts/a-successful-git-branching-model/

Recap of Gitflow:

regular development work:

  • main branch is the latest stable version of the code
  • develop branch is where development happens
  • features in develop will make it to main in a release
  • features split off from it, and are merged back into it

preparing for release:

  • create new branch, release/vX
  • update version numbers, changelog, readme, etc.
  • cut a release to main - force update main pointer to point to tip of the release/vX branch

hotfixes:

  • branches that are created from the release branch, merged back into release branch and main

Deploying and Releasing

There are two main actions that an operator can take, using the system we have devised:

  • deploy code (upload the code to a cloud stage, and have that version of the code running in the cloud)
  • release code (update which version of the code is considered the "latest version")

Additionally, the actions the operator can take will require certain variables to be defined - the location of the source code, and which environment (development, integration, production) we're deploying to, for example. Those variables are defined using environment variables, so that we can use scripts to perform the deploy or run actions.

Environment File

Before covering the deploy and release scripts, here is an example environment file. The example release script below refers to these variables. The deploy script (no example provided, since it will inevitably be case-specific) also uses these environment variables. More environment variables can be added as needed, to grow with the complexity of the software and deployment process. 

#!/bin/bash

export PROJECTNAME_PACKAGENAME_HOME="${HOME}/path/to/package"
export PROJECTNAME_PACKAGENAME_STAGE="dev"

Deploy Action

The deploy action is the action that takes the code in its current state and uploads it to the cloud. That can take many forms depending on the cloud service being used, but for example deploying an AWS lambda function would mean creating a new .zip file from the lambda function code, and uploading that zip file to the corresponding lambda function using the AWS CLI.

The deploy action is a script that is run. It deploys the code in its current state in the repository. Environment variables are used to parameterize the script. There are environment variables to specify the repository location on disk, to specify the deployment stage, and any others that are needed.

The deploy script is also the place to assert certain conditions are true before the deploy action is taken. We keep it simple, but these can be expanded on:

  • Check various environment variables are set
  • Check environment variable values

These can be done in the deploy script directly, if the checks are simple, or they can be moved to an entire separate script like check_env.sh, which would be run before the deploy script (using a make rule dependency).

(Note that if checks are TOO strict, they can interfere with deployment, so they should be adjusted accordingly.)

Another step the deployment action might have before the actual deployment is to assemble any deployment configuration files. For example, if the deployment process requires a JSON file that is dynamically assembled from other bits of information, that could go in its own script that would be run before the deploy script (using a make rule dependency).

For this writeup, we keep things simple and do all the checks in the deploy script.

Release Action

The release action is a git repository operation.

When does a release happen?

When using gitflow, features and bugfixes will accumulate in the develop branch. Eventually the time for a new release will come (scheduled, or because enough changes have accumulated). A new branch will be created that will "freeze" the code in whatever state the develop branch is in. (Freeze is in quotes, because there are small changes that need to be made to that "frozen" code before a release happens, but those are changes like bumping the version number - no core changes.)

Where does the release script come in?

Once the code in the release branch is ready to go, the release script is run. The script will create a git tag, reset the head of the branch corresponding to the release (the "stable release" branch, usually main) to the head of the pre-release branch - whatever branch you're on when the release script is being run. Using our convention, we name this branch release/v1 (prefixed with release/ and a v plus the major version number only).

What does the release script do?

The release script starts by running some checks to make sure the code is in a state that's ready to release. There must be no uncommitted files in the repo, there must be no changes to tracked files, and there must be no unpushed changes in the repository (local commits that haven't been pushed to the remote).

If those conditions are met, then the release script proceeds. The release script will start by creating a git tag, which records the date, time, and branch being released to. Next, the head of the branch to release to (usually main) will be reset to the head of the branch to release from (the "frozen" code that's all fixed up for the release). Finally, the release script will push the results of the reset operation, and the new tag, to the remote.

To keep it more general, the release script uses the concept of a "source" and "destination" branch - the source is the branch to release from, the destination is the branch to release to. The destination branch is the one whose head is reset to the source branch's head. (That should help clarify the script below a bit more.)

Also, the Makefile (which we cover below) will take care of providing the right destination and source branch names.

Release Script

Here's an example release script: 

#!/bin/bash
set -euo pipefail
set -x

REMOTE="origin"

# Check that environment file has been sourced
if [ -z "${PROJECTNAME_PACKAGENAME_HOME}" ]; then
	echo 'You must set the $PROJECTNAME_PACKAGENAME_HOME environment variable to proceed.'
	exit 1
fi

# Check the script is being called correctly
if [[ $# != 2 ]]; then
    echo "Given a source (pre-release) branch and a destination (release) branch,"
	echo "this script does the following:"
	echo " - create a git tag"
	echo " - reset head of destination branch to head of source branch"
	echo " - push result to git repo"
    echo
    echo "Usage: $(basename $0) source_branch dest_branch"
    echo "Example: $(basename $0) release/v2 main"
    exit 1
fi

# Check that all changes are committed
if ! git diff-index --quiet HEAD --; then
    echo "You have uncommitted files in your Git repository. Please commit or stash them."
    exit 1
fi

export PROMOTE_FROM_BRANCH=$1 PROMOTE_DEST_BRANCH=$2

# Check that there are no local commits that haven't been pushed yet
if [[ "$(git log ${REMOTE}/${PROMOTE_FROM_BRANCH}..${PROMOTE_FROM_BRANCH})" ]]; then
    echo "You have unpushed changes on your promote from branch ${PROMOTE_FROM_BRANCH}! Aborting."
    exit 1
fi

RELEASE_TAG=$(date -u +"%Y-%m-%d-%H-%M-%S")-${PROMOTE_DEST_BRANCH}.release

# Check whether there are commits on the destination branch that aren't on the source branch (changes would be thrown away)
if [[ "$(git --no-pager log --graph --abbrev-commit --pretty=oneline --no-merges -- $PROMOTE_DEST_BRANCH ^$PROMOTE_FROM_BRANCH)" != "" ]]; then
    echo "Warning: The following commits are present on $PROMOTE_DEST_BRANCH but not on $PROMOTE_FROM_BRANCH"
    git --no-pager log --graph --abbrev-commit --pretty=oneline --no-merges $PROMOTE_DEST_BRANCH ^$PROMOTE_FROM_BRANCH
    echo -e "\nYou must transfer them, or overwrite and discard them, from branch $PROMOTE_DEST_BRANCH."
    exit 1
fi

# Check that untracked files are not present
if ! git --no-pager diff --ignore-submodules=untracked --exit-code; then
    echo "Working tree contains changes to tracked files. Please commit or discard your changes and try again."
    exit 1
fi

# Perform the actual release operations
git fetch --all
git -c advice.detachedHead=false checkout ${REMOTE}/$PROMOTE_FROM_BRANCH
git checkout -B $PROMOTE_DEST_BRANCH
git tag $RELEASE_TAG
git push --force $REMOTE $PROMOTE_DEST_BRANCH
git push --tags $REMOTE

Let's break down the essential commands, starting with the checks:

  • git diff-index --quiet HEAD --
  • git log ${REMOTE}/${PROMOTE_FROM_BRANCH}..${PROMOTE_FROM_BRANCH}
    • Checks the difference between the remote and local versions of the promote from branch
    • If this command turns up any commits, those are all local, unpushed commits
  • RELEASE_TAG=$(date -u +"%Y-%m-%d-%H-%M-%S")-${PROMOTE_DEST_BRANCH}.release
    • Creates a name for the git tag with the date and time, and branch being released to
    • This stores the release history in git tags
  • git --no-pager log --graph --abbrev-commit --pretty=oneline --no-merges -- $PROMOTE_DEST_BRANCH ^$PROMOTE_FROM_BRANCH)
    • This command checks for any commits that are on the destination branch and not on the source branch
    • Because the destination branch's head will be reset, commits on the destination branch but not on the source branch would be lost
    • (This script could optionally add a --force flag, to power through the release even if this check fails)
  • git --no-pager diff --ignore-submodules=untracked --exit-code
    • Checks if there are any untracked changes in the working tree
    • If there are, the release can't proceed

Now here's a breakdown of the release process commands:

  • git fetch --all
    • Ensures we have an up-to-date picture of where the remotes are at
  • git -c advice.detachedHead=false checkout ${REMOTE}/$PROMOTE_FROM_BRANCH
    • Checks out the source branch from the remote
    • This command is the reason why we have to make sure all local commits are pushed to the remote
    • The release process uses the remote version of the source/destination branches
  • git checkout -B $PROMOTE_DEST_BRANCH
    • This step forces the destination branch to be the same as the source branch.
  • git tag $RELEASE_TAG
    • Creates a record of what was released and when via a git tag
  • git push --force $REMOTE $PROMOTE_DEST_BRANCH
  • git push --tags $REMOTE
    • Pushes the results of the operations to the remote

Makefile

Now we add a Makefile with rules for releasing and deploying.

The first rule is the release rule - run this when the current branch is release/vX and it's ready for it's final release. 

CB := $(shell git branch --show-current)

release_mainx:
	@echo "Releasing current branch $(CB) to mainx"
	scripts/release.sh $(CB) mainx

The $(CB) is short for current branch.

Now on to the deploy rule.

Because we might want to deploy the code in an arbitrary state (temporarily deploying a feature branch to the development stage, for example), the deploy script and deploy rule are intended to deploy the code in its current state, and don't have the same kinds of checks as the release script. 

deploy:
	scripts/deploy.sh

As mentioned above, the deploy script will be case-specific so we don't provide an example.

How does the deploy script know which environment to deploy to? It depends on the environment file that was sourced, and the value of the PROJECTNAME_PACKAGENAME_STAGE variable. The deploy script should be checking that that environment variable is set to a valid value before proceeding with the deploy.

Retrieved from "https://charlesreid1.com/w/index.php?title=Applied_Gitflow&oldid=28741"