Moving commits

Shepherding Git commits from dev to stage to production.

I like following a Git development model with 3 main branches. A good description of the model (with diagrams) can be found in the Successful Git branching model blog post. The main features of this model are:

  1. There is a master (or development) branch. All commits are merged there (usually after code review), and I like to collapse separate feature branches into a single commit before landing the feature on the master branch.
  2. The latest master branch is deployed to a dev server for testing.
  3. Once a feature is confirmed working on the dev server, we can cherry pick it onto stage branch. Again, there is a corresponding staging server, where the latest stage branch gets deployed after each commit.
  4. Once a feature is confirmed to work in staging, we cherry pick it to the production branch.
  5. The decision when and how to deploy the production branch is complex. I recommend waiting for a meaningful list of features to release and then use "blue green" deployment model to ensure clean release.

If each commit corresponds to (roughly) a single feature, then we need to a way to figure out which commits are still on dev, which ones are on stage and which ones are already on prod branches. This can get quite complex, since the cherry picking the commits might be done by a separate testing team, not the original developer. Here are a couple of Git commands to use when trying to figure out which features (commits) can be moved from branch to branch.

Plain file comparison

Imagine we have the latest master (which is our development) branch locally. How is the file different from the same file on stage branch? If both branches are available locally then it is simply

git diff master stage <filename>

If you are already on the "master" branch and want to compare the file to "stage", you can use "..stage" notation (".." means "from the current branch")

git diff ..stage <filename>

In general, Git uses the notation <to>..<from> when comparing things (files, commits)

git diff master..stage <filename>

In our case, the master is the current code, the stage is behind, thus stage is the from point.

Most likely you will not have the remote branches locally (that's why they are remote!). In this case, you will need to fetch information about available remote branches (without the content) and then compare the file

git fetch --all
git diff master origin/stage <filename>
git diff ..origin/stage <filename>

Comparing logs

Next, let us look at the big picture - there are lots of commits on "master", which ones are on the remote "stage" already and which ones are not yet? I use the following command to see the list of local commits on the current branch:

git log --oneline

Since I am mostly interested in my own commits

git log --oneline --author gleb

Similarly, we can view the list of commits by specific user on the remote branch

git fetch --all
git log --oneline --author gleb origin/stage

Since our local environment might be behind, I recommend looking at the remote master as the ground truth

git log --oneline --author gleb origin/master

Now we can compare the two lists to find my commits that are on master but not on stage yet.

git log --oneline --author gleb origin/stage..origin/master
14a0104 feature one
c8c910f feature two
...

Similarly, we can check which commits are already on stage, but not on prod

git log --oneline --author gleb origin/prod..origin/stage

With this information we can cherry pick specific commits from one branch to the next one.

Picking (moving) commits

Once we have a bunch of commits, let us move them from one branch to another. For this you do need the actual branches locally. Once you have local branches "stage" and "master" let us get the "stage" commits

git checkout stage
git pull --rebase origin

Then compare the local list of commits again (using the local branch "stage" against the local branch "master")

git log --oneline --author gleb ..master

Then take the commit you want to pick and apply, for example aaabbb

git checkout stage
git cherry-pick -x aaabbb

The option -x is important here - the commit lands on "stage" with a NEW SHA ID. Thus to know the original commit, we want to know where it came from. The option -x adds a line to the commit on "stage" with the source information, something like

commit dddeee
feature ...
(cherry picked from commit aaabbb)

Which will be very useful in the future when trying to figure out which commits are still have not been moved from "master" to "stage"

Another example

Let's say our work branch is called develop and our production branch is master. When we are happy with the code in develop branch we want to push some commits to master. Pull all code then switch todevelopbranch. Then show all commits that are NOT onmaster` yet.

1
2
git checkout develop
git log --oneline master..

If we switch to master we need to reverse the direction and place .. before the source branch name

1
2
git checkout master
git log --oneline ..develop

List of commits yet to be cherry picked

With the above approach, if we do not rebase, pretty soon we will see a lot of commits in the difference log between the two branches. The git log does not understand that the two branches have the same commits (since cherry picked commit gets a new ID), thus shows the same stuff over and over. A good thing is that if we try to cherry pick the same commit several times, the diff will be empty (unless there are file changes), and will not go through without --allow-empty option.

One way to show which commits have not been cherry picked yet is to use the git cherry command

git cherry stage master
- aaabbb
+ ffffff

Which shows a list of commits marked '+' that have not been cherry picked yet (ffffff in this case). I prefer adding -v option to show the messages with each commit ID.

This is much shorter than the options available in git log command for filtering the already picked commits:

git log --left-right --cherry-pick --oneline stage...master
> ffffff

Test sandbox

For practice, here are a couple of commands to run in an empty folder that will create a couple of branches and will add a few commits. Just make a new folder and copy/paste

git init
git touch a.txt
echo start >> a.txt
git commit -am "start"
git checkout -b stage
git checkout master
echo one >> a.txt
git commit -am "feature one"
echo two >> a.txt
git commit -am "feature two"
echo three >> a.txt
git commit -am "feature three"

At this point we have 4 commits on "master", and 1 commit on "stage"

$ git checkout master
$ git log --oneline
824b453 feature three
aa3c02e feature two
4a038e3 feature one
79d06af start

$ git checkout stage
$ git log --oneline
79d06af start

There are 3 commits on "master" not on "stage"

git log --oneline ..master
824b453 feature three
aa3c02e feature two
4a038e3 feature one

All 3 commits can be cherry picked from "master" to "stage"

$ git cherry -v stage master
+ 4a038e331b1c16a1a8847a6e91a5516dbd06bd6b feature one
+ aa3c02ec512acc57a1c02ac5ddc7aff480b4cb1d feature two
+ 824b453d61477e9d59c963892eab0e712be0a5f4 feature three

Let us pick "4a038e3 feature one" commit to the "stage" branch.

git cherry-pick -x 4a038e3

Now we have the same commit but under the different SHA. We can plot the divergent branches using git log with a few more options

1
2
3
4
5
6
7
$ git log --oneline --graph --all --decorate
* 727bd14 (HEAD -> stage) feature one
| * 824b453 (master) feature three
| * aa3c02e feature two
| * 4a038e3 feature one
|/
* 79d06af start

The list of commits that can be cherry picked is shorter

1
2
3
4
$ git cherry -v stage master
- 4a038e331b1c16a1a8847a6e91a5516dbd06bd6b feature one
+ aa3c02ec512acc57a1c02ac5ddc7aff480b4cb1d feature two
+ 824b453d61477e9d59c963892eab0e712be0a5f4 feature three

We can bring the second feature from "master" to "stage"

1
2
3
4
5
$ git cherry-pick -x aa3c02e
$ git cherry -v stage master
- 4a038e331b1c16a1a8847a6e91a5516dbd06bd6b feature one
- aa3c02ec512acc57a1c02ac5ddc7aff480b4cb1d feature two
+ 824b453d61477e9d59c963892eab0e712be0a5f4 feature three

At some point the log becomes too long, and you would want to limit it using a commit ID to stop the list (see git help cherry).

Good tutorials