Handling merges and conflicts in an encrypted GitHub Zettelkasten

Git-crypt doesn’t do merges out-of-the-box

Since I implemented an encrypted GitHub repository for my Zettelkasten, I was thrilled using it, suspending work on my Chromebook to resume it on my Desktop PC, later continuing on my smartphone using the excellent Epsilon Notes app — until I ran into a merge conflict. I had edited the same (daily) note on two devices without prior syncing.

At this time, I realized, git-crypt does not support merging! Consequently, it always treats conflicting files as having no difference from the local version of the main branch. Which, if you commit them, essentially leads to you always overwriting what had been pushed already with what you have locally: You will lose the remote changes.

Of course, you can avoid that. The easiest option when encountering a merge conflict would be to just git clone the GitHub repository into a different folder and then inspect the conflicting notes or using a command line or GUI diff (and merge) tool.

However, we need to fix the merge-inability of git-crypt. Luckily enough, there is a fix: we "just" need to add a merge handler script to git-crypt.

Installing a merge handler for git-crypt

Obsidian / Computer

I have taken the essentials of this solution from this GitHub issue, which looks promising. So, first, we need to add to .git/config. In preparation for other repositories, we create a file with these contents, name it git-config-for-merge, and place it in the root of our vault:

At the root of the repository, our vault, we also create the script my-merge-tool.sh:

And the final file we create is an installer, which we call install-merge-tool.sh:

Then we make sure it’s executable and run it:

This copies the merge tool into the bin folder of your home folder and makes sure it's executable and updates your git config file accordingly. Unfortunately, the merge tool cannot be executed from within the repository's root folder, since on Android, the repository has to reside in the shared folder, which does not allow for executable permissions and removes them automatically.

Finally, we modify your .gitattributes to add a merge option at the end of each git-crypt-relevant line:

Voila! Now we have the tooling and config in place.

One last thing: We add and push the merge tool to git:

• either via Obsidian push
• or manually:

So from now on, we’ll have it available on all other systems, same goes for .gitattributes. That will save us some time on other systems.

Other systems

Since we checked in the merge tool, we need to pull in those changes and run the installer:

• via Obsidian: git pull
• or via command line: git pull

Then we execute the merge tool installer:

Android

We repeat what we did for Obsidian on other systems:

We open termux and type:

Note that your .git directory will be in a different place if your Zettlkasten folder is not named zettelkasten.

How to handle merges

Four kinds of merge situations can come up when pulling remote changes:

1. Pull fails due to uncommitted changes
2. Push fails because the GitHub repository is ahead
3. Standard merge, where git handles everything for you
4. Merge fails with conflict markers in affected files

Merge situations

Case 1: Pull fails due to uncommitted changes

In Obsidian, on pull, you will get the notification that “pull failed due to uncommitted changes”.

In this case, use its “push” function to commit your local changes. Of course, the push itself will fail because you need to pull in the remote changes first. But that doesn’t need to be an issue yet.

If the push fails:

Case 2: Push fails because GitHub repository is ahead

This is the case when you change a file, like your daily note, that previously has been changed on another device and when those changes have been pushed already. In this case, the push will fail, and git will ask you to pull first.

When pulling now, you’ll encounter one of the other two scenarios.

Case 3: Standard merge

This happens when your changes affect different files or if you changed a file, like your daily note, in two distinct locations on two different devices. For example, you change the top paragraph of your daily note on your main computer and forget to push those changes. Then, on your smartphone, you change the very last paragraph and push these changes. Back on your computer, you want to push, and it fails (but your local changes will be committed locally). Since git asks you to pull first, you pull.

If you try pulling with the Obsidian git plugin now:

• The notification will be the regular, small “… Changed n files” notification
• git will merge the changes and commit them
• the next time you push, this merge will be pushed along with the other changes you will have made by then

So all is fine!

On Android or the command line:
On Android, when you tap the pull.sh home screen widget or on the command line, if you prefer that, you issue the git pull command inside your Zettelkasten folder, the following will happen:

Git will see that the changes do not conflict: It will fix your daily note for you: taking changes from the GitHub version and the local changes into account. However, git usually wants to commit these changes immediately. So it will drop you into an editor where you can edit the pre-defined commit message:

You can safely exit the editor, saving this message. In case you’ve been dropped into the vi editor, just press :x - all other editors will display some menu.

Now you can push the combined changes with the git push command.

Case 4: Merge fails with conflict markers in affected files

This is a tricky one since the plugin gives you no clear indication that it happened. That means:

Any time a push fails, either pull from the command line or pay extra attention to the pull notification size! It will be large in case of a failed merge!

When you pull, the Obsidian git plugin will not warn you, but your files will end up containing conflict markers with both conflicting version and original version. See Appendix A: Testing merge for more information.

To actively search for conflict markers, search for <<<<<<< in your vault.

From the appendix:

Any conflicts can be found when searching for <<<<<. This line is followed by the local version of the line in question. In case of multiple consecutive lines, all of them will show up here. This section is followed by the separator =======. After that, the remote version of the chunk in question is presented until the end marker >>>>>>>.

Now you can decide how to go about these conflicting changes.

On Android or the command line:
Here, the pull and merge conflict situation is easier to detect: The attempt to auto-merge changes upon pulling results in a clear error message. It looks like this:

At the end, in the line:

git tells you exactly which files contain conflicts. Now you need to hunt for conflict markers inside those files, not your entire vault.

And that’s it. You see, the worst that can happen now is conflict markers in your notes that are easy to find. You will never lose any of your changes.

Appendix A: Testing merge

First, we create a new branch in our Zettelkasten, so we can play around without causing permanent damage:

OK, let’s create a note mergetest.md and push it:

Then, we will clone the GitHub repository into a second working copy named mergetest:

So now it’s time to make a change in the mergetest vault:

We have just modified the note. But we didn’t commit or push it. So what happens when we change this very note in our main vault, push the change, and try to pull it into our test vault? We will have a merge conflict!

So let’s try it out:

Now let’s return to our mergetest vault and see what happens when we pull:

And voila! We get a serious error message 🙂!

We can look at the differences:

OK, so far, this is working. But this merely prevents us from pulling and doesn’t cause any merge conflicts yet.

To provoke a conflict, let’s commit in the mergetest branch and try to pull:

WOW! We can see the git-crypt merge tool was called!

So let’s examine mergetest.md:

Yayyy! 🥳 This is exactly what we wanted! It may look cryptic, but git tells us precisely what is going on:

Any conflicts can be found when searching for <<<<<. The local version of the line follows this line in question. In the case of multiple consecutive lines, all of them will show up here. The separator follows this section =======. After that, the remote version of the chunk in question is presented until the end marker >>>>>>>.

Now you can decide how to go about these conflicting changes.

In the end, don’t forget to switch back to the main branch:

And all your tests are gone.

Appendix B: Future-proofing the Android scripts

Here, we will tidy up our scripts — so they become cross-platform friendly — and we don’t get any surprises if we replace them with some other script we developed for another system.

Problem with widgets: /usr/bin/env does not exist

So consider log.sh:

Ideally, we’d like to prefix that with a shebang line: #!/usr/bin/env sh:

Even though our script runs on the command line, when we invoke it with

it now stopped working when tapped on the home screen, thanks to the “shebang line”.

So what we do, is: We create a wrapper called log in .shortcuts/ that invokes log.sh and remove the link to log.sh, so that log can become our new shortcut.

We’ll go into more details about how and why the solution works later.

This is how we do it:

On our Android home screen, we remove log.sh, and add log via the termux widget.

We repeat the same procedure for pull.sh and push.sh.

How and why this works

The source of the solution is termux-exec.

According to the termux wiki, termux-exec is already installed on current versions of termux.

How does termux-exec do that? It installs a wrapper to the execve() system call that translates all paths to /bin/ and /usr/bin/ to their termux counterparts.

How can it do that? With the LD_PRELOAD mechanism: You can set the environment variable LD_PRELOAD to one or more libraries you want to load before all others. This can be used to override functions in other shared objects selectively. In the case of termux-exec, the execve() system call is overridden. When the termux console starts, it automatically checks for an LD_PRELOAD override configuration - which it finds since termux-exec is installed.

Why do our wrapper scripts work in combination with the home screen widgets? Apparently, the widget executes our scripts differently from the termux shell in the console: The LD_PRELOAD is not in effect. However, once our wrapper script is running, the shell, not a widget, executes the script named in the wrapper - this is much more similar to the command line invocation, and the LD_PRELOAD trick works again.

Originally published at https://renerocksai.github.io on January 21, 2021.