# Part 3 - Publish Obsidian Articles to Grav CMS

**Written by:** Jan Bachelor

**Date:** 2026-08-27

[TOC]
## Introduction to Part 3

Part 1 got Obsidian self-hosted with sync across devices. Part 2 got Grav running publicly with some customizations. This article (Part 3) covers how they sync together using an isolated git clone, a small Python conversion script, and a systemd timer, publishing real Obsidian notes to Grav automatically, with no plugin holding write access to live content.

It also covers a second thing that turned out to need solving along the way: getting screenshots pasted straight from Obsidian to stop showing up as multi-megabyte, randomly named files, and getting that whole publish step down to a single deliberate action, without leaning on Obsidian Git's own commit button at all in the end.


### The pipeline, end to end

```mermaid
flowchart LR
    A["Obsidian vault<br/>Published/ folder"] -->|Commit and Publish<br/>custom plugin, on demand| B["publish_vault.py<br/>resize + rename images,<br/>commit + push"]
    B -->|via scoped SSH deploy key| C["Gitea repo<br/>obsidian_grav_tutorials"]
    C -->|git pull, every 5 min<br/>via a separate,<br/>read-only SSH deploy key| D["Isolated clone<br/>/opt/grav_source/repo<br/>(kept outside Grav's<br/>user/ tree)"]
    D -->|publish_inbox.py| E["Grav pages<br/>user/pages/NN.slug/"]
    E -->|bin/grav clearcache| F["tutorials.bachelor-tech.com"]
```

## The publishing pipeline
The actual chain when you press the hotkey:

1. Once you want to publish, you run the **Commit and Publish** from within Obsidian using a custom plugin.
2. This triggers `main.js` **on the desktop**, which computes the repo path (`<vault>/Published`, the `REPO_SUBPATH` constant) and runs `pythonw.exe publish_vault.py` with that as the working directory.
3. Then `publish_vault.py` kicks in, sitting directly at the _top_ of `Published/` (not inside any article's subfolder), imports `optimize_images.py` from that same top-level location, **runs the resize/rename pass**, then does `git add`, `commit`, and `push` itself.
4. Content gets **pushed to Gitea** with the resized + renamed images.
5. Within 5 minutes, a timer-based script called `publish_inbox.py` on the server (`web1` in my case) notices new content, pulls it and **converts it to a Grav page**. 

What we will do below is to cover the pipeline below step-by-step.

## Using deploy keys with Gitea/Forgejo

While other commercial tools like Github/Gitlab allow you to create a personal key that is scoped to a repository, this restriction is (so far) not possible with Gitea and Forgejo. However, there is a different option to use with these - deploy keys!

NOTE: Gitea v1.23 supports them now!

The first working version of this pipeline used a single Gitea Personal Access Token, scoped to the whole account rather than one repository, for both directions: the desktop pushed with it, and the web host pulled with it. That works, but it means one leaked credential (in a config file, a shell history, a screenshot) exposes every repository on the account, not just this one, and it can't be revoked for one consumer without breaking the other.

The fix: a **deploy key** per repository per direction, each one scoped to `obsidian_grav_tutorials` alone and nothing else.

- `desktop-obsidian-grav-push`, write access, lives on the desktop, used only to push.
- `web1-grav-pull-readonly`, read-only, lives on web1, used only to pull. Read-only matters here: this side of the pipeline never needs to write back to Gitea, so the key simply can't, even if something on that host were compromised.


### Desktop side

```powershell
git remote set-url origin "git@<gitea-host>:jan/obsidian_grav_tutorials.git"
git config core.sshCommand "ssh -i C:/Users/<you>/.ssh/desktop-obsidian-grav-push -o StrictHostKeyChecking=accept-new"
```

Two gotchas worth knowing before you hit them:

- **Use the Gitea host's LAN IP directly, not its public hostname.** The reverse proxy in front of it only forwards HTTP and HTTPS, not raw SSH, so pointing `git@` at the hostname just times out. The IP only needs to resolve to something on the same LAN (or reachable over your VPN) as the desktop.
- **Use forward slashes in the identity path, even on Windows.** `core.sshCommand` is handed off to Git for Windows' bundled `sh.exe`, which treats a backslash as an escape character, not a path separator. A path typed as `C:\Users\jan\.ssh\key` silently comes out the other side as `C:Usersjan.sshkey`, ssh can't find the file, and it quietly falls back to a password prompt instead of failing loudly. Forward slashes sidestep the problem entirely.

A quick way to confirm it's actually using the key rather than falling back: `git fetch` should return cleanly with no password prompt at all.

### web1 side

```bash
git -C /opt/grav_source/repo remote set-url origin "git@<gitea-host>:jan/obsidian_grav_tutorials.git"
git -C /opt/grav_source/repo config core.sshCommand "ssh -i /opt/grav_source/deploy_key -o StrictHostKeyChecking=accept-new"
```

No backslash problem on Linux, same direct-IP requirement. Once both sides were confirmed working (a real `git push` from the desktop, a real `git pull` on web1, both clean), the old account-wide token was deleted from Gitea entirely.

## Keeping the clone outside Grav's content tree

Given what Git Sync did to `user/`, the isolated clone lives at `/opt/grav_source/repo`, a plain directory with no relationship to Grav at all beyond being read by the conversion script below. It is not mounted into the container, not referenced anywhere in Grav's own config, and gitignoring or deleting anything in `user/` has no effect on it.

Create it once, then point it at the deploy key that we set up earlier:

```bash
sudo git clone "git@<gitea-host>:jan/obsidian_grav_tutorials.git" /opt/grav_source/repo
sudo git -C /opt/grav_source/repo config core.sshCommand "ssh -i /opt/grav_source/deploy_key -o StrictHostKeyChecking=accept-new"
```

If this clone were ever wiped by accident, the fix is exactly the same command again, nothing more.



## A timer, not a webhook

The original open question was webhook versus polling. Polling won, for the same reason it was already used for the WordPress uploads sync elsewhere in this environment: no inbound endpoint to secure, and it self-heals automatically if the host was offline when a push happened, rather than needing that push replayed later.

```ini
# /etc/systemd/system/grav-publish.timer
[Unit]
Description=Run grav-publish.service every 5 minutes

[Timer]
OnBootSec=2min
OnUnitActiveSec=5min
AccuracySec=30s

[Install]
WantedBy=timers.target
```

```ini
# /etc/systemd/system/grav-publish.service
[Unit]
Description=Convert Grav _inbox/ content into published pages
After=docker.service

[Service]
Type=oneshot
ExecStart=/usr/bin/python3 /opt/grav/publish_inbox.py
```

Save both files to `/etc/systemd/system/`, save `publish_inbox.py` itself to `/opt/grav/publish_inbox.py` (matching the `ExecStart` path above), then enable and start the timer:

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now grav-publish.timer
```

Trigger one run immediately rather than waiting up to 5 minutes, and confirm it actually worked:

```bash
sudo systemctl start grav-publish.service
journalctl -u grav-publish.service --no-pager -n 20
```




## Converting Markdown into Grav pages: `publish_inbox.py`


Each run does three things: pull the isolated clone, walk its content, and write Grav pages under `user/pages/`, numbered from `10.` upward so `01`/`02` stay free for Grav's own default pages.

- **Loose `.md` files** at the repo root become single, standalone pages.
- **Subfolders** become multi-part series. Every `.md` file inside is a part, ordered by a leading "Part N" in its filename where present, alphabetically otherwise. Any image anywhere in that folder (including a nested `assets` subfolder) is available to every part in the series. A folder with exactly one `.md` part is collapsed to a direct single page instead, no index page with a one-item "Parts in this series" list, that was just an extra click to reach the only real content. The URL still comes from the *folder's* own slug in that case, not the part's own title-derived one, on purpose: a one-off tutorial that lives in its own folder for the sake of a nearby `assets/` folder shouldn't have its URL depend on whether it happens to have one part or three.
- **Title** comes from the first real `# Heading` line in the file, with any heading that falls inside a fenced ```` ``` ```` code block correctly ignored (more on why below). Falls back to the filename if no real heading exists at all.
- **Category and tags** come from an optional YAML frontmatter block at the very top of the file, Obsidian's own Properties panel writes this natively, no plugin needed:

  ```yaml
  ---
  category: OPNSense
  tags: [firewall, vlan]
  ---
  # The actual article title
  ```

  Stripped from the body before title extraction runs, and fed into that page's Grav `taxonomy:` frontmatter, which is what the categorized blog homepage (Part 2) filters and groups by. Entirely optional, an article with no frontmatter block just has no category or tags, nothing breaks. A series takes its category and tags from whichever part defines them first, in part order (usually Part 1), since the whole series shows as one card on the homepage, not one per part.
- **Wikilinks** (`[[Note Name]]`, `[[Note Name|Display text]]`) get rewritten to relative Grav links, resolved against a title-to-slug map built across the whole batch before any page is written.
- **Images**, whether Obsidian's `![[image.png]]` embed syntax or plain Markdown `![alt](image.png)`, get resolved by filename, copied alongside the generated page, and given a lightbox plus a size class based on the image's own height, so a short, wide screenshot doesn't render as a tiny sliver. A URL-encoded path (`%20` for spaces, common when standard Markdown syntax is written by an editor rather than typed by hand) is decoded before that filename lookup, otherwise it silently never matches anything on disk.
- **Images written as raw HTML**, the shape the sibling Notion-to-Gitea backup script emits (`<p align="center"><a href="assets/x.png"><img src="assets/x.png" ...></a></p>`), never Markdown syntax, get converted into plain Markdown image syntax first, then flow through the exact same resolution/copy/classing logic as the bullet above. Patching the raw HTML's `src`/`href` in place directly was tried first and still 404'd: a bare relative path in raw HTML is resolved by the *browser's* own URL rules, which drop the page's last path segment, fine on a page with a single-segment route, wrong (one directory too shallow) on a nested series/part page. Grav's own Markdown image handling doesn't have that problem, it resolves against the page's actual route however deep the nesting, which converting to Markdown syntax first gets for free.
- **Mermaid diagrams**: a ```` ```mermaid ```` fenced block gets translated into `[mermaid]...[/mermaid]`, the shortcode Grav's `mermaid-diagrams` plugin actually expects, since that plugin has no idea what a fenced code block is. Keeping the source in standard Markdown syntax means it still renders correctly in Obsidian and in Gitea's own preview too, only Grav needs the translation. That plugin has to actually be installed for this to do anything, `docker exec grav bin/gpm install mermaid-diagrams -y`, without it the `[mermaid]...[/mermaid]` shortcode this script produces just shows up as literal text on the page, nothing breaks, it simply isn't a diagram.
- **Obsidian callouts** (`> [!info] Title`, `> [!warning]`, and so on) get their opening line rewritten into GitHub's own alert syntax instead: `> [!NOTE]`, `TIP`, `IMPORTANT`, `WARNING`, or `CAUTION`, exactly five fixed types, Obsidian's wider vocabulary mapped down to whichever is closest. This Grav install already has a plugin for that exact syntax (`github-markdown-alerts`), so the result is a real titled, colored box instead of a plain blockquote with `[!info]` sitting there as literal text, which is all Grav's own Markdown parser would otherwise make of it. GitHub's syntax has no slot for a custom title, so an Obsidian title becomes a bold first line of the body instead of getting silently dropped. Everything else in the callout, lists, bold text, links, is left completely untouched and renders exactly like anywhere else on the page, since it's still the same `>` blockquote marker underneath, this script only ever touches the opening line.
- **Fenced code blocks and inline code spans** are protected from all of the rewriting above. Nothing inside a fence or a single-backtick `` `span` `` (a bash comment, a mermaid subroutine shape that happens to look like a wikilink, or a literal `` `[[Note Name]]` `` written as a syntax example) is ever mistaken for real Obsidian syntax.
- **A plain-text download copy** of each article gets written alongside `default.md` (see Part 2 for the "Download Markdown" link and "Save as PDF" button that actually expose it): the *original* markdown, the same content the rewriting above is about to transform, not Grav's shortcode/querystring output, so it's something a visitor could genuinely reuse elsewhere, another vault, GitHub, wherever. Written with a `.txt` extension rather than `.md`: confirmed directly that Grav refuses to serve a bare `.md` file sitting in a page folder as a static download (the same bytes come back `404` as `.md`, `200` as `.txt`), presumably deliberate since `.md` is Grav's own content-file extension.
- **Ordering** at the top level is newest first, by each entry's earliest git commit date.
- **Cleanup** runs via a manifest file (`/opt/grav/.publish_manifest.json`) recording what the script itself created. A renamed or removed article has its old page removed automatically. Anything not in the manifest, meaning Grav's own default pages, is never touched.
- **`Home.md`** at the vault root overwrites Grav's own reserved homepage (`01.home/default.md`) directly, instead of becoming a numbered page like every other standalone article. Left untouched, never reset, if `Home.md` isn't present in a given run. That page is forced to `template: blog` (the categorized homepage covered in Part 2), so `Home.md` itself only ever supplies its title and any intro text above the listing, same relationship the next two bullets have to their own pages.
- **`Search Articles.md`** at the vault root works the same way for `02.search/default.md` (the flat, searchable listing, "Search Articles" in the sidebar, formerly named `All Articles.md` / `02.articles` before the site grew a proper categorized homepage, see Part 2), only controlling that page's title and intro text. The actual listing there is generated live by a custom Twig template (`articles.html.twig`), not stored as page content anywhere.
- **`About.md`** at the vault root is the same idea again, for `03.about/default.md`, the personal "About" page.
- **Ownership**: the script runs as root (via the systemd service), so it re-chowns everything under `user/pages/` to the container's `www-data` UID/GID (`1000:33` in this setup, confirmed with `docker exec grav id www-data`) at the end of every run. Without this, Grav's own admin UI, which edits and deletes pages as `www-data` from inside the container, fails with a permission error the moment it touches anything this script wrote.


## Verifying it end to end

No dashboard for any of this, so checking it directly:

```bash
# does the clone see the latest commit from Gitea
git -C /opt/grav_source/repo fetch
git -C /opt/grav_source/repo log --oneline -3 origin/main

# is the timer actually enabled and running on schedule
systemctl status grav-publish.timer

# what happened on the last few runs
journalctl -u grav-publish.service --no-pager -n 40

# what actually landed on disk
find /opt/grav/user/pages -maxdepth 3
```

A clean run's log looks like `wrote 10.some-article/default.md <- Source File.md` for each page, followed by `Done. N top-level page(s) processed, cache cleared.` A `removed stale ...` line means the cleanup above just caught a rename or deletion, which is expected behaviour, not an error.

## Why not Obsidian Git's own commit either

Everything above only cares that a commit lands on Gitea; it doesn't care who made it. Obsidian Git handled that well enough on its own for a while. Then a second problem showed up: screenshots pasted straight from Obsidian kept landing in Gitea as multi-megabyte files with names like `Pasted image 20260825221518.png`, which gets messy the moment you have more than a few.

The obvious place to fix it was a **git pre-commit hook**: resize and rename anything staged, right before the commit that would carry it to Gitea. That's the same shape as the sibling Notion-to-Gitea backup script this whole idea is modeled on, which does its own image handling inline as part of one script that owns the entire upload.

It doesn't work with Obsidian Git, though. A pre-commit hook only ever fires for a commit made through the real `git` binary, and testing this directly showed that Obsidian Git's own commits don't invoke local git hooks at all: a real "add" commit went through with the original, untouched file, no hook output whatsoever. Obsidian Git most likely commits through its own bundled JavaScript git implementation rather than shelling out to the system's `git.exe`, even on desktop. Whatever the exact reason, no hook can ever fire for it.

So the fix doesn't try to hook into Obsidian Git's commit at all. Instead, a single script (`publish_vault.py` does the resize/rename pass and the commit and the push itself, driving `git` directly. A small custom Obsidian plugin gives that script one deliberate trigger: a command in the palette, or a hotkey, run whenever you're actually ready to publish.

## 7. Optimizing images before they ever reach Gitea

`optimize_images.py` scans the whole vault and, for every image it finds:

- **Downscales it** if wider than 1600px, preserving aspect ratio. This is what actually fixes an oversized screenshot's appearance in Gitea's own file preview: plain Markdown image syntax has no width attribute the way raw HTML does, so the only real fix is making the file itself smaller. Grav's own rendering is unaffected either way, it sizes images by relative CSS percentage of the page column, not fixed pixels, so a smaller source file just means less to download, not a different displayed size.
- **Renames it** to `Part-N-Heading-Slug-NN.ext`: `N` is the part number from the referencing note's own filename, `Heading-Slug` is the nearest H2 heading above the image (falling back to the nearest heading of any level if there's no H2 above it), and `NN` is a two-digit counter scoped to that exact heading, so the first image under a given H2 is `01`, the second is `02`, and so on. Case is preserved from the heading text on purpose, this is meant to read as a real, SEO-friendly filename, not a lowercased slug. The file gets moved into an `assets/` folder next to the note that references it, matching the convention already used elsewhere in this vault.
- **Rewrites the reference** to a renamed file, in every note, to the correct new path.

It's idempotent: a file already named exactly what a given run would produce is left alone entirely, so running it repeatedly never reshuffles numbers or touches an image twice. Numbering also picks up correctly from whatever's already on disk, so adding a third image under a heading that already has `01` and `02` from an earlier session correctly becomes `03`, not another `01`.

This applies to every image it can find a note reference for, not only Obsidian's own auto-generated paste names. A deliberately-placed diagram or icon gets the same treatment, since the goal is a consistently organized `assets/` folder, not just cleaning up pasted screenshots specifically.

### Two things worth knowing if you adopt this yourself

**It regenerates `__pycache__`, and that's worth excluding from sync.** Importing this module from `publish_vault.py` creates a `.pyc` bytecode cache file right inside the vault folder by default. Left alone, git would happily commit it as a new untracked file, and worse, if the same vault is also synced elsewhere (LiveSync, in this project's case), a churning binary file that keeps changing hash can produce a real sync conflict on a file that never should have been synced in the first place, which is exactly what happened during testing. The fix is one line, `sys.dont_write_bytecode = True`, set before the import, plus a `.gitignore` entry for `__pycache__/` as a second layer of protection.

**Renaming everything retroactively is a real decision, not just a default.** The first version of this script only touched Obsidian's own auto-generated paste names, leaving anything already named by hand alone. Extending it to rename every image it finds, including ones already live and referenced on the published site, was a deliberate choice: cleaner, more consistent naming across the whole vault, at the cost of changing URLs for content that's already public. If you only want this going forward, gate the rename on the filename still looking auto-generated (a simple regex check) rather than applying it to everything.

## Commit and Publish: a small custom Obsidian plugin

A custom Obsidian plugin was created to orchestrate the commit + push from Obsidian to Gitea, it is executed manually. Why a custrom plugin instead of the main Git plugin supplied with Obsidian?

### What it does

A single command, **Commit and Publish**, that runs `publish_vault.py` (which does its own image optimization, commit, and push, all in one pass) and shows one short notification with the result, something like `Published, 1 image renamed, 1 resized`, rather than a wall of console text to read through.

### Installing it

Download it from my [public Github repo](https://github.com/lbakyl/Converting-Markdown-into-Grav/tree/main/commit-and-publish-plugin). The plugin is two files, `manifest.json` and `main.js`, placed in `<vault>/.obsidian/plugins/commit-and-publish/`, then added to `.obsidian/community-plugins.json` and enabled the normal way (Settings → Community plugins), or picked up automatically on the next restart if you add its id to that file directly.

### Setting a hotkey for it

Obsidian doesn't ship a default hotkey for this, or for Obsidian Git's own commit command either, community plugin commands never get one automatically. To set one:

1. Settings → Hotkeys
2. Search for **"Commit and Publish"**
3. Click the **+** next to it, then press your chosen key combination

Any combination that isn't already bound to something else works. There's nothing to "match" from Obsidian Git specifically, since it never had a default of its own.

Alternatively, you can run it by pressing Ctrl (CMD) + P while in Obsidian and selecting 'Commit and Publish'.

## What's next

Only `web1` runs the pull/convert side today. The original plan sketched `web2` and `web3` pulling the same way, and nothing about the design prevents that: it's the same isolated clone, the same read-only deploy key (issued separately per host, same as web1's), and the same timer and script, copied to each additional host. Not yet built simply because there's only one live site to publish to so far.

### What the original design questions turned into

The first draft of this article listed five open questions before any of this was built. For the record, here's what each one turned into:

1. **What marks a note as ready to publish?** Folder location alone. A note either lives in the vault's `Published/` folder or it doesn't; there's no separate frontmatter flag yet. Worth revisiting if "finished but not ready" ever becomes a real, common state.
2. **Wikilink rewriting.** Handled by the title-to-slug map, built across the whole batch before any page is written, so a link's final Grav path is always known ahead of time.
3. **Folder structure mapping.** Numbered folders (`NN.slug`) generated deterministically from each entry's earliest commit date, newest first, starting at `10` to leave Grav's own default pages alone.
4. **Webhook versus polling.** Polling, on a five-minute systemd timer.

## Software versions at the time of write-up

- Python: 3.11.2 (web1), 3.11.5 (desktop)
- git: 2.39.5
- OS: Debian GNU/Linux 12 (bookworm) on web1
- Gitea: 1.21.10
- Obsidian Git plugin: 2.39.0