Skip to content

Git Sync

Git Sync keeps workflows and their supporting files synchronized with a Git repository. Pulling changes places DAG definitions, scripts, templates, binaries, Wiki pages, and Wiki attachments into their respective Dagu storage directories.

Repository layout

Set git_sync.path when workflow files occupy a subdirectory within a larger repository. Dagu ignores tracked files outside that directory.

text
repository
`-- workflows/                     # git_sync.path
    |-- daily-report.yaml          # DAG
    |-- scripts/
    |   `-- generate.py            # supporting file
    |-- templates/
    |   `-- summary.html           # supporting file
    `-- wiki/
        `-- operations/
            `-- daily-report.md    # Wiki page

After a pull:

text
paths.dags_dir/
|-- daily-report.yaml
|-- scripts/
|   `-- generate.py
`-- templates/
    `-- summary.html

paths.wiki_dir/
`-- operations/
    `-- daily-report.md

Paths under git_sync.path remain relative to their destination roots. Leave path empty when the repository itself is dedicated to Dagu files.

What Git Sync tracks

Git Sync assigns every managed file an itemId and kind. The Web UI, CLI, and REST API use these values.

Repository fileLocal destinationitemIdkind
daily-report.yamlpaths.dags_dir/daily-report.yamldaily-reportdag
scripts/generate.pypaths.dags_dir/scripts/generate.pyscripts/generate.pyfile
skills/review/SKILL.mdpaths.dags_dir/skills/review/SKILL.mdskills/review/SKILL.mdfile
wiki/operations/runbook.mdpaths.wiki_dir/operations/runbook.mdwiki/operations/runbookdoc
wiki/.attachments/operations/runbook/chart.pngpaths.wiki_dir/.attachments/operations/runbook/chart.pngwiki/.attachments/operations/runbook/chart.pngdoc-asset

Supporting-file IDs retain their complete relative path and file extension. DAG and Wiki page IDs omit .yaml, .yml, and .md.

An existing repository that contains docs/ but not wiki/ continues using docs/ for Wiki content. Git Sync rejects a repository containing both directories rather than merging them.

Supporting files

Any Git-tracked regular file in scope that is not handled as a DAG, Wiki page, or Wiki attachment is a supporting file. Common examples include:

  • shell, Python, PowerShell, and JavaScript files
  • SQL, JSON, TOML, and dotenv configuration
  • templates, certificates, and binary data
  • authoring files under skills/

Git Sync does not automatically discover new supporting files created directly in paths.dags_dir. Commit new files to the Git repository first, then pull them into Dagu. Once tracked, local edits and deletions appear in Git Sync status and can be published.

Within the synchronized scope, reserve .yaml and .yml extensions for DAG definitions. Base configuration files and YAML files under the Wiki tree are excluded from supporting files.

Git symlinks and submodules are not synchronized. Additionally, a repository cannot contain items that resolve to the same itemId. For example, workflow.yaml and an extensionless supporting file named workflow collide, causing the pull operation to fail without writing either item.

Review the synchronized path

A pull copies every tracked supporting file in scope. In a general-purpose repository, set git_sync.path to the workflow subdirectory so unrelated files are not copied into paths.dags_dir.

Use supporting files from a workflow

Set working_dir: . when local steps should run from the directory containing the DAG file:

yaml
working_dir: .

steps:
  - id: generate
    run: python scripts/generate.py --template templates/summary.html

For distributed execution, declare each file or directory under dependencies. Git Sync places the source files on the Dagu host; dependencies sends a snapshot to the worker.

yaml
steps:
  - id: generate
    run: python scripts/generate.py --template templates/summary.html
    dependencies:
      - scripts/generate.py
      - templates/summary.html

The worker still requires the runtime that executes the file (for example, synchronizing a Python script does not install Python). See File Dependencies for directories, globs, limits, and worker behavior.

Configuration

yaml
git_sync:
  enabled: true
  repository: github.com/your-org/automation
  branch: main
  path: workflows
  push_enabled: true

  auth:
    type: token
    token: ${GITHUB_TOKEN}

  auto_sync:
    enabled: true
    on_startup: true
    interval: 300

  commit:
    author_name: Dagu
    author_email: dagu@localhost

push_enabled: false allows pulls but blocks publish, delete, and move operations. This is the recommended setting for deployments that receive reviewed changes from a promotion branch.

Authentication

HTTPS token

yaml
git_sync:
  repository: github.com/your-org/automation
  auth:
    type: token
    token: ${GITHUB_TOKEN}

SSH key

yaml
git_sync:
  repository: git@github.com:your-org/automation.git
  auth:
    type: ssh
    ssh_key_path: /home/user/.ssh/id_ed25519
    ssh_passphrase: ${SSH_PASSPHRASE}

For an SSH server that requires a username other than git, include it in the repository URL:

yaml
git_sync:
  repository: deploy@git.example.com:your-org/automation.git
  auth:
    type: ssh
    ssh_key_path: /home/user/.ssh/id_ed25519

Work with changes

A typical update follows this sequence:

  1. Check the current status.
  2. Pull remote changes.
  3. Review local and remote differences.
  4. Publish selected local changes.
  5. Resolve conflicts or clean up missing items.

Status values

StatusMeaning
syncedLocal content matches the last synchronized version.
modifiedA tracked local file changed.
untrackedA local DAG, Wiki page, or Wiki attachment has not been published. New local supporting files are not discovered.
conflictLocal and remote content both changed, or a locally edited supporting file was deleted remotely.
missingA previously tracked local file is gone.

Pull and remote deletion

Pulling preserves local edits. If the remote repository deletes a supporting file:

  • An unchanged local copy is removed.
  • A locally edited copy remains and becomes a conflict.
  • Discarding that deletion conflict removes the local copy.
  • Force publishing the conflict restores the local file to the repository.

The CLI and Web UI report how many items were updated and removed.

Text, binary, and executable files

Text files show local and remote content in the diff view. Binary files show local and remote sizes instead of loading their content into the viewer.

Git Sync preserves whether a supporting file is executable. On Unix systems, changing that permission marks the file as modified so it can be published. Windows does not track this file permission mode directly, so published changes preserve the executable state recorded in the repository.

Web UI

Open Git Sync from the server navigation. The page provides separate filters for workflows, Wiki pages, attachments, and supporting files.

Use it to:

  • pull remote changes and view updated or removed counts
  • filter items by type and status
  • compare text diffs, binary sizes, executable permissions, and remote deletions
  • publish selected changes
  • discard, move, delete, or forget an item when permitted by its status
  • inspect the configured repository, branch, push mode, and auto-sync interval

Pull and other write operations require a write-capable role and permissions.write_dags. Publishing, deleting, and moving also require push_enabled: true.

CLI

bash
dagu sync status
dagu sync pull

dagu sync publish daily-report -m "Update workflow"
dagu sync publish scripts/generate.py -m "Update report script"
dagu sync publish skills/review/SKILL.md -m "Update review skill"
dagu sync publish --all -m "Update automation files"

dagu sync discard scripts/generate.py
dagu sync mv scripts/generate.py scripts/render.py -m "Rename report script"
dagu sync delete scripts/render.py -m "Remove report script"

dagu sync forget missing-item
dagu sync cleanup

See CLI Reference for every option and confirmation flag.

REST API

MethodEndpointDescription
GET/api/v1/sync/statusOverall sync status and tracked items
POST/api/v1/sync/pullPull from the remote repository
POST/api/v1/sync/publish-allPublish selected or all changed items
POST/api/v1/sync/test-connectionVerify repository access
GET/api/v1/sync/configRead current Git Sync config
PUT/api/v1/sync/configUpdate Git Sync config
GET/api/v1/sync/items/{itemId}/diffShow local and remote differences
POST/api/v1/sync/items/{itemId}/publishPublish one item
POST/api/v1/sync/items/{itemId}/discardDiscard one item
POST/api/v1/sync/items/{itemId}/forgetRemove stale sync state for one item
POST/api/v1/sync/items/{itemId}/deleteDelete one item locally and remotely
POST/api/v1/sync/items/{itemId}/moveRename one item
POST/api/v1/sync/delete-missingDelete all missing items from the remote
POST/api/v1/sync/cleanupRemove all missing entries from sync state

URL-encode item IDs in REST paths. For example, scripts/report.py becomes scripts%2Freport.py, and assets/100%.txt becomes assets%2F100%25.txt.

See REST API for request and response examples.

Permissions and production use

Treat a synchronized repository as executable code. Review changes before promotion, restrict repository write access, and keep secrets out of tracked files.

ActionRequirement
View status, diffs, and configurationAuthenticated access
Pull, publish, discard, delete, move, forget, and cleanuppermissions.write_dags and a write-capable role
Update Git Sync configurationAdmin role

Publishing, deleting, and moving also require push_enabled: true.

For staging and production environments, use read-only Git Sync and promote reviewed commits between environment branches. See Multi-Environment Deployments.

Dagu is open source under the GNU General Public License v3.0.