gitea-heatmap-backfill/README.md

gitea-heatmap-backfill

Retroactively populate the Gitea activity heatmap from existing Git history.

When you migrate repositories into Gitea, past commits do not appear in the contribution heatmap because no action records exist for them. This script reads the commit history directly from the bare repositories on disk and inserts the missing records into the Gitea SQLite database.

---

Tested on

Component	Version
Gitea	1.26.2
Database	SQLite 3

Other Gitea versions using SQLite may work but have not been tested. PostgreSQL and MySQL backends are not supported.

---

How it works

1. Discovers all bare repositories under the Gitea repository root
2. Runs git log --all on each one to extract commit SHAs, author emails and dates
3. Resolves each email against the email_address table, primary address and all registered alternative addresses are matched
4. Inserts one action row per commit (op_type = 5, PUSH_EVENT) into the Gitea SQLite database
5. Deduplicates via a SHA-256 content hash so the script is safe to run multiple times

---

Requirements

• Python 3.7+
• git available on the machine running the script
• Read/write access to the Gitea SQLite database file
• Gitea using SQLite as its database backend

---

Setup

1. Clone the repository

git clone https://github.com/Maethik/gitea-heatmap-backfill.git
cd gitea-heatmap-backfill

No external dependencies, only the Python standard library is used.

2. Register all your commit emails in Gitea

The script matches commits by email. Open Settings → Email Addresses in your Gitea profile and add every email address you have ever used in your commits (work addresses, old personal addresses, GitHub no-reply addresses, etc.).

You can check which emails appear in your history:

# Replace the path with your actual repository root
for repo in /path/to/gitea/repositories/owner/*.git; do
  git -C "$repo" log --all --format="%ae"
done | sort | uniq

3. Configure paths

The script is configured via two environment variables or by editing the constants at the top of the file.

Variable	Default	Description
DB_PATH	/etc/komodo/stacks/gitea/data/gitea/gitea.db	Absolute path to gitea.db on the host
GITEA_ROOT	/etc/komodo/stacks/gitea/data/git/repositories	Absolute path to the Gitea repository root on the host

To find the correct paths for your setup:

# Find the repository root declared in app.ini
docker exec gitea cat /data/gitea/conf/app.ini | grep -E "^ROOT\s*="

# Find the host path mounted into the container
docker inspect gitea | grep -A 10 '"Mounts"'

---

Usage

The script runs on the Docker host, not inside the container. Python is not available inside the minimal Gitea image.

# Using environment variables
DB_PATH=/path/to/gitea.db \
GITEA_ROOT=/path/to/repositories \
python3 gitea_backfill.py

# Or edit the constants directly at the top of the script, then run
python3 gitea_backfill.py

Fix git safe.directory errors

If git refuses to read the repositories due to ownership mismatch, run this once before executing the script:

git config --global --add safe.directory '*'

Expected output

Gitea Heatmap Backfill
============================================
[1] Creating backup → /path/to/gitea.backup-before-backfill.db
    [✓] Backup created.

[✓] 5 email(s) loaded from email_address table
[✓] 8 repo(s) loaded from repository table
[✓] 8 bare repo(s) found on disk

[2] Existing PUSH_EVENT rows: 0

[→] owner/my-project (id=3)
    847 commits found.
    [500 inserted so far]
[→] owner/another-repo (id=4)
    312 commits found.
...
────────────────────────────────────────────
  Backfill complete
────────────────────────────────────────────
  Inserted  : 1159
  Skipped   : 0       (already present or bad date)
  No user   : 4       (email not registered in Gitea)
  No repo   : 0       (repo not found in database)
────────────────────────────────────────────

Restart Gitea to refresh the heatmap:
  docker restart gitea

After the script completes

Restart Gitea so the heatmap is recomputed from the new records:

docker restart gitea

---

Troubleshooting

No user count is high

Some commit emails are not registered in Gitea. Run the email audit command from the Setup section above, then add the missing addresses in your Gitea profile settings.

No repo count is non-zero

The repository name on disk does not match the owner_name/name stored in the database. Check for casing differences:

# Names in the database
docker exec gitea sqlite3 /data/gitea/gitea.db \
  "SELECT owner_name, name FROM repository;"

# Names on disk
ls /path/to/gitea/repositories/owner/

git log fails with dubious ownership

git config --global --add safe.directory '*'

Running the script a second time

Safe. Already-inserted rows are detected via their content hash and skipped.

---

Rollback

A backup of the database is created automatically before any writes at:

/path/to/gitea.backup-before-backfill.db

To restore it:

docker stop gitea
cp /path/to/gitea.backup-before-backfill.db /path/to/gitea.db
docker start gitea

---

Limitations

• SQLite only — not compatible with Gitea instances using PostgreSQL or MySQL
• Commit emails must be registered in Gitea; unmatched emails are skipped
• Wiki repositories (*.wiki.git) are intentionally ignored