Preskočiť na obsah

Webhooky

Sprievodca konfiguráciou GitLab webhookov pre real-time notifikácie.

Prehľad

sequenceDiagram
    participant Dev as Developer
    participant GL as GitLab
    participant GP as GitPulse Webhook
    participant Q as Queue
    participant W as Worker

    Dev->>GL: git push
    GL->>GL: Process push
    GL->>GP: POST /webhooks/gitlab
    Note over GP: Verify signature
    GP->>Q: Enqueue event
    GP-->>GL: 200 OK (fast)
    W->>Q: Dequeue
    W->>W: Process & store

Konfigurácia v GitLab

Na úrovni projektu

  1. Prejdite do projektu v GitLab
  2. Settings -> Webhooks
  3. Vyplňte konfiguráciu:
Pole Hodnota
URL https://gitpulse.kpi.fei.tuke.sk/api/v1/webhooks/gitlab
Secret token Vygenerovaný secret (pozri nižšie)
Trigger Push, MR, Issues, Pipelines
SSL verification Yes Enabled

Na úrovni skupiny

Pre hromadnú konfiguráciu všetkých projektov:

  1. Prejdite do skupiny
  2. Settings -> Webhooks
  3. Rovnaká konfigurácia ako vyššie

Group webhooks

Group webhooky automaticky platia pre všetky projekty v skupine.

Generovanie webhook secret

Bash
1
2
3
4
5
6
# Vygenerovanie bezpečného secret
WEBHOOK_SECRET=$(openssl rand -hex 32)
echo "GITLAB_WEBHOOK_SECRET=${WEBHOOK_SECRET}"

# Pridajte do .env
echo "GITLAB_WEBHOOK_SECRET=${WEBHOOK_SECRET}" >> .env

Trigger events

Push Events

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
{
  "object_kind": "push",
  "event_name": "push",
  "before": "abc123...",
  "after": "def456...",
  "ref": "refs/heads/main",
  "checkout_sha": "def456...",
  "user_name": "John Doe",
  "user_email": "john@example.com",
  "project": {
    "id": 123,
    "name": "my-project",
    "path_with_namespace": "group/my-project"
  },
  "commits": [
    {
      "id": "def456...",
      "message": "feat: add feature X",
      "timestamp": "2024-01-15T10:30:00Z",
      "author": {
        "name": "John Doe",
        "email": "john@example.com"
      },
      "added": ["new_file.py"],
      "modified": ["existing.py"],
      "removed": []
    }
  ],
  "total_commits_count": 1
}

Spracované metriky: - Počet commitov - Autor (mapovanie na člena tímu) - Pridané/modifikované/odstránené súbory - Časové rozloženie

Merge Request Events

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "object_kind": "merge_request",
  "event_type": "merge_request",
  "user": {
    "id": 1,
    "username": "johndoe"
  },
  "object_attributes": {
    "iid": 42,
    "title": "Feature: User authentication",
    "description": "Implements OAuth2 login",
    "state": "merged",
    "source_branch": "feature/auth",
    "target_branch": "main",
    "author_id": 1,
    "assignee_id": 2,
    "created_at": "2024-01-10T10:00:00Z",
    "updated_at": "2024-01-15T14:30:00Z",
    "action": "merge"
  },
  "changes": {
    "state": {
      "previous": "opened",
      "current": "merged"
    }
  }
}

Spracované metriky: - MR creation/merge rate - Time to merge - Review participation - Branch activity

Issue Events

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
{
  "object_kind": "issue",
  "event_type": "issue",
  "user": {
    "id": 1,
    "username": "johndoe"
  },
  "object_attributes": {
    "iid": 15,
    "title": "Bug: Login fails on mobile",
    "state": "closed",
    "action": "close",
    "labels": [
      {"title": "bug"},
      {"title": "priority:high"}
    ]
  }
}

Pipeline Events

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{
  "object_kind": "pipeline",
  "object_attributes": {
    "id": 789,
    "status": "success",
    "duration": 245,
    "created_at": "2024-01-15T10:00:00Z",
    "finished_at": "2024-01-15T10:04:05Z"
  },
  "builds": [
    {
      "id": 1001,
      "stage": "build",
      "name": "compile",
      "status": "success",
      "duration": 60
    },
    {
      "id": 1002,
      "stage": "test",
      "name": "unit-tests",
      "status": "success",
      "duration": 120
    }
  ]
}

Verifikácia webhook signature

GitPulse verifikuje každý webhook pomocou secret token:

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# src/app/api/webhooks.py
import hmac
import hashlib

def verify_gitlab_signature(payload: bytes, signature: str, secret: str) -> bool:
    """Verify GitLab webhook signature."""
    expected = hmac.new(
        secret.encode(),
        payload,
        hashlib.sha256
    ).hexdigest()

    return hmac.compare_digest(f"sha256={expected}", signature)

GitLab posiela signature v hlavičke:

Text Only
1
X-Gitlab-Token: <secret>

Webhook handler

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# src/app/api/webhooks.py
from fastapi import APIRouter, Request, HTTPException, Header
from app.config import settings
from app.workers.event_processor import process_gitlab_event

router = APIRouter()


@router.post("/webhooks/gitlab")
async def gitlab_webhook(
    request: Request,
    x_gitlab_token: str = Header(None),
    x_gitlab_event: str = Header(None),
):
    """Handle GitLab webhook events."""

    # 1. Verify token
    if x_gitlab_token != settings.GITLAB_WEBHOOK_SECRET:
        raise HTTPException(status_code=401, detail="Invalid token")

    # 2. Parse payload
    payload = await request.json()

    # 3. Enqueue for async processing
    job = process_gitlab_event.delay(
        event_type=x_gitlab_event,
        payload=payload
    )

    # 4. Return immediately
    return {"status": "accepted", "job_id": str(job.id)}

Testovanie webhookov

Manuálny test

Bash
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Simulácia push event
curl -X POST http://localhost:8000/api/v1/webhooks/gitlab \
  -H "Content-Type: application/json" \
  -H "X-Gitlab-Token: ${GITLAB_WEBHOOK_SECRET}" \
  -H "X-Gitlab-Event: Push Hook" \
  -d '{
    "object_kind": "push",
    "ref": "refs/heads/main",
    "commits": [{
      "id": "test123",
      "message": "Test commit",
      "author": {"name": "Test", "email": "test@example.com"}
    }]
  }'

GitLab test webhook

  1. V GitLab webhooks settings
  2. Kliknite Test pri uloženom webhook
  3. Vyberte event type
  4. Skontrolujte response a GitPulse logy

Ngrok pre lokálny vývoj

Bash
1
2
3
4
5
# Spustenie ngrok tunela
ngrok http 8000

# Použite vygenerovanú URL v GitLab
# https://abc123.ngrok.io/api/v1/webhooks/gitlab

Troubleshooting

Webhook nedoručený

Bash
1
2
3
4
5
6
7
8
# 1. Skontrolujte GitLab webhook logy
# Project -> Settings -> Webhooks -> Recent deliveries

# 2. Overte dostupnosť GitPulse
curl -I https://gitpulse.kpi.fei.tuke.sk/api/v1/health

# 3. Skontrolujte firewall
sudo ufw status

401 Unauthorized

Bash
1
2
3
4
5
# Token nesedí
# 1. Overte GITLAB_WEBHOOK_SECRET v .env
# 2. Overte Secret token v GitLab webhook settings
# 3. Reštartujte GitPulse
docker compose restart api

500 Server Error

Bash
1
2
3
4
5
6
7
# Pozrite GitPulse logy
docker compose logs api | grep webhook

# Bežné príčiny:
# - Malformed JSON payload
# - Database connection error
# - Redis unavailable

Duplicitné eventy

Python
1
2
3
4
5
6
# GitPulse používa idempotency key
# Event s rovnakým ID je spracovaný len raz

# Ak vidíte duplicity, skontrolujte:
# 1. Či nie sú nakonfigurované webhooky na project aj group úrovni
# 2. Retry logic v GitLab

Monitoring webhookov

Prometheus metriky

PromQL
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Webhook request rate
rate(gitlab_webhook_requests_total[5m])

# Error rate
sum(rate(gitlab_webhook_requests_total{status="error"}[5m])) /
sum(rate(gitlab_webhook_requests_total[5m]))

# Processing latency
histogram_quantile(0.99, 
  rate(gitlab_webhook_duration_seconds_bucket[5m])
)

Alerting

YAML
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# monitoring/prometheus/alerts/webhooks.yml
groups:
  - name: webhooks
    rules:
      - alert: WebhookHighErrorRate
        expr: |
          sum(rate(gitlab_webhook_requests_total{status="error"}[5m])) /
          sum(rate(gitlab_webhook_requests_total[5m])) > 0.1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High webhook error rate"

Bezpečnosť

IP Whitelist

Nginx Configuration File
1
2
3
4
5
6
7
# Caddy/Nginx - povoliť len GitLab IP
location /api/v1/webhooks/ {
    allow 10.0.0.0/8;      # GitLab server IP range
    deny all;

    proxy_pass http://api:8000;
}

Rate limiting

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# src/app/api/webhooks.py
from slowapi import Limiter
from slowapi.util import get_remote_address

limiter = Limiter(key_func=get_remote_address)

@router.post("/webhooks/gitlab")
@limiter.limit("100/minute")
async def gitlab_webhook(...):
    ...

Ďalšie čítanie

Vytvorené na TUKE v Košiciach ako súčasť diplomovej práce.