4. GitPulse platform and prototype design

The analysis identified the need for a layer that connects data from multiple projects with course-specific checks while retaining a link to the original GitLab evidence. Webhooks alone are not sufficient. GitPulse therefore combines fast event intake, REST API enrichment, and metric calculation over locally stored data.

4.1 Requirements and usage scenarios

The primary user is the instructor. At the start of a semester, they need to import a course without creating every group and project manually. During the semester, they need to find groups with missing process steps, drill down from a summary to a student, and open the original issue, merge request, review comment, or CI/CD process when a result is unclear. Students can see only data related to their own work and course.

The functional prototype supports:

1. GitLab sign-in and administrator, instructor, mentor, and student roles;
2. course creation or GitLab group/project import;
3. a course-group-project-member hierarchy without duplicated inherited members;
4. initial and manual synchronisation of issues, merge requests, notes, approvals, commits, and CI/CD processes;
5. continuous GitLab event processing without double counting;
6. R01-R13 evaluation with configurable thresholds, weights, and active checks;
7. course and group summaries with project and student details;
8. explicit synchronisation and check runs with visible progress;
9. optional exercises for merge conflicts, Dockerfiles, and CI failures.

Check results include an explanation and are not automatic grades. The core analytics path is read-only. Repository writes are restricted to separate exercise modules and require an explicit action by an authorised user.

4.2 Architecture and event flow

GitPulse is a standalone web service outside student repositories. Caddy accepts public requests and forwards them to FastAPI. PostgreSQL is the persistent store. Redis provides caching, live notifications, and RQ queues. A worker enriches objects through the GitLab REST API and recalculates affected metrics.

flowchart LR
  GL[GitLab TUKE] -->|webhook| Caddy[Caddy / TLS]
  Caddy --> API[FastAPI]
  API --> DB[(PostgreSQL)]
  API --> Redis[Redis + RQ]
  Redis --> Worker[Worker]
  Worker --> DB
  Worker -. REST API .-> GL
  API -->|SSE| UI[Web interface]

Webhook intake verifies the project secret, stores the original payload, and uses the event identifier as a unique key. A repeated delivery cannot create a second record. If the queue is temporarily unavailable, the event remains in the database and can be recovered later.

flowchart LR
  Recv[1. Receive and verify]
  Save[2. Persist event]
  Queue[3. Enqueue]
  Enrich[4. Enrich via REST API]
  Eval[5. Evaluate checks]
  Store[6. Store snapshot and notify]
  Recv --> Save --> Queue --> Enrich --> Eval --> Store

The prototype does not periodically sweep GitLab. Updates are triggered by a webhook or an explicit manual synchronisation, avoiding invisible background work and unnecessary GitLab requests.

4.3 Course and data model

A course contains check configuration and user memberships. A group belongs to one course and may contain multiple projects. A project represents a concrete GitLab repository. Import is idempotent: repeated imports update existing records, while missing projects become inactive.

Issues, merge requests, notes, approvals, commits, and CI/CD processes retain both local and GitLab identifiers. Original webhook payloads are stored separately as audit input.

4.4 Process checks

Each check returns an identifier, pass state, explanation, weight, and partial score. Only enabled checks contribute to the result. The default traffic-light thresholds are 80 percent for green and 50 percent for yellow, and instructors can change them. The state prioritises attention; it is not converted directly into a grade.

R01-R13 cover the student's assigned issue, feature branch and merge request, tests, issue linking and description quality, independent reviewers, discussion and author responses, approval, and a successful CI/CD process. A course may define the academic week from which a check is expected. Weights do not change automatically by week, and GitPulse does not duplicate the official assignment site.

4.5 Optional exercise tools

Three optional modules are disabled by default and use a separate bot token:

• the conflict simulator creates a controlled merge conflict;
• Dockerfile analysis runs deterministic checks on a selected file;
• the CI failure generator creates a scenario in an exercise branch and can revert it.

Language-model text analysis exists in the codebase but is disabled in production, is not used by R01-R13, and does not produce a grade.

4.6 Security boundaries

Sign-in uses GitLab OAuth. Authorisation is enforced on server routes and at the course boundary. Every project has an independent webhook secret, stored GitLab tokens are encrypted, and state-changing requests have anti-forgery protection. Read-only analytics tokens are separated from exercise-module tokens that can write to a repository.

4.7 Main scenario and prototype status

The prototype is available at gitpulse.kpi.fei.tuke.sk. The instructor's main scenario starts with an alert for a group that requires inspection. The alert prioritises work; it is not an automatic grade.

Step one: an alert for a group with a red compliance state.

After opening the alert, the instructor compares group members by score, state, completed checks, and basic activity. Values come from the live prototype; names and usernames are anonymised in the screenshot.

Step two: comparing group members before detailed inspection.

In the third step, the instructor opens the selected student's detail. The summary shows the latest score, traffic-light state, and completed checks. The same page provides commits, issues, merge requests, CI/CD processes, and discussions from which the result was derived.

Step three: result summary before inspecting source data.

A focused technical run covered 350 automated tests for the check engine, event flow, webhooks, group import, and the three optional tools; all passed. Instructor validation and impact measurement belong to a later thesis stage.