3. Verifying data acquisition options
The design depends on whether course requirements can be mapped to data GitLab actually provides. A webhook announces a change but may not contain all data needed for evaluation. The REST API provides detailed current state, while periodically scanning every project would be slow and unnecessarily expensive.
3.1 Verification method
The verification had three steps:
- R01-R13 were decomposed into the smallest data items needed for a decision.
- Those items were checked against the official GitLab webhook and REST API documentation.
- The mapping was compared with the implementation and exercised by automated tests.
Push, Issue, Merge Request, Note, Pipeline, Tag Push, and membership changes were considered. For each item, the analysis distinguished between a complete event payload, an event that only triggers enrichment, and a full synchronisation.
A focused set of 350 tests ran on 19 June 2026 with Python 3.12, and all passed. This demonstrates the implementation contract for prepared inputs. It does not demonstrate pedagogical impact or correctness for every real project. Synthetic test fixtures are not presented as observations from a real course.
3.2 A webhook announces change
GitLab may deliver one event more than once. The prototype stores X-Gitlab-Event-UUID; if the header is absent, it computes a deterministic fingerprint from the payload. A unique database key prevents double counting.
A webhook is not a complete copy of a GitLab object. A Merge Request event announces a request change, while file changes, discussions, and approvals are enriched through REST endpoints. A Note event carries a new comment, not the whole discussion. A Pipeline event reports process state, while R12 also needs the job list. Push events may contain only a limited commit list.
The resulting model is "event as signal, REST API as detail source". Webhooks start short intake work, enrichment continues in a queue, and manual synchronisation is reserved for initial import, repair, or an explicit refresh. The prototype does not periodically sweep the whole GitLab instance.
3.3 Mapping checks to data sources
| ID | Trigger | REST enrichment | Interpretation limit |
|---|---|---|---|
| R01 | Issue | issue and assignee | Assignment does not prove completed work. |
| R02 | Merge Request | source branch | Naming conventions depend on the course. |
| R03 | Merge Request | changed files | A file name does not prove test quality. |
| R04 | Merge Request | description and issue link | A link does not prove semantic agreement. |
| R05 | Merge Request | full description | Length and template are formal minima only. |
| R06 | Note | notes and discussions | Comment length is only a proxy. |
| R07 | Note | notes across requests | Count does not express review difficulty. |
| R08 | Note | threads and author replies | Standalone notes can be paired only approximately. |
| R09 | Merge Request | approvals | Approval does not prove the review process. |
| R10 | Merge Request | merged_by | The allowed actor depends on course workflow. |
| R11 | Issue + Merge Request | linked-object state | The result is less certain without an explicit link. |
| R12 | Pipeline | process and jobs | Success proves only what the process ran. |
| R13 | Push + synchronisation | commit history | Message pattern does not prove the editor used. |
R03-R05 check formal merge-request evidence, not implementation correctness. R06-R09 observe review participation but cannot judge technical truth. R13 uses an uncertain proxy and therefore has zero weight by default.
3.4 Design findings
All thirteen checks have a technical input, but their evidential strength differs. Every result must retain its source and explanation so an instructor can compare it with GitLab. Webhooks provide a fast signal, REST provides fuller state, and manual synchronisation provides a recovery path.
A technical check may flag a missing artefact, but it must not decide ethical behaviour or grades. The earlier "gaming detection" concept was removed because it mixed incomplete signals with pedagogical conclusions. The prototype uses transparent R01-R13 checks and manual review of disputed cases.