Pilot deployment¶

This page will help you prepare and pilot a GitPulse in your course.

Why the pilot?¶

The pilot deployment serves to:

1. Validation of metrics - verification of relevance for your context
2. Gathering feedback - from both teachers and students
3. Calibration thresholds - setting appropriate limits for warning
4. Identification of problems - both technical and methodological

Phases of the pilot¶

98 days (September 1 - December 11, 2025)

Phase 1: Preparation¶

Technical setting¶

1. Create a test environment
2. Verify that GitPulse is available
3. Configure the course and projects

4. Test webhooks

5. Set Baseline

6. Import historical data (if any)
7. Adjust the metric weights according to your course
8. Define thresholds for warnings

Informed consent¶

Inform students about data collection and processing.

Prepare information containing:

• What data do we collect (GitLab activity)
• How we use the data (evaluation of cooperation)
• How long will we keep the data (until the end of the semester + 1 year)
• Student rights (access, erasure)

Training¶

For teachers/trainers, prepare:

• Dashboard demo
• Metrics explained
• Examples of interpretation
• Support contact

Phase 2: Silent mode¶

In the first 2 weeks:

1. Monitor without interventions
2. Collect data
3. Observe the patterns

4. Do not alert students

5. Calibrate Thresholds

6. Identify outliers
7. Adjust thresholds for warnings

8. Note the false positives

9. Document Issues

10. Technical problems
11. Ambiguity in metrics
12. Suggestions for improvement

Phase 3: Active use¶

Integration into teaching¶

flowchart LR
    A["Weekly check"] --> B{Problems?}
    B -->|Yes| C["Intervention"]
    B -->|No| D["Continue"]
    C --> E["Feedback loop"]
    E --> A

Types of interventions¶

Data collection¶

GitPulse automatically collects telemetry:

• Time spent in the dashboard
• Number of reports generated
• Interventions performed
• Feedback from teachers

Feedback from students¶

Halfway through the semester:

1. Give students an anonymized view of their metrics
2. Explain what the metrics mean
3. Collect feedback (anonymous form)

Phase 4: Evaluation¶

Pilot success metrics¶

Questionnaire for teachers¶

1
2
3
4
5
6

1. Ako často ste používali GitPulse dashboard? (denne/týždenne/zriedka)
2. Aké funkcie boli najužitočnejšie? (multiselect)
3. Identifikoval GitPulse problémy, ktoré by ste inak prehliadli? (áno/nie/neviem)
4. Koľko času ste ušetrili vďaka GitPulse? (hodiny/týždeň)
5. Čo by ste zlepšili? (voľný text)
6. Odporučili by ste GitPulse kolegom? (1-5)

Questionnaire for students¶

1
2
3
4
5

1. Vedeli ste o používaní GitPulse vo vašom kurze? (áno/nie)
2. Ovplyvnilo to váš spôsob práce? (áno pozitívne/áno negatívne/nie)
3. Považujete hodnotenie spolupráce cez metriky za férové? (1-5)
4. Máte obavy o súkromie? (1-5)
5. Návrhy na zlepšenie? (voľný text)

Final report¶

Prepare a report containing:

1. Executive summary
2. Key findings

3. Recommendations

4. Quantitative results

5. Number of monitored teams/students
6. Saving time

7. Detection accuracy

8. Qualitative Findings

9. Feedback from teachers
10. Feedback from students

11. Identified problems

12. Recommendations

13. Technical improvements
14. Methodological changes
15. Next deployment plan

Checklist of the pilot¶

In front of the pilot¶

• GitPulse is technically functional
• Course and projects are configured
• Webhooks are working
• Teachers are trained
• Students are informed about data collection
• Baseline metrics are set

During the pilot¶

• Weekly dashboard check
• Documentation of issues
• Threshold calibration
• Regular feedback from teachers
• Mid-semester feedback from students

After the pilot¶

• Final questionnaires distributed
• Data exported
• Analysis done
• Report prepared
• Lessons learned documented

Examples from practice¶

Example: DevOps 2025 pilot course¶

Context: - 25 teams, 100 students - 12-week semester - Team projects in GitLab

Results: - Time saving: 35% - Problem detection: 85% - Teacher satisfaction: 4.⅖ - Student satisfaction: 3.8/5

Lessons learned: 1. Thresholds for gaming were too strict 2. Students appreciated the transparency of the metrics 3. The real-time dashboard was most useful

Support during the pilot¶

For support during the pilot, contact:

• mykyta.olym@student.tuke.sk
• In-app support
• Consultation hours (by agreement)

Further reading¶

• Dashboard and metrics - Detailed description of metrics
• Result Interpretation - How to read the results
• FAQ - Frequently Asked Questions