Version control
Version control is described in the handbook as a critical aspect of the orchestration platform, and the reason is the pace at which an IMP changes.
A moving target
A working IMP environment is often very dynamic throughout a project. Five things change, often independently:
| What changes | Example |
|---|---|
| Coupling configurations | How one model's output is transformed into another's input |
| Models | A component swapped for an alternative |
| Model versions | A new release of an existing model |
| Data sources | An updated dataset or a different provider |
| Bespoke user programs | The transitional scripts between models |
Why this list is the problem, not a detail
The overview notes that transitional programs between models are often bespoke. Those scripts are the least formal part of the system and the most frequently edited.
Without version control, a result becomes unreproducible the moment someone improves the script that produced its inputs, and nobody notices until the numbers disagree.
The mechanism
Using version control, such as Git, ensures the orchestration system is up to date for each of these.
In the architecture, this is the GitHub repository the Deployment Server pulls from. What runs is what the repository says should run.
Historical lineage
Version control provides detailed history for every component in the system.
Combined with metadata, it delivers a comprehensive set of historical lineage information for each IMP processing scenario.
Lineage is assembled from three places
No single component provides it. Lineage emerges from the combination:
| Component | Contributes |
|---|---|
| Git | Which code, coupling configuration and scripts were in force |
| Docker images | Which exact model build was executed |
| PostgreSQL metadata | Which run produced which outputs, with which parameters |
Together these answer the question that matters when a published result is challenged: what exactly produced this number, and can we reproduce it?
Why this matters beyond good practice
The IMP produces evidence intended to inform infrastructure investment and policy advice to local authorities, by way of the DSP.
Evidence of that kind can be revisited years after it was generated, by people who were not involved in producing it. Lineage is what makes that revisiting possible, and it is why version control appears as a first-class component of the platform rather than as a development convention.