Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,25 @@ The README's screenshots and demo GIF come from one such test,
`make demo-gif` whenever a view they show changes; `make samples` refreshes
the fictional `docs/samples/uebung.fwincident` after a schema change.

## Schema migrations

A feature branch that adds a migration owns version N only until `main` takes
N. Before merging, rebase and **renumber** the branch's migrations above
`main`'s current `Migrations.CurrentVersion` — two branches shipping different
"V18"s is how a file ends up stamped with a version whose migrations it never
received.

For the same reason, never run a branch build against real Einsatzdateien: it
stamps its own `schema_version` onto them, and the released app then skips
every migration numbered below it. `SchemaGuard` repairs missing columns after
the fact, but it cannot restore data a foreign build dropped, and it
deliberately does not touch tables.

Any migration that adds a column must also declare it in
`SchemaGuard.ExpectedColumns`
(`src/LageBuch.Persistence/Sqlite/SchemaGuard.cs`);
`SchemaReconciliationTests` fails the build otherwise.

## Git commits

All commits must be:
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,15 @@ once we reach 1.0.
on the system". Install it with `apt` rather than `dpkg -i`, which cannot resolve
dependencies. Verified on Debian 12/13 and Ubuntu 22.04/24.04.
- CO-Messung: corrected the empty-state message to use German typographic quotes and added the missing comma before "um zu beginnen".
- An Einsatzdatei that a build from another development branch had written can be opened again.
Each file records a schema version, and the app runs only the migrations numbered above it —
sound as long as that number means the same thing everywhere. A parallel branch numbers its own
migrations too, so a file it touched could come back carrying a version the released app had
never applied: the app then skipped a migration the file genuinely needed, and opening it failed
with "no such column: previous_zugfuehrer_count" and no way forward. Opening a file now also
compares its actual columns against the expected schema and adds whatever is missing, regardless
of the recorded version, so an affected file repairs itself the next time it is opened. Columns
and tables the app does not know are left untouched.

### Security
- Attachment names are stripped of invisible formatting characters, not just control characters.
Expand Down
5 changes: 4 additions & 1 deletion src/LageBuch.Persistence/IncidentRepository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -402,7 +402,10 @@ public static Incident Load(string path)
// Bring an older file up to the current schema before reading it. A file last written
// by an earlier version sits at its old schema_version; without this, Load would query
// tables that don't exist yet. Migration is version-gated/idempotent and only adds empty
// tables, so re-opening a closed incident read-only afterwards still reads no new content.
// tables or empty columns, so re-opening a closed incident read-only afterwards still reads
// no new content. The version gate alone is not enough -- a build from a parallel branch can
// stamp a marker whose meaning differs from this lineage's, so Migrate also reconciles the
// file's real columns against the expected schema on every open (see SchemaGuard).
using (var migrateCn = SqliteConnectionFactory.OpenExisting(path))
{
Migrations.Migrate(migrateCn);
Expand Down
4 changes: 4 additions & 0 deletions src/LageBuch.Persistence/LageBuch.Persistence.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@
<PackageReference Include="Microsoft.Data.Sqlite" />
</ItemGroup>

<ItemGroup>
<InternalsVisibleTo Include="LageBuch.Persistence.Tests" />
</ItemGroup>

<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
Expand Down
7 changes: 7 additions & 0 deletions src/LageBuch.Persistence/Sqlite/Migrations.cs
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,13 @@ public static void Migrate(SqliteConnection cn)
ApplyV21(cn, tx);
}

// The version gate above is not proof that the steps it skipped ever ran: a build from a
// parallel branch numbers its own migrations too, so a file can carry a marker this lineage
// never put there and be missing a column no `version < N` will ever add back. Reconcile the
// file's real shape against the expected schema on every open -- after the versioned steps,
// because V3/V6/V17 rebuild scba_trupps and only the final shape is worth widening.
SchemaGuard.Reconcile(cn, tx);

SetVersion(cn, tx, CurrentVersion);
tx.Commit();
}
Expand Down
185 changes: 185 additions & 0 deletions src/LageBuch.Persistence/Sqlite/SchemaGuard.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
using Microsoft.Data.Sqlite;

namespace LageBuch.Persistence.Sqlite;

/// <summary>
/// Reconciles an incident file's real columns against the schema this build expects, on every open.
/// </summary>
/// <remarks>
/// <para>
/// <see cref="Migrations"/> gates every step behind <c>version &lt; N</c>, which is only sound if
/// every build that ever stamped version N applied exactly this lineage's steps 1..N. Parallel
/// feature branches break that: a branch that numbers its own migrations 18 and 19 stamps
/// <c>schema_version = 19</c> onto a file that never received main's V18, and main then skips V18
/// forever — the file is missing a column no future migration will ever add back. That is not
/// hypothetical; it is how <c>force_units.zugfuehrer_count</c> went missing from files carrying
/// the current version marker, and every read of them failed with "no such column".
/// </para>
/// <para>
/// So the version marker is treated as an optimisation, not as proof of what is on disk. This pass
/// runs unconditionally and adds any expected column the file actually lacks. It is deliberately
/// one-directional: it never drops a table or column it does not recognise, so a file that a
/// feature branch widened keeps that branch's data and still opens on both builds.
/// </para>
/// <para>
/// <see cref="ExpectedColumns"/> covers every column of the current schema that
/// <c>ALTER TABLE ... ADD COLUMN</c> can legally add — not a primary key, and either nullable or
/// carrying a default. That is exactly the set repairable after the fact, and it has a mechanical
/// membership rule (<c>pk = 0 AND (notnull = 0 OR dflt_value IS NOT NULL)</c>) that
/// <c>SchemaReconciliationTests</c> evaluates against a freshly migrated database, so a migration
/// that adds a column without listing it here fails CI instead of a user's file.
/// </para>
/// </remarks>
internal static class SchemaGuard
{
private static readonly string[] Tables =
[
"audit_events",
"checklist_items",
"co_buildings",
"co_dwellings",
"etb_entries",
"etb_entry_edits",
"force_unit_edits",
"force_units",
"incident_files",
"incident_meta",
"incident_tasks",
"incident_timers",
"role_assignments",
"scba_pressure_readings",
"scba_trupp_members",
"scba_trupps",
];

private static readonly ExpectedColumn[] Columns =
[
new("checklist_items", "note", "TEXT", false, null),
new("checklist_items", "is_mandatory", "INTEGER", true, "0"),
new("checklist_items", "kind", "INTEGER", true, "0"),
new("co_buildings", "floor_descriptions", "TEXT", true, "'{}'"),
new("co_buildings", "apartment_labels", "TEXT", true, "'{}'"),
new("co_buildings", "underground_floor_count", "INTEGER", true, "0"),
new("co_buildings", "apartment_counts", "TEXT", true, "'{}'"),
new("co_dwellings", "resident_name", "TEXT", false, null),
new("co_dwellings", "key_available", "INTEGER", false, null),
new("co_dwellings", "co_value", "INTEGER", false, null),
new("etb_entries", "from_party", "TEXT", false, null),
new("etb_entries", "to_party", "TEXT", false, null),
new("force_unit_edits", "previous_zugfuehrer_count", "INTEGER", true, "0"),
new("force_units", "call_sign", "TEXT", false, null),
new("force_units", "status", "TEXT", false, null),
new("force_units", "notes", "TEXT", false, null),
new("force_units", "scba_count", "INTEGER", true, "0"),
new("force_units", "officer_count", "INTEGER", true, "0"),
new("force_units", "zugfuehrer_count", "INTEGER", true, "0"),
new("incident_files", "display_name", "TEXT", false, null),
new("incident_meta", "incident_number", "TEXT", false, null),
new("incident_meta", "ils_number", "TEXT", false, null),
new("incident_meta", "keyword", "TEXT", false, null),
new("incident_meta", "street", "TEXT", false, null),
new("incident_meta", "district", "TEXT", false, null),
new("incident_meta", "status", "TEXT", false, null),
new("incident_meta", "closed_at", "TEXT", false, null),
new("incident_meta", "closed_by", "TEXT", false, null),
new("incident_tasks", "completed_at", "TEXT", false, null),
new("incident_tasks", "completed_by", "TEXT", false, null),
new("role_assignments", "call_sign", "TEXT", false, null),
new("role_assignments", "from_time", "TEXT", false, null),
new("role_assignments", "to_time", "TEXT", false, null),
new("role_assignments", "section", "TEXT", false, null),
new("role_assignments", "phone", "TEXT", false, null),
new("scba_trupps", "call_sign", "TEXT", false, null),
new("scba_trupps", "task", "TEXT", false, null),
new("scba_trupps", "start_time", "TEXT", false, null),
new("scba_trupps", "entry_pressure", "INTEGER", false, null),
new("scba_trupps", "withdraw_time", "TEXT", false, null),
new("scba_trupps", "exit_time", "TEXT", false, null),
];

/// <summary>Every column this build can repair, in the order it is declared above.</summary>
public static IReadOnlyList<ExpectedColumn> ExpectedColumns => Columns;

/// <summary>Every table of the current schema, so a newly added one cannot go unnoticed.</summary>
public static IReadOnlyList<string> ExpectedTables => Tables;

/// <summary>
/// Adds every expected column the file is missing. A healthy file is left byte-identical.
/// </summary>
public static void Reconcile(SqliteConnection cn, SqliteTransaction tx)
{
ArgumentNullException.ThrowIfNull(cn);
var present = ReadColumnsByTable(cn, tx);
foreach (var column in Columns)
{
// A table that is absent entirely is out of scope -- AddColumnIfMissing would no-op on
// it anyway, and reporting "no such table" is more honest than half-building it here.
if (present.TryGetValue(column.Table, out var names) && !names.Contains(column.Name))
{
SchemaHelpers.AddColumnIfMissing(cn, tx, column.Table, column.Name, column.Ddl);
}
}
}

/// <summary>
/// Reads the file's real shape in one sweep -- one sqlite_master query plus one
/// pragma_table_info per table -- so the common case (nothing missing) costs no ALTER probing.
/// </summary>
private static Dictionary<string, HashSet<string>> ReadColumnsByTable(
SqliteConnection cn, SqliteTransaction tx)
{
var tables = new List<string>();
using (var list = cn.CreateCommand())
{
list.Transaction = tx;
list.CommandText = "SELECT name FROM sqlite_master WHERE type='table';";
using var reader = list.ExecuteReader();
while (reader.Read())
{
tables.Add(reader.GetString(0));
}
}

var result = new Dictionary<string, HashSet<string>>(StringComparer.Ordinal);
foreach (var table in tables)
{
var names = new HashSet<string>(StringComparer.Ordinal);
using var info = cn.CreateCommand();
info.Transaction = tx;
info.CommandText = "SELECT name FROM pragma_table_info($t);";
info.Parameters.AddWithValue("$t", table);
using var reader = info.ExecuteReader();
while (reader.Read())
{
names.Add(reader.GetString(0));
}

result[table] = names;
}

return result;
}
}

/// <summary>
/// One column of the current schema that <c>ALTER TABLE ... ADD COLUMN</c> can restore, kept
/// structured rather than as a DDL string so the drift test can compare type, nullability and
/// default field by field against <c>pragma_table_info</c>.
/// </summary>
/// <param name="Table">The table the column belongs to.</param>
/// <param name="Name">The column name.</param>
/// <param name="Type">The declared SQLite type, e.g. <c>TEXT</c>.</param>
/// <param name="NotNull">Whether the column is declared NOT NULL.</param>
/// <param name="Default">
/// The default as SQLite reports it in <c>pragma_table_info.dflt_value</c> — a raw literal, so a
/// string default carries its own quotes (<c>'{}'</c>). Null when the column has no default.
/// </param>
internal sealed record ExpectedColumn(
string Table, string Name, string Type, bool NotNull, string? Default)
{
/// <summary>The column definition as ALTER TABLE wants it.</summary>
public string Ddl =>
Type
+ (NotNull ? " NOT NULL" : string.Empty)
+ (Default is null ? string.Empty : $" DEFAULT {Default}");
}
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,13 @@ public void A_file_from_a_newer_version_is_refused_and_its_marker_left_alone()
// file claim a schema it does not have, and the newer build would then skip the very
// migration that produced it.
Assert.Equal(newer, Migrations.GetVersion(cn));

// Nor may the schema be touched. The reconciliation pass runs after this guard, so a
// file from ahead of us is left exactly as its own build wrote it -- rebuilding tables
// this build happens to know about would be the same over-reach as stamping the marker.
using var check = cn.CreateCommand();
check.CommandText = "SELECT count(*) FROM sqlite_master WHERE type='table' AND name<>'schema_version';";
Assert.Equal(0L, (long)check.ExecuteScalar()!);
}
}

Expand Down
Loading
Loading