Architecture Decision Records¶
| # | Decision | Status | Date | Amended |
|---|---|---|---|---|
| 0001 | Python DataSource V2 instead of a JVM connector | accepted | 2026-09-28T16:13:29-03:00 | 2026-09-30T11:07:08-03:00 |
| 0002 | Offsets are hex LSNs carrying the commit time | accepted | 2026-09-28T16:13:29-03:00 | |
| 0003 | mssql-python as default driver, arrow-odbc as fallback |
accepted | 2026-09-28T16:13:29-03:00 | 2026-09-28T21:44:31-03:00, 2026-09-30T16:14:54-03:00, 2026-10-01T17:55:59-03:00, 2026-10-05T00:15:23-03:00, 2026-10-08T14:28:19-03:00, 2026-10-08T14:37:54-03:00 |
| 0004 | finalized_until in a control table, never in table properties |
accepted | 2026-09-28T16:13:29-03:00 | |
| 0005 | Data first, verdict after, monotonic | accepted | 2026-09-28T16:13:29-03:00 | |
| 0006 | A file-backed CDC fake to test the real Spark engine | accepted | 2026-09-28T16:13:29-03:00 | |
| 0007 | Infer captured columns from CDC metadata | accepted | 2026-09-28T17:03:21-03:00 | 2026-09-28T18:11:50-03:00, 2026-10-01T17:55:59-03:00, 2026-10-05T12:19:45-03:00 |
| 0008 | Detect the server time zone by name | accepted | 2026-09-28T17:05:54-03:00 | 2026-09-28T20:19:40-03:00, 2026-09-28T21:44:31-03:00, 2026-10-05T00:15:23-03:00, 2026-10-06T21:44:52-03:00, 2026-10-07T00:06:28-03:00 |
| 0009 | Read the change table directly, re-check retention after the read | accepted | 2026-09-28T17:39:20-03:00 | 2026-09-28T18:11:50-03:00 |
| 0010 | Idle lag of ~5 minutes; an optional Agent heartbeat for less | accepted | 2026-09-28T19:03:23-03:00 | |
| 0011 | numPartitions defaults to the compute's cores |
accepted | 2026-09-28T20:55:34-03:00 | 2026-09-29T23:02:40-03:00 |
| 0012 | Delta tables through the DeltaTable API, created typed and commented |
accepted | 2026-09-28T21:09:56-03:00 | 2026-09-28T21:15:37-03:00, 2026-09-28T21:26:25-03:00, 2026-10-05T12:04:55-03:00, 2026-10-06T07:45:42-03:00 |
| 0013 | Schema migrations per table kind | accepted | 2026-09-28T21:26:25-03:00 | 2026-10-05T06:01:09-03:00 |
| 0014 | Network and read metrics in the ingestion facts | accepted | 2026-09-29T10:12:04-03:00 | 2026-09-29T10:48:30-03:00, 2026-09-29T22:15:55-03:00, 2026-09-30T15:16:41-03:00, 2026-09-30T17:47:32-03:00, 2026-10-05T13:53:56-03:00, 2026-10-09T11:14:11-03:00 |
| 0015 | Split batches by the change table's rows | accepted | 2026-09-29T10:31:46-03:00 | 2026-10-05T13:53:56-03:00 |
| 0016 | Bootstrap with a snapshot stamped with an LSN recorded before the read | accepted | 2026-09-29T12:38:28-03:00 | 2026-09-30T11:02:01-03:00, 2026-09-30T15:21:04-03:00, 2026-10-01T17:55:59-03:00, 2026-10-02T20:30:12-03:00, 2026-10-03T18:10:05-03:00, 2026-10-08T10:59:48-03:00 |
| 0017 | Retention headroom in the ingestion facts | accepted | 2026-09-29T14:58:53-03:00 | 2026-09-30T15:16:41-03:00, 2026-09-30T17:47:32-03:00 |
| 0018 | Automatic re-snapshot after CDC data loss | accepted | 2026-09-29T17:46:41-03:00 | 2026-10-02T20:30:12-03:00, 2026-10-03T18:10:05-03:00, 2026-10-05T12:46:11-03:00, 2026-10-05T20:51:10-03:00, 2026-10-06T07:45:42-03:00 |
| 0019 | A silver helper applies the bronze change log to a current-state table | accepted | 2026-09-30T11:07:22-03:00 | 2026-10-02T20:30:12-03:00, 2026-10-03T14:30:38-03:00, 2026-10-07T04:49:27-03:00, 2026-10-08T13:06:12-03:00 |
| 0020 | Capture and ingestion lag in the ingestion facts | accepted | 2026-09-30T10:33:11-03:00 | 2026-09-30T15:16:41-03:00, 2026-10-05T13:53:56-03:00 |
| 0021 | Compatibility policy for 0.x: the state contract is stable | accepted | 2026-09-30T11:07:08-03:00 | 2026-10-01T15:55:00-03:00, 2026-10-05T06:01:09-03:00, 2026-10-05T12:35:09-03:00, 2026-10-05T12:49:29-03:00, 2026-10-05T20:51:10-03:00, 2026-10-07T01:34:59-03:00, 2026-10-07T01:41:18-03:00, 2026-10-07T09:14:31-03:00, 2026-10-09T11:39:56-03:00, 2026-10-10T18:02:10-03:00, 2026-10-13T18:45:19-03:00, 2026-10-14T09:06:30-03:00, 2026-10-15T09:39:58-03:00, 2026-10-15T09:49:09-03:00, 2026-10-15T09:50:28-03:00 |
| 0022 | Defer a Spark CHANGES changelog connector |
accepted | 2026-09-30T11:07:08-03:00 | |
| 0023 | Schema changes on the source, and switching to a newer capture instance | accepted | 2026-09-30T18:40:00-03:00 | 2026-09-30T20:09:44-03:00, 2026-09-30T21:25:30-03:00, 2026-10-01T15:55:00-03:00, 2026-10-01T17:25:27-03:00, 2026-10-01T18:15:53-03:00, 2026-10-07T01:41:18-03:00, 2026-10-13T18:45:19-03:00, 2026-10-15T09:50:28-03:00 |
| 0024 | A documentation site built by Zensical, hosted on GitHub Pages | accepted | 2026-10-01T14:43:57-03:00 | 2026-10-14T09:21:02-03:00, 2026-10-15T09:43:36-03:00, 2026-10-15T12:20:00-03:00 |
| 0025 | Seed a target from an existing copy of the table | accepted | 2026-10-01T16:53:59-03:00 | 2026-10-01T19:44:50-03:00, 2026-10-02T20:30:12-03:00 |
| 0026 | Continuous-mode finalization through a streaming query listener | accepted | 2026-10-01T16:36:15-03:00 | 2026-10-01T19:44:50-03:00 |
| 0027 | Many tables: one stream per capture instance, started by a fan-out helper | accepted | 2026-10-01T16:30:09-03:00 | 2026-10-13T20:07:28-03:00 |
| 0028 | Chunked snapshots read next to the running stream | accepted | 2026-10-02T20:30:12-03:00 | 2026-10-03T14:30:38-03:00, 2026-10-03T18:10:05-03:00, 2026-10-04T17:20:07-03:00, 2026-10-06T21:14:15-03:00, 2026-10-07T00:06:28-03:00, 2026-10-08T13:47:54-03:00, 2026-10-10T18:02:10-03:00, 2026-10-13T13:33:12-03:00, 2026-10-13T18:45:19-03:00, 2026-10-13T18:52:13-03:00 |
| 0029 | Driver-side retries, and an optional lock timeout | accepted | 2026-10-05T12:23:04-03:00 | |
| 0030 | typing.Protocol for the pluggable seams |
accepted | 2026-10-07T09:05:52-03:00 | 2026-10-15T09:35:58-03:00 |
| 0031 | Strict typing: mypy strict on src, pyright --verifytypes on the public API |
accepted | 2026-10-07T10:49:11-03:00 | 2026-10-07T13:39:33-03:00, 2026-10-15T09:29:08-03:00 |
| 0032 | Facts without caching where the platform refuses it | accepted | 2026-10-09T11:39:56-03:00 | 2026-10-09T19:26:50-03:00 |
| 0033 | Spark Connect, tested against a local Connect server | accepted | 2026-10-09T11:39:43-03:00 | 2026-10-09T19:26:50-03:00 |
| 0034 | Credentials outside the connection string: user, password, an Entra ID token or a token provider |
accepted | 2026-10-10T17:59:29-03:00 | 2026-10-13T13:33:12-03:00 |
New ADRs: copy the format, next number, one decision per file. Put the time the
decision is recorded in **Date:** (ISO-8601 with the UTC offset, e.g.
2026-09-28T17:03:21-03:00; the commit time works) and add an **Amended:** line with
the time and a few words for every later change, here and in the table above.