Docs Event Sourcing PoC Event Sourcing PoC backend

Event Sourcing PoC backend

System name used: Federated zone event store -> F.Z.E.S. -> F6 F6 is the collection of components that make up the event store.

Concept

Modeling data surrounding human more closely

The current status quo is storing state and history. This approach has some issues when historical data has to be reinterpreted.

In this proof of concept a system is built to explore and demonstrate an event based alternative. We want to be able to:

Federation

Separated based on responsibilities / ownership is done by federating by organisation and zone. We see municipalities as organisations, and divisions of the municipalities as zones.
An example of this can be seen below:

C4Context

Boundary(b1, "F6 system", "Federated zone event store system") {
	Boundary(b2, "Gemeente Zonnendaal", "organisation") {
		System(SystemA, "Burgelijke stand")
		System(SystemB, "Adres huishouding")
	}
	
	Boundary(b3, "Gemeente Woudendijk", "organisation") {
		System(SystemC, "Burgelijke stand")
		System(SystemD, "Adres huishouding")
	}
}

Events

Event flow

To design the system we have made an example of the event flow involved in the birth of a human.

Legend:

System diagram

Event data structure

We expose events as follows:

struct EventValue {
	id: Uuid,
	created_at: DateTime<Utc>,
	occurred_at: DateTime<Utc>,
	object_ids: Vec<Uuid>,
	event_type: String,
	event_version: String,
	event_kind: EventKind,
	event_source_id: Uuid,
	event_source_type: String,
	event_reference: Vec<Uuid>,
	event_data: Value,
}

Id

Uuid used to identify the event

Object ids

Event type

Type of the event (e.g. Geboorte orVerhuizing )

Event version

Events are versioned so that changes to their format do not require data migrations

Event kind

Event source id

The uuid of the event source creating event, intended for use in audits

Event source type

The type of authority (burger, ouder, ambtenaar) that the event source creating the event has

Event reference

If the event is based on one or more previous events, this must contain the ids of those events

Event data

Event specific data, the data format is set in the write_config file, read from the write_config_path given to the write server

Material vs Formal history

! todo: improve description in associated issue

Material history: what has changed in reality Formal history: what has changed in registry

Dates available:

System architecture

System diagram

An overview of the planned system can be seen below, this has not yet been completely implemented. This design is not final, and may require changes as the PoC progresses.

System diagram

Event database

Responsible for the storage of events

Read model

Responsible for building a view of the data in the system, retrieves data from all F6s that have relevant data.

Command API

A CQRS command API, commands can be sent, these commands should be validated, and result in events being added to the event database.

Command validation will require a read model for the objects in the events object_ids, to ensure that the command is valid for the objects current state.

! note: This seems like a possible location for race conditions

Query API

A CQRS query API, exposes a way to query the event database.

Not yet implemented:

F6 directory

Read models need to query every F6 that contains data about an object_id to make this efficient a directory is required.

In the diagram the directory is split into parts:

! note: we do not yet have a definition of what contracts are exactly

! note: there is no global directory linking object_ids to F6s, this is done because knowledge of inclusion in some zones may be privacy sensitive (e.g. dienst justitiële inrichtingen)

Event subscription API

Allows automatic systems to take actions when new events are added, to provide better service to citizens (e.g. automatically generate a BSN when a human is born)

Management API

To keep management burden of the system low an API and UI should be made to manage the F6 system

FSC

To ensure a safe data connection between multiple organisations FSC will be used

Development environment

Tooling used

Setup