Write the code first. Let it tell you what the book says.

Back in December 2024 I asked here whether anyone would be interested in a book about Event Sourcing and CQRS. I had a six page table of contents and a lot of doubt about whether I wanted to commit the time.

Well, I committed the time. Two years of it.

And I did one thing differently from what I planned, which turned out to matter more than anything else: I built the implementation first and let the code tell me what the book should say.

Sounds like a small decision, right? It is not. It rewrote chapters.

Let me show you one.

Snapshotting looks simple until you build it

Everybody describes snapshotting the same way. Your event stream gets long, replaying it on every load gets slow, so you periodically save the aggregate’s state and on the next load you restore that and replay only what came after.

That description is correct. It is also useless if you are the one writing the code.

I found four things while implementing it that no diagram tells you. Three of them ended up in the chapter.

The trigger is boundary math, not a modulus

You want to snapshot every fifty events. So you check whether the version is a multiple of fifty, right? version % 50 == 0. Done.

No. And here is the part that should worry you: it fails silently.

One command appends three events. Your stream goes from version 49 to version 52. It steps right over 50 and never lands on it. The check never fires, the snapshot never happens, and nobody tells you anything, because a missing snapshot looks exactly like an aggregate that has not hit the threshold yet. The load still works. It is just slow. You find out six months later when somebody asks why one aggregate takes 400 ms to load.

What you want is to ask whether the append moved the stream into a new bucket:

postVersion / interval > preVersion / interval

Integer division, evaluated across the append instead of at a single point. A multi-event append that jumps a boundary captures once, at the post-append version.

Simple fix. But I did not see it until I wrote the test that appends three events at once.

Do not upcast your snapshots

Events get upcast. Event shapes change, you write an upcaster to lift the old shape to the new one, and you maintain that chain forever, because events are immutable and you cannot go back and rewrite history.

So when the snapshot shape changes, you upcast the snapshot too. Right?

Wrong, and I am glad I found this in code rather than in print. A snapshot is a cache. The aggregate can always rebuild it from events. So when the shape changes, let the old snapshot read as a miss, rebuild from history, and capture a fresh one at the current shape the next time you cross a boundary.

Upcasting snapshots buys you nothing that a discard does not already give you, and it costs you a second chain of upcasters to keep correct next to your event upcasters. Two lineages that have to agree with each other, forever, where zero would do.

Put the schema version in the WHERE clause, by the way. Do not read the row and compare in memory. A snapshot at a shape you cannot read should never cross the wire in the first place.

Guard the restore or corrupt your writes

This one bit harder.

RestoreFrom(snapshot, version) seats state onto your aggregate and sets its version. That version is the concurrency token your next append checks against.

Now let somebody call that on an aggregate that has already applied events, or is holding uncommitted work. You just overwrote the token. The next append goes out with an expected version describing a state the aggregate is not in, and a stale write lands as if it were current.

In an event-sourced system! Where the whole promise is that the log tells you the truth!

So the restore throws unless the aggregate is pristine. Loud failure right at the seam, instead of silent corruption three layers downstream that you find out about in production.

Your speedup test cannot be a stopwatch

Obvious test: load with a snapshot, load without, assert the first is faster.

I have the scar on this one. The same code passed five out of five on a 32-core machine and starved on a shared 4-core CI runner. The scheduler widened the window and the timing budget stopped meaning anything at all.

Snapshotting does not save you milliseconds. It saves you replays. So test replays. Seed a stream past two boundaries, load through the snapshotting repository over a store that records where the read started, and assert two things: the read began at the snapshot’s version, and it replayed strictly fewer events than the full stream.

Machine independent, and it measures what the pattern is actually for.

This happened across the whole book

The snapshotting chapter is not the chapter I outlined. Neither is the one on correlation tracing. Neither is the one on event versioning, and that one moved the most.

Every single time the implementation and the manuscript disagreed, the implementation won and the chapter got rewritten. Not once did I look at working code and decide my prose had been right all along.

And I think this is the thing nobody tells you about writing on architecture. Any pattern you can draw on a whiteboard has a layer underneath it where the real decisions live. Interval math. Concurrency tokens. What your test can honestly assert on a CI runner you do not control. You cannot write about that layer if you have not been down there, and readers can tell.

So here is my advice to anybody thinking about writing a technical book: build it first. All of it. Ship the code, run it, break it, and let it tell you what your chapters should say.

The manuscript is at 449 pages across 18 chapters now, with a production-grade reference implementation in .NET running on PostgreSQL, SQL Server, KurrentDB and DynamoDB behind one contract.

What would you want to see from a book like this? I am still listening.

Proposed book about Event Sourcing & CQRS

For years, I wish I had a source that I can direct people to learn more about Event Sourcing & CQRS. Greg Young never created a book, sadly. The difficulty in Event Sourcing & CQRS is not so much the technical side but, as usual, the people you need to convince and work with. Typically, Event Sourcing allows for capabilities in your software that are extremely hard or almost impossible to do with old style software development. It is fairly easy to convince executives, product, sales, etc. that these new capabilities and insights for your customers can easily beat competition. So, why is Event Sourcing not used in like 85% of all software projects? I believe that there are different reasons but at the core, where can people go to? The Web? The Web shows all sorts of different views and many of them show inexperience and brushing Event Sourcing off as “too difficult” or “increased complexity”. WTF? Software by it’s definition is hard to do properly if you stay aligned with the business and the vision your organization has. I argue that Event Sourcing makes this actually easier, not harder!!! The CRUD virus shows its ugly head again!

So, after years of not having a great source for people to direct to, I’ve been seriously thinking of writing a book about Event Sourcing and CQRS. I have not pulled the trigger because I know this a huge time commitment. However, I did create a six page table of contents and surprised myself of how much information can and should be shared on this topic. This book would be very comprehensive and covers Event Sourcing from every angle including the business and customers side and not just the technical side.

My question would be, are you interested in this type of book?

My new course on Udemy: Build an Event Store in C# .NET for CQRS and Event Sourcing

I’m excited to announce my new course “Build an Event Store in C# .NET for CQRS and Event Sourcing“.

Real-time communication and feedback to your customers are more important then ever. Customers have become used to immediate feedback on the actions they take in your software. It is very hard in create, read, update, and delete (CRUD) based applications to provide this sort of responsiveness that your customers are demanding. Event-based solutions that are based on CQRS, Event Sourcing, and Domain-Driven Design (DDD) can offer deep insights in real-time to your customers and to your business. More importantly, you won’t loose data in an event-sourced solution when compared to CRUD-based solutions because your solution will be able to provide the context on why changes happened and record changes into an immutable log, the event store.

In this course, you will learn about the importance of using domain events as your source of truth instead of pieces of data that are incomplete in CRUD-based applications. You will be able to tell stories on what happened when and why. You will be able to answer future questions by your customers and business even though you may not have all the requirements at hand when you design and build your event-sourced solution.

At the very heart of your solution will be the event store. The event store is the source of truth in your entire solution. We will be building an event store in C#. NET and utilizing AWS DynamoDB as the persistence mechanism. However, the provided C# source code can easily be converted to Java or other languages. For the actual persistence, you could also use MySQL, PostgreSQL, MongoDB, and others. The C# code is abstracted so that can you can re-use it for specific persistence implementations. The concepts and code can work for on-premise, cloud only, or hybrid models. For an example read model implementation, we will be creating a read model using MySQL in AWS.

Once you understand the power of event sourcing, you won’t go back.

What are Projections in an Event Sourced Microservice Architecture?

In this video I will show you the different types of projections in an event-based microservice architecture and how you can use them.

Projections can be created in code in your microservice or in read-models when domain events are received and processed, for example, you process a published domain event and then store the result of the analysis in Oracle, MySQL, SQL Server, etc.

Serverless Microservices Online Courses

Update 04/03/2019: I have completed the FREE course: “Why you need serverless microservices, yesterday!“. Enroll for FREE!
At the moment, I’m working on four online courses in the following order:

 

Why You Need Serverless Microservices, Yesterday, FREE Course

WhyYouNeedServerlessMicroservices_960x520

In this course I will walk you through the many benefits of creating serverless microservices instead of the traditional node / instance approach including the use of containers. There are more than enough things to worry about when you want to create a new cloud system or transform a legacy system to operate in the cloud.

From a business point of view, there are huge benefits in going serverless rather than instance based (including containers). A very large jump in business agility can be achieved through focusing on the problems and opportunities rather than the technical jungle of traditional computing solutions.

From a technical point of view, it is almost nirvana where you can eliminate many points of failures in the architecture.

 

“How To Build An Event Store”, Paid Course

eventstore_960_520_darker

Your event store is the heart of an event sourced system. The event store is the source of truth for all business events. It needs to be able capture and replay all domain events in your system reliably and with great performance.

In this course, I will walk you through building an event store that you can re-use in your own projects. In addition, I will walk you through building a read model that allows you to query domain events from the read model.

I will also go through why building your own event store has many more advantages over using a third-party event store.

We will be building the event store in AWS DynamoDB but you can apply the design to a traditional RDMS SQL storage just as well. I will go over the pros and cons in doing so.

 

“Architecting And Designing Event Based Microservices”, Paid Course

ArchitectAndDesign

Learn the details on how to design and architect event based microservices using Domain Driven Design (DDD), Event Storming, CQRS, and EventSourcing techniques. I will show you how best combine many principles, patterns, and techniques to create an architecture with as few points of failures as possible and still deliver a great solution.

What you will learn can be applied to cloud based systems but also to traditional, on-premise systems. The benefits are great in either environments.

Whether you are designing a new system in a greenfield environment or transforming a legacy system, I will show tips & tricks that you can use depending which type of project you are in.

 

“Implementing A Serverless Microservice in AWS”, Paid Course

Code_960x520

Learn the details on how to implement a serverless microservice in AWS using Domain Driven Design (DDD), CQRS, and EventSourcing techniques.

We will build a fully functioning billing system in AWS using Visual Studio and C# .NET Core. Even if you are not familiar with C# and .NET Core, you will learn a lot of practical tips on using the different AWS services including building fully automated CI/CD pipelines.

 

I’m still working on these courses but I’m planning on publishing “Why You Need Serverless Microservices, Yesterday” and “How To Build An Event Store” first.

My New YouTube Channel

I just created my new YouTube channel “Creating Great Software”. Have a look.

I will be publishing videos about creating great software including serverless computing in AWS. In addition, I will be publishing my STRONG opinions about the state of the software industry from time to time.

I’m super excited about this new channel and I’m looking forward in seeing your feedback. See you there!

Serverless System Course

In the past 27 professional years, I have been fortunate enough to gain a lot of experience and wisdom in 10 different software industries. These experiences are not just successful accomplishments but also plenty of failures and mistakes made along the way. It is ok to make mistakes as long as you learn from them, only then you will gain true wisdom in anything you do.

Over the years, I have always thought, “If I could gather a small team, I could share my knowledge and we can create great things“. Due to many circumstances, I never had the opportunity to mentor a small team and being in full control on what we would create. By full control I mean free from politics and other constraints. The sort of freedom you need to create and explore ideas without worrying about budgets, timelines, etc. When you can foster such an environment, great things get invented. Just look at the past successes in Silicon Valley.

How can I share my experiences and wisdom most efficiently and help you achieve your goals much more rapidly rather than doing it alone? I have decided it is time to create an online school where I can share all my knowledge and wisdom from the software industry. Well, how about an online course that is very deep in all aspects. An online course that would teach you the birds eye view of concepts and then drills down into the very deep aspects of it. A course that could show you actually how to put the conceptual components together in a very cohesive way. All the different pieces put together in an architecture that makes real sense and is practical and maintainable. We’ll leave the fluff out and concentrate on the entire system starting from a users perspective and walk backwards into the technical implementation details. In short, a

Serverless System Master Class

This master class would be extensive and we will go into a lot of details until we have built a working system that you could use in your future projects. As a minimum, you would learn a lot of concepts and software architecture. Since technology is always changing, I believe that the concepts will be more valuable then the implementation details in the course. In addition, how you bring these concepts together and why are extremely important because in your day-to-day work, you will have constraints that you need to work with and make compromises. There is no perfect architecture and even in this master class, you will touch on many different options one could take and why.

Please let me know if you are interested in the Serveless System Master Class. I would love to hear your feedback as I work on the course. I plan to post regular updates on my blog here.

Leave a comment below or contact me directly at thomasjaeger at gmail dot com and let me know your level of interest, please!

Software Quality

As you may know, I’m working on an open source project called Visual MASM which is an IDE to create assembly applications for Windows just as easy as the Delphi or Visual Studio IDEs provide. Well, at least that is my goal.

I was looking back on why I want to create a commercial application in the Real Estate industry using my very own assembler IDE and was reading what I wrote about using assembly in the first place: Why Assembler?

There are so many languages and tools available today in order to program a solution. I have used almost all of them in the past 32 years of programming. Why on earth use assembly for Windows? For a clear business reason, Windows “owns” the world wide market and is used in over 90% of computers as of today. Never mind Linux, MacOS, etc. I mean, Windows “owns” it. Period. It makes total sense to develop for Windows when you create a commercial solution.

The fact that Windows owns the market; however, is still secondary to my motivation to create a commercial application for Windows using the assembly language. The main reason is really: Software Quality.

Quality in general, in my opinion, means paying attention to every detail. Because one pays attention to every detail, you are forced to make many decisions. As you abstract into higher languages, these decisions have been made for you. Because of this, you are no longer able to make detailed decisions. In order to pay attention to the detail of software quality, you have passion to follow through with it. Being able to make these detailed decisions also offers freedom and control of the creation process. That’s the ultimate power of high, software quality. You must have control in order to make software quality decisions.

So, paying attention to detail with passion is equivalent to high software quality. It is that simple.

 

What is a great software design?

I know everyone has their own opinion about what a great software design really is. For me, it’s rather very simple and be summarized in one sentence:

“A great software design can easily be changed in the future.”

In all of my designs, this is my primary goal in any architecture. Be very, very, careful of any library, frameworks, etc. you are thinking about adding. Do not add accidental complexity to your system because it saved you a few lines of code. Do not do it. Keep it simple before you start adding dependencies in your code. You will own this dependency for a long time.

If you are starting a greenfield project, start simple and stay simple until you have no choice and add additional complexity. Resist the temptation of adding this “cool” library or framework.

So, remember, “A great software design can easily be changed in the future.” Stick to your software architecture and stay disciplined and avoid feature-creep in the design. Don’t cut corners and let the team members know that it is important to keep the design simple.

There is enough complexity to deal with because you are creating software. Concentrate on the Ubiquitous Language. Make sure you see that language in your code.

Well, enough rambling, just wanted to write down some thoughts and to remind myself as well.