AWS

Python is 72x slower and uses 76x more energy than C

Thomas Jaeger Architecture, AWS, C Programming , , , ,

According to a recent study named “Ranking Programming Languages by Energy Efficiency” conducted by several institutions in Portugal, Python is incredibly energy wasteful. In addition, Python is also extremely slow when compared to the C programming language. See table below:

This comes to no surprise to me. Interpreted languages like Python, Perl, Lua, JavaScript, TypeScript etc. are by their very design, slow and energy wasteful because priority has been put on “ease of use” for programmers. This comes at a very heavy price: slow performance and energy wastefulness.

Compiled languages like C will always be outperforming and be much more energy efficient because C is much closer to the hardware and operating systems than interpreted languages.

Do people or organizations care of wasting energy and have lousy performance? They should. If organizations do not care, your customers eventually will see a difference in quality between scripted languages and taking the time to produce fast and energy efficient code. One can always see a difference between native applications and scripted/browser applications.

For example, let’s say you are creating a cloud-based applications that uses AWS Lambda to execute your functions. If that Lambda is written in Python, you are wasting time and money. Since Python so incredibly slow, you are being charged in 100ms increments when you execute your Lambda function in AWS. Now, imagine your Lambda function is written in C? Your Lambda function will execute so much faster and you are being charged much less. It will be more cost efficient to execute in C rather than Python. So, it would be cheaper and you are contributing to a lower energy consumption.

The same idea can be applied when you create a Linux daemon service, a Windows service, or even a native Linux, MacOS, or Windows application. Have a look at NappGUI, a cross-platform C SDK that allows you to create super fast and beautiful C applications.

C is a really powerful language. Have a look at it and start using it in your next project.

My new course on Udemy: Build an Event Store in C# .NET for CQRS and Event Sourcing

Thomas Jaeger .NET, Architecture, AWS, C#, Domain Driven Design, DynamoDB, Event Sourcing , , , ,

I’m excited to announce my new course “Build an Event Store in C# .NET for CQRS and Event Sourcing“.

Real-time communication and feedback to your customers are more important then ever. Customers have become used to immediate feedback on the actions they take in your software. It is very hard in create, read, update, and delete (CRUD) based applications to provide this sort of responsiveness that your customers are demanding. Event-based solutions that are based on CQRS, Event Sourcing, and Domain-Driven Design (DDD) can offer deep insights in real-time to your customers and to your business. More importantly, you won’t loose data in an event-sourced solution when compared to CRUD-based solutions because your solution will be able to provide the context on why changes happened and record changes into an immutable log, the event store.

In this course, you will learn about the importance of using domain events as your source of truth instead of pieces of data that are incomplete in CRUD-based applications. You will be able to tell stories on what happened when and why. You will be able to answer future questions by your customers and business even though you may not have all the requirements at hand when you design and build your event-sourced solution.

At the very heart of your solution will be the event store. The event store is the source of truth in your entire solution. We will be building an event store in C#. NET and utilizing AWS DynamoDB as the persistence mechanism. However, the provided C# source code can easily be converted to Java or other languages. For the actual persistence, you could also use MySQL, PostgreSQL, MongoDB, and others. The C# code is abstracted so that can you can re-use it for specific persistence implementations. The concepts and code can work for on-premise, cloud only, or hybrid models. For an example read model implementation, we will be creating a read model using MySQL in AWS.

Once you understand the power of event sourcing, you won’t go back.

Design things to be replaceable, not reusable!

Thomas Jaeger Architecture, AWS, Cloud Computing, Design, Design Patterns, Domain Driven Design, Microservices, Serverless

I love this quote by Andy Hunt

“I find aiming to make things “replaceable” far better than making things reusable. Reusable means you’re fortune-telling, trying to divine an unknowable future.”

This is what I’m aiming for as well especially when you bring in the Sacrificial Architecture. Love it.

As you create your architecture, design all components to be replaceable, not reusable. This is a whole lot easier nowadays with AWS and Lambda creating serverless masterpieces, for example.

What are Projections in an Event Sourced Microservice Architecture?

Thomas Jaeger .NET, Architecture, AWS, Cloud Computing, Domain Driven Design, Microservices , ,

In this video I will show you the different types of projections in an event-based microservice architecture and how you can use them.

Projections can be created in code in your microservice or in read-models when domain events are received and processed, for example, you process a published domain event and then store the result of the analysis in Oracle, MySQL, SQL Server, etc.

Why you need serverless microservices, yesterday!

Thomas Jaeger Architecture, AWS, Cloud Computing, Course, Design, Domain Driven Design, Event Sourcing, Persistence, Programming, Serverless , ,

I have just published my FREE course: “Why You Need Serverless Microservices, Yesterday“. Enroll for FREE!

WhyYouNeedServerlessMicroservices_960x520In this course I will walk you through the many benefits of creating serverless microservices instead of the traditional node / instance approach including the use of containers. There are more than enough things to worry about when you want to create a new cloud system or transform a legacy system to operate in the cloud.

From a business point of view, there are huge benefits in going serverless rather than instance based (including containers). A very large jump in business agility can be achieved through focusing on the problems and opportunities rather than the technical jungle of traditional computing solutions.

From a technical point of view, it is almost nirvana where you can eliminate many points of failures in the architecture.

Serverless Microservices Online Courses

Thomas Jaeger .NET, Architecture, AWS, C#, Cloud Computing, Course, CQRS, Design, Design Patterns, Domain Driven Design, DynamoDB, Event Sourcing, Serverless, SQS , , , , ,

Update 04/03/2019: I have completed the FREE course: “Why you need serverless microservices, yesterday!“. Enroll for FREE!
At the moment, I’m working on four online courses in the following order:

 

“Why You Need Serverless Microservices, Yesterday“, FREE Course

WhyYouNeedServerlessMicroservices_960x520

In this course I will walk you through the many benefits of creating serverless microservices instead of the traditional node / instance approach including the use of containers. There are more than enough things to worry about when you want to create a new cloud system or transform a legacy system to operate in the cloud.

From a business point of view, there are huge benefits in going serverless rather than instance based (including containers). A very large jump in business agility can be achieved through focusing on the problems and opportunities rather than the technical jungle of traditional computing solutions.

From a technical point of view, it is almost nirvana where you can eliminate many points of failures in the architecture.

 

“How To Build An Event Store”, Paid Course

eventstore_960_520_darker

Your event store is the heart of an event sourced system. The event store is the source of truth for all business events. It needs to be able capture and replay all domain events in your system reliably and with great performance.

In this course, I will walk you through building an event store that you can re-use in your own projects. In addition, I will walk you through building a read model that allows you to query domain events from the read model.

I will also go through why building your own event store has many more advantages over using a third-party event store.

We will be building the event store in AWS DynamoDB but you can apply the design to a traditional RDMS SQL storage just as well. I will go over the pros and cons in doing so.

 

“Architecting And Designing Event Based Microservices”, Paid Course

ArchitectAndDesign

Learn the details on how to design and architect event based microservices using Domain Driven Design (DDD), Event Storming, CQRS, and EventSourcing techniques. I will show you how best combine many principles, patterns, and techniques to create an architecture with as few points of failures as possible and still deliver a great solution.

What you will learn can be applied to cloud based systems but also to traditional, on-premise systems. The benefits are great in either environments.

Whether you are designing a new system in a greenfield environment or transforming a legacy system, I will show tips & tricks that you can use depending which type of project you are in.

 

“Implementing A Serverless Microservice in AWS”, Paid Course

Code_960x520

Learn the details on how to implement a serverless microservice in AWS using Domain Driven Design (DDD), CQRS, and EventSourcing techniques.

We will build a fully functioning billing system in AWS using Visual Studio and C# .NET Core. Even if you are not familiar with C# and .NET Core, you will learn a lot of practical tips on using the different AWS services including building fully automated CI/CD pipelines.

 

I’m still working on these courses but I’m planning on publishing “Why You Need Serverless Microservices, Yesterday” and “How To Build An Event Store” first.

My New YouTube Channel

Thomas Jaeger .NET, Amazon S3, Architecture, AWS, C#, Cloud Computing, Course, CQRS, Design, Design Patterns, Domain Driven Design, DynamoDB, Event Sourcing, Persistence, Programming, Serverless, YouTube

I just created my new YouTube channel “Creating Great Software”. Have a look.

I will be publishing videos about creating great software including serverless computing in AWS. In addition, I will be publishing my STRONG opinions about the state of the software industry from time to time.

I’m super excited about this new channel and I’m looking forward in seeing your feedback. See you there!

Serverless System Course

Thomas Jaeger .NET, Amazon S3, Architecture, AWS, C#, Course, Design, Design Patterns, Domain Driven Design, DynamoDB, Event Sourcing, Persistence, Programming, Serverless, SQS

In the past 27 professional years, I have been fortunate enough to gain a lot of experience and wisdom in 10 different software industries. These experiences are not just successful accomplishments but also plenty of failures and mistakes made along the way. It is ok to make mistakes as long as you learn from them, only then you will gain true wisdom in anything you do.

Over the years, I have always thought, “If I could gather a small team, I could share my knowledge and we can create great things“. Due to many circumstances, I never had the opportunity to mentor a small team and being in full control on what we would create. By full control I mean free from politics and other constraints. The sort of freedom you need to create and explore ideas without worrying about budgets, timelines, etc. When you can foster such an environment, great things get invented. Just look at the past successes in Silicon Valley.

How can I share my experiences and wisdom most efficiently and help you achieve your goals much more rapidly rather than doing it alone? I have decided it is time to create an online school where I can share all my knowledge and wisdom from the software industry. Well, how about an online course that is very deep in all aspects. An online course that would teach you the birds eye view of concepts and then drills down into the very deep aspects of it. A course that could show you actually how to put the conceptual components together in a very cohesive way. All the different pieces put together in an architecture that makes real sense and is practical and maintainable. We’ll leave the fluff out and concentrate on the entire system starting from a users perspective and walk backwards into the technical implementation details. In short, a

Serverless System Master Class

This master class would be extensive and we will go into a lot of details until we have built a working system that you could use in your future projects. As a minimum, you would learn a lot of concepts and software architecture. Since technology is always changing, I believe that the concepts will be more valuable then the implementation details in the course. In addition, how you bring these concepts together and why are extremely important because in your day-to-day work, you will have constraints that you need to work with and make compromises. There is no perfect architecture and even in this master class, you will touch on many different options one could take and why.

Please let me know if you are interested in the Serveless System Master Class. I would love to hear your feedback as I work on the course. I plan to post regular updates on my blog here.

Leave a comment below or contact me directly at thomasjaeger at gmail dot com and let me know your level of interest, please!

Event Store in AWS DynamoDB

Thomas Jaeger AWS, CQRS, DynamoDB, Event Sourcing, Persistence, Programming

Update 05/27/2019: “How to build an event store masterclass” is now available. Learn how to build an event store using C#. NET Core, DynamoDB, MySQL for read models, and more.

In the past few weeks, I have been working on creating an event store in AWS DynamoDB and AWS S3. I use an event store for the domain driven design (DDD) concept in that system. Specifically, one of my systems uses CQRS and Event Sourcing (which is awesome, btw).

The idea for the event store started when I wanted to create my own event store for several reasons. I played with EventStore from Greg Young available at https://eventstore.org/ I encountered errors when I tested EventStore and I expected 100% functionality without issues. Besides, I did not want to babysit another persistence mechanism. I just do not have time for that.

I have created an event store before that was based entirely on Redis. That had worked great and it was super-fast. I used https://redislabs.com/ service to allow zero-maintenance of a fully clustered Redis solution. This has been working for some time now. The only problem was that it can have inconsistencies when it comes to running many applications nodes that use the Redis event store. Long story short, this has to do with complex timing, concurrency violations, etc.

So, I thought there must be a better way. I want zero administration headaches ideally, but I want to take advantage of the consistency capability of the underlying storage mechanism. I want the event store to be a service to the application itself that runs within the same process of the application. So, no need to babysit a separate event store cluster. I want the event store conceptually living side by side within the application.

If I can only take advantage of the consistency of the persistent mechanism, the application can then handle error conditions etc. accordingly based on what the use cases are.

After some experimenting and doing quiet a few load tests using http://loader.io, I can now say that the Event Store in DynamoDB has been born. My first 10,000 user / min load test has passed with flying colors and reached API response times of less than 10 ms with sustained rates of 170 clients / second. All this was running on a single cheap t2.medium instance. DynamoDB had to increase the throttling automatically to take this kind of load. This is one of those cool features, btw.

The event store I created in DynamoDB uses two concepts, Aggregates and Change Sets. Aggregates are the aggregates in your DDD system but only a handful of meta data. Change sets are one ore more domain events that are created when your DDD system processes a command.

I know I can get much higher numbers even by adding additional nodes and doing further tweaking. I would have to do much more load testing of course and tweaking but so far this has been a tremendous success and I’m super excited to take advantage of DynamoDB.

Anyways, I wanted to share this information. Maybe in the in future I can go into much more details. Time is the only problem I have really.

Object Persistence Reference Implementation

Thomas Jaeger .NET, AWS, C#, Cloud Computing, db4o, Design, Design Patterns, Persistence, SimpleDB , , , , ,

I’ve been updating my reference implementation in the last few days. I’m actually using this reference implement in my own projects. You can download the latest version on my GitHub repo.

This is a complete .NET C# reference implementation to help you jump start a service oriented system running in a cloud environment such as Amazon’s EC2 or on-premis clusters.

This reference implementation shows you how to build a client and the server side. The client side is a sample WPF application that communicates via http REST requests using JSON payloads to the service side. Of course, you can use any type of client as long as the client can communicate via http and REST based JSON’s.

The service side is using a Web API 2 service layer that communicates to a central domain model. The service side demonstrates how to handle exceptions and edge cases and how to communicate failure to the client.

The persistence layer demonstrates the extreamly powerful provider pattern to store the domain objects into the following databases:

  1. db4o (an object database)
  2. Redis (a NoSQL database)
  3. SimpleDB (a NoSQL database)
  4. SQL Server (comming soon)

Please note that the entire system has no knowledge on how the objects are stored. All implementation details are in the individual providers listed above. This means that you can switch the persistence provider without having to recompile and therefore switch a running system from one persistence store to another.

I will try to create a sample SQL Server provider soon.