Event Driven Architecture and its Microservices

Event-Driven Architecture for Microservices

What is Event-Driven Architecture?

Event-Driven Architecture (EDA) is a model or an architectural paradigm for software that supports the production, detection, consumption of, and reaction to the event or a significant system state change. This structure consists of event creators and event consumers. Creators are the ones who produce events and have knowledge about what events occurred. On the other hand, consumers are the ones who become affected by events occurring, and they are also involved in processing events.

The cloud-native products are mostly open source with a commercially hosted version available in the cloud. Source - Beyond Kubernetes - 5 Promising Cloud-Native Technologies

Its architecture and it's patterns are sometimes related to stream processing architecture or message-driven architecture based on its fluctuations. Message Driven Architecture center combines the categorized systems by transferring messages from one module to another module of the system using a standard Message Broker. Event stream processing runs by controlling a data set by one data point at a time and architecture pattern places events at the heart of systems.

Why does its architecture matter?

Loosely Coupled Structure – Event creators and consumers loosely coupled with event bus help, which helps them to the couple and decouple into different networks in response to various events. It also helps in giving different developers to develop different event processors. And use these components and also reused by many different networks.
In Real-time Analytics – Optimize it for real-time analytics. It has a much higher capacity to find, analyze, and then respond to patterns in time before critical events happen. Use complex event processing to achieve real-time analytics.
Versatility – A system that follows its facilitates greater responsiveness because its systems are, by design, normalized to unpredictable, nonlinear, and asynchronous environments. These systems are highly adaptable to different circumstances.

What are the characteristics & considerations for choosing EDA?

The central characteristics & considerations of it:

Multicast communications: The publishers can send events to various systems that have subscribed to them. This is opposite to unicast communication in which publishers can send the information only to one subscriber.
Real-time transmission: Publishers generate the events and when the event occurs in real-time to the subscribers. The mode of processing is real-time rather than batch processing.
Asynchronous communication: The publisher doesn't wait for the receiver to prepare an event before sending the new event. It can perform asynchronous operations in correspondence. This results in more reliable performance of the architecture, irrespective of the time stay involved in queuing and dequeuing messages.
Fine-grained communication: Publishers keep publishing small and specific events instead of waiting for a single aggregated event.
Event ontology: Its systems always have a technique to classify events in terms of group/hierarchy based on their general characteristics which gives flexibility to the subscribers to subscribe to a specific event and subscribers to subscribe to a specific category
Deployment and development: Its pattern elements are loosely coupled, and development is also easy because of the pattern's asynchronous nature.

Digital Business Transformation is the application of technology to build new business models, processes, software and systems. Click to explore about, Top 5 Application Architecture for Digital Transformation

Event-Driven Architecture in the Cloud-Native World

Event-driven architectures and reactive programming models aren't new ideas. But, during Cloud-Native architectures with microservices, container-based workloads, and serverless computing, you can revisit its approaches in the cloud-native context. Two phases of a cloud-native architecture are necessary for developing it:

Micro-services implement the loosely coupled application architecture that allows deployment in highly distributed patterns for flexibility, agility, and scale.
Cloud-native platforms with containers and serverless deployments present the application platform and tools that realize the flexibility, agility, and scalability of the microservices architecture.

How does it works?

Listed below are the working stages of Event-Driven architecture:

Publish/Subscribe (Pub/Sub) Messaging

Pub/Sub is a form of asynchronous service-to-service communication used in microservices and serverless architectures. When publishing or generating, an event it sends the message to each subscriber. A massage doesn’t repeat after its received, and new subscribers do not see the event. Subscribers do not need to know anything about the publishers, and vice-versa other than the publisher is a legitimate publisher and the entitled subscriber is to receive information.

Event Source

The event source sends events to the event processing system. These sources can be one or more which can send an event to a single or multiple event processors.

Event Processor

This system can perform various actions on events –

Merely enrich a single event; for example, adding a timestamp to the event data called Simple Event Processing.
Add information about the source of the event. Process multiple single events, from multiple event sources against event patterns to produce a new derived or compound event called Complex Event Processing.

Event Consumer

The event consumer reacts to the event. An event consumer can be as simple as updating a database or business dashboard, or as complex as required, carrying out new business processing as a result of the event.

Here are some examples of consuming an event –

Updating a business dashboard
Updating a database
Interacting with Web 2.0 components to dynamically update web pages
Sending an alert by using email or SMS based on event data
Putting event data on a message queue
Starting a new piece of application work

Event processing is categorized by processing complexity and sophistication having two categories –

Simple Event Processing – Simple events are events that are independent of other events and these events are processed under this category. These events do not summarize. Such events can provide considerably and provide excellent value business information. Simple processing includes processing such as augmenting the event payload with additional data, changing the events schema from one form to another, generating multiple events based on the payload of a single event, and redirecting the event from one channel or stream to another.
Complex Event Processing – Complex events are events that are dependent on other events and these events are processed under this category. These events do summarize. Complex processing involves applying a collection of evaluation constraints or conditions over an event set. Results of processing should not be complicated for the user, distinguish between the complex and vibrant nature of the information available from this processing.

Because of their flexibility, failure, and self-healing properties, Kubernetes implementation is the best for production purposes. Source - Implementing a Kubernetes Strategy in Your Organization

How to implement event source in Kubernetes?

Event Sources are Kubernetes Custom Resources which implement a tool for expressing interest in a collection of events from a particular software system. For different events, sources may be defined by different CustomResources.

Kubernetes Events

Kubernetes events are a source type in Kubernetes that are automatically generated when other sources have state changes, errors, or other messages that should be declared to the system. While there is not a lot of documentation available for events in Kubernetes, they are an invaluable source when debugging issues in the Kubernetes cluster. It is necessary to understand that events do not develop in a way that you might expect. For example, if the container is weak to start because the docker image cannot be pulled the events are only visible on the pod, not on any Replica-set, StatefulSet, Deployment, or any other resource that generated the pod.

Event-Driven Architectures and Microservices

Modern microservices designs are reactive and event-driven and built to perform particular tasks. These tasks are usually based on the occurrence of some event. As a result, they lose connection and require updation and maintenance. You may use its microservices to create more adaptable and less complicated applications to maintain over time. Therefore, its architectures form the backbone of microservices.

What are the best tools?

A message broker is a middle-ware software, tool or SaaS used to send events between event producers and consumers in a publish-subscribe pattern. All implementations of its architecture need to use some technology in an event broker or message broker's role. there are various Tools for message brokers but the top three tools which are used:-

KubeMQ

KubeMQ is a Cloud Native, enterprise-grade, message broker for shared services architecture. It is given as a small, lightweight Docker container, created for any workload and architecture working in Kubernetes or any new container orchestration system which supports Docker. Each container's capacity is 30MB, which gives it natural to perform in a microservice architecture. The capacity of a cluster of three containers is 90 MB. KubeMQ was developed in GO to maximize computer resources and work smoothly with Kubernetes. According to Kubernetes experts when they benchmarked KubeMQ using a cluster with a 1k message size and sent one million messages with the same hardware specs. KubeMQ beat another message broker by 20% in speed.

Kafka

Apache Kafka is a very successful publish/subscribe system that can probably process a stream of data. Kafka's central concept is a topic that can be replicated across a cluster giving a safe data warehouse. By performing processed message offsets following Kafka, it is nearly straightforward to perform guaranteed "at-least-once" processing. Kafka is a stateful service, and this does make the Kubernetes configuration more difficult than stateless micro-services. Zookeeper is a centralized set of services for managing configuration data, labeling, providing shared synchronization, and providing group services. Deploy to manage a cluster, both Kafka and Zookeeper. Each Kafka container weighs about 600 MB, and each Zookeeper container weighs about 100MB, totaling 2.1GB (three Kafka containers + three Zookeeper containers). Kafka is heavy and manages large microservice without any problem.

NATS

NATS is a simple, secure, and high-performance, open-source messaging system for cloud-native applications, IoT messaging, and microservices architectures. The NATS server develops in the Go language. The center design systems of NATS are performance, ease of use, and scalability. A data streaming system powered by NATS combines persistence, message replay, and durable subscriptions to NATS. NATS supports four main patterns for communicating messages across entities. These are Subject-Based messaging, Publish-Subscribe, Request-Reply, and Queue Group. Shows a quick summary of difference between KubeMQ, Kafka and NATS

What are the challenges of it?

The key difficulties in an EDA are Guaranteed delivery, Processing events in order or exactly once, and managing consistency across service boundaries.

Almost once: The producer transfers a message, and the consumer demand may / may not receive it.
At least once: The producer transfers a message, and the consumer may prepare duplicate occurrences of the message.
Exactly once: The producer transfers a message exactly once, and the consumer processes it exactly once in EDA.

Top Terminologies of Event-Driven Architecture

The top 4 terminologies of Event-Driven Architecture are defined below:

Event Bus

An event bus is a system or program or application which receives events from event creators and sends that event to any event consumers used in decoupling event creators and event consumers. Consumers can also post events to event buses if they are dependent on other events and the event bus is not responsible if the receiver does not receive any message.

Event Creator/Producers

An event creator or producer or publisher is a program or application which creates events and sends events to the event bus. The event creator knew what events occurred.

Event Consumer/Processor

An event consumer or processor or subscriber is a program or application which receives events from the event bus without knowing which creator creates that event. The event processor processes the events and from that take particular actions written in events.

Event Log

Store the messages written to event logs on a disk. If the system crashes, the system can resume its state by merely replaying the events logs. Take the backup of event logs and use them in performance tests on new releases before deploying them to production.

Cloud-native applications are deployed using Kubernetes which is an open-source platform designed for automating deployment, scaling, and management of containerized applications. Click to explore about, Cloud-Native Application Architecture

Topologies In Event-Driven Architecture

The Topologies In Event-Driven Architecture are defined below:

Mediator Topology

It has a single event queue in to store in order and pop out in order only. Use a mediator to direct each event to relevant event processors. Filtre each event is or pre-processes in an event channel before fed into the event processors.

Broker Topology

There is no event queue in this topology. Broker in this topology is event processors that are responsible for everything like chain events, obtaining events, processing, and publishing another event indicating the end. Once a processor processes an event, another event is published so that another processor can proceed.

What are the best practices of it?

Supports the business demands for better services (no batch, less waiting).
No point-to-point integrations.
High-performance and highly scalable systems.
Components can remain autonomous, being capable of coupling and decoupling into different networks in response to different events.
Complete advanced analytics using complex event processing.

Our solutions cater to diverse industries with a focus on serving ever-changing marketing needs. Click here for our Cloud Native Application Development

Conclusion

The event-driven architecture enables real-time, complex transactions among IoT devices, data streams, and workflow systems. Every enterprise needs to respond quickly and effectively to evolving markets or circumstances. Greater operating efficiencies: a loose combining of the EDA without disrupting the layering, which allows for smooth, cost-effective service gives enterprises the competitive advantage they need.