Microservice Architecture & Implementation With Asp.Net Core

Demo application is available on https://github.com/suadev/dotnet-istanbul-microservices-demo.

Transcript

1. Microservice Architecture & Implementation With Asp.Net Core Suat Köse 29th March, 2019 İstanbul Contact : suadev@gmail.com Blog : medium.com/@suadev Code : github.com/suadev Tweet : kose__suat
2. Agenda • Definition of Microservice Architecture • Do You Really Need Microservices? • Ready to go for Microservices? • Advantages and Challenges • Data Consistency in Microservice Architecture • Communication Between Microservices • Demo - Event-Driven Communication 2 / 20
3. Microservice Architecture , is a kind of distributed system that consist of multiple single purpose microservices. A microservice , is a service that independently developable, deployable, scalable and designed as simple as possible. Always remember these 2 principles for your microservices ; - Be fully independent. - Do one thing and do it well. Definition of Microservice Architecture 3 / 20
4. Do You Really Need Microservices? - Yes, because it is hype. - Stay away from Hype-Driven Development. - Monoliths are not outdated. - Yes, because our application and code-base are growing day by day. - You need to scale out your entire app or some key functionalities ? - You want to use right tool for the right task, but you can’t ? - You want to be agile, but you can’t ? 4 / 20
5. Ready to go for Microservices? - Team(s) that have adopted DevOps culture and Agile methodologies. - Container Technology and Cloud Provider Expertise - Monitoring & Alerting System - Centralized Logging Infrastructure - Good Communication Between teams. 5 / 20
6. Advantages of Microservice Architecture - Being independent of technology. ( language, framework, db, etc. ) - Splitting the code-base into small service and teams. ( easy to manage ) - No single point of failure. - Easy and effective scalability. - Easy to understand. ( fast adaptation of new employees ) 6 / 20
7. Challenges of Microservice Architecture - DevOps Cost → Containers, Cloud Providers, CI-CD pipelines etc. - Integration Test Cost → It’s not easy even in monolith systems. Give a try to CDC testing? - Code Duplication → DRY principle ? - Being Independent → Shared libs, Synchronous communication needs etc. - Defining Boundaries of Microservices → If the two services are communicating intensively, maybe they must be turned into a single service. 7 / 20
8. Data Consistency in Microservice Architecture - How does the data can be inconsistent and how do we prevent it ? - How do we manage data consistency in distributed systems ? - Workflow Engine ? A lead actor may cause a tightly-coupled system. - Two-Phase Commit (2PC) ❌ It is a performance killer. - Event-Driven Architecture ✅ - Saga Pattern. Let’s talk about 2PC and Saga. 8 / 20
9. Two-Phase Commit 2PC consist of two stages that managed by a coordinator. ( which is a single-point-of-failure ) - Prepare Step ( also known as voting ) - Commit Step 9 / 20 ⚠ Participants keep locks on resources until they receive the voting result from the coordinator.
10. - A Saga is a chain of local transactions. - Saga approach removes the need for 2PC, but also brings new challenges. - In case of failure, all committed transactions must be rollbacked by compensation transactions. - Compensation transactions must be idempotent and cannot be abort. Saga Pattern 10 / 20 image from https://blog.bernd-ruecker.com
11. Saga Pattern - Choreography / Events Method Each transaction publishes domain events for trigger others. As the number of steps increases, the complexity also increases. ⚠ - Orchestration Method An orchestrator tells the services ( participants ) what local transactions to execute. May cause a more tightly-coupled design than Choreography. ⚠ 11 / 20 - You have to manage concurrency and race condition if you implement Saga by yourself. - Or you may want to use a framework such as Masstransit, NServiceBus etc. ⚠
12. Saga Pattern - Choreography Method image from https://microservices.io 12 / 20
13. Saga Pattern - Orchestration Method image from https://microservices.io 13 / 20
14. Communication Between Microservices - Request / Response ( Rest over Http - Synchronous ) A microservice makes a real time http request to communicate with another one. - gRPC ( Remote Procedure Call - Synchronous ) High-performance communication framework designed for making RPC calls. - Event-Driven ( Messaging - Asynchronous ) A microservice publishes an event to the message broker. It has no idea about which services are interested in the event. 14 / 20
15. Request / Response Communication - We need to reduce http calls between our services as much as possible. Because ; Http is a synchronous protocol. It violates the ‘independent microservices’ rule. - How can we reduce http calls ? ( if you can’t reduce, do it async. ) Combine heavily communicating microservices. Give a try to Domain-Driven Design approach. ( sharing data across bounded contexts ) 15 / 20
16. How DDD Approach Helps for Reducing Http Calls? If service A depends on the data that service B provides, it may store a replica of it to avoid the http request. But this brings a new challenge like syncing data between A and B. Customer Service BasketTable Id CustomerId CreateDate Order Service BasketTable CustomerId Suppose that, Order service validates that customer has a basket on each new order request. Basket doesn’t belong to Order domain. 16 / 20 Eventual consistency based on async communication <= =>
17. Event-Driven Communication - Producers and consumers are decoupled from each other. - Enables to design a fully-asynchronous, long-running transactions. - Enables high scalability. - Easy to trigger multiple services with single event. 17 / 20 What if service A not only depends on the data but also needs to trigger service B ? Then communication is must, but it doesn’t need to be request / response style.
18. Demo - Architecture Api Gateway Customer Service Order Service Product Service Notification Service RabbitMQ Login Request Command Query Validate Credential Jwt Token 202 Accepted Query Result Command - Token verification is done only by api gateway for each request. - Services can be placed in a private network and accessible by only api’s ip address. Event Event Query 18 / 20 DB DB DB
19. Demo - Tool Set - Asp.Net Core 2.2 - Entity Framework Core 2.2 - PostgreSQL - Npgsql - Serilog - Elasticsearch - Kibana - RabbitMQ - Docker Containers ( PostgreSQL, RabbitMQ, Elasticsearch and Kibana ) - pgAdmin or Azure Data Studio - VS Code Show me the code ! 19 / 20
20. Thank You ! - https://docs.microsoft.com/tr-tr/dotnet/standard/microservices-architecture - https://sookocheff.com/post/architecture/fault-tolerance-and-data-consistency-using-distributed-sagas - https://solace.com/blog/messaging-between-microservices - https://youtu.be/E8-e-3fRHBw - Managing Data in Microservices - https://youtu.be/KPtLbSEFe6c - Data Design and Modeling for Microservices - https://stackify.com/communication-microservices-avoid-common-problems - https://microservices.io - https://www.youtube.com/channel/UCc3apIciZhgTUw_kk6C9EJQ - Dev Mentors - http://udidahan.com/2011/07/13/the-danger-of-centralized-workflows Contact : suadev@gmail.com Blog : medium.com/@suadev Code : github.com/suadev Tweet : kose__suat