-
Notifications
You must be signed in to change notification settings - Fork 15
Limitations
Single document (header/event/snapshot): 2MB
Single stream: 20GB
Number of streams: unlimited
Events in transaction: 99
Current Cosmos DB limits: https://docs.microsoft.com/en-us/azure/cosmos-db/concepts-limits
Eveneum partitions data by stream (StreamId
property), following best practices of data partitioning in Cosmos DB.
Each stream becomes a logical partition in Cosmos DB and events appended will appear in a consistent manner in Cosmos DB Change Feed. Large number of small logical partitions guarantees even distribution among physical partitions and allows for maximum utilization of provisioned throughput.
Current limitation for Cosmos DB physical partitions limit single stream to 20GB and single document to 2MB, which should be plenty in most scenarios.
Eveneum uses transactional batch feature to ensure header and events are written in a transaction, so either all writes succeed or none. CosmosDB has a limit of 100 operations in a single batch, which mean that the header and first 99 events will be written in a single transaction. When writing more than 99 events in one go, remaining events will be written in batches of 100.
If you expect a single stream to grow beyond 10GB, you might consider the following:
-
Are you storing very large payloads?
It is a good practice to store large payloads (images / binary data / xml documents) in a different data store (Azure Blob Storage, for example) and only include a link / identifier in the actual event. Cosmos DB is not designed for asset storage and each document (event) is limited to 2MB. -
Can you redesign your aggregate boundaries?
Consider splitting the stream into smaller ones. For example, instead of having a single aggregate that represents the number of available rooms in a hotel, create a smaller aggregate that represents availability for a single day. As time passes, streams that represent past days won't be used and checking availability for a few days won't load all events for other days. -
Do you really need to store ALL events?
Depending on your business requirements, you may not need to persist all events forever. You might want to capture stream state in a snapshot and delete previous events. Snapshots can be taken every set number of events (eg. every 10,000 events) or at a specific time (eg. end of business day / end of month). Your stream will contain a collection of snapshots and all events since the last snapshot. -
Contact Cosmos DB Team (AskCosmosDB@microsoft.com) for a possibility of provisioning larger instances.