Setting database for performance #1422
Unanswered
cezar-radan
asked this question in
Q&A
Replies: 1 comment 10 replies
-
Hi @cezar-radan, thanks for reaching out. Your description is very well described :) Currently, to get benefits from concurrent writers, transactions must be read-only e.g. Set, SetAll. ExecAll is acquiring a lock, thus why increasing the number of writers is not producing a significative benefit. Multi-version Concurrency Control (MVCC) is partially implemented but will be fully implemented in the near future. With that, it will be possible to obtain benefits from concurrency when the transaction does not only involves writes but read and writes. |
Beta Was this translation helpful? Give feedback.
10 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi
I performed some load tests over an immudb database and I didn't get good scores, therefore I decided to ask for guidance related to the database settings / immudb service settings, in case of heavy load.
Testing environment:
test machine
CPU - Intel® Core™ i7-10750H CPU @ 2.60GHz × 12
MEM - 32,0 GiB
SSD disk
immudb (v 1.3.2) - docker container running locally, on test machine
immuclient (v 1.3.2) - used in a golang application running locally, on test machine
Test scenario is:
Step1 - create some entries (
- create client + open session
- prepare 10 key-values
- insert key-values using ExecAll (precondition key must not exist)
)
Step2 - update entries created in Step1 (
- create client + open session
- prepare new values for the 10 keys
- update keys using ExecAll (precondition key must exist))
)
Step3 - get entries created in Step2(
- create client + open session
- retrieve key-values using GetAll
)
Test Results:
Serial running, 1 process that cycle (Step1 -> Step2 -> Step3) 1000 times.
In this case the runtime is acceptable, example:
- create client + open session = 50 - 62 milliseconds
- ExecAll = 10 - 13 milliseconds
- GetAll = 5 - 10 milliseconds
Concurrent running, 100 process that cycle (Step1 -> Step2 -> Step3) 1000 times.
In this case the runtime is not so good, example:
- create client + open session = 600 - 800 milliseconds
- ExecAll = 6500 - 9000 milliseconds
- GetAll = 100 - 150 milliseconds
So, it is very clear that in a concurrent environment MY immudb doesn't respond as expected.
I had a look over your mentions and over your code :
https://github.com/codenotary/immudb#performance-figures
https://github.com/codenotary/immudb/tree/master/tools/testing/stress_tool_test_kv
https://github.com/codenotary/immudb-tools/tree/main/stresstest
and, it seems there are various database parameters/configs that could be changed in order to improve the performance:
maxConcurrency (30 <-> 100)
maxIOConcurrency (1 <-> 100)
indexSettings.cleanupPercentage (0.1 <-> 10)
.... so on
I have run tests with various database parameters values (many combinations) but, unfortunately, I didn't obtain any improvement.
Please, could you provide some guidance for database settings/configs in order to obtain a better performance (heavy inserts) ?
I am pretty sure, in my case, the indexing process should be configured with care too.
Thank you very much.
Beta Was this translation helpful? Give feedback.
All reactions