summaryrefslogtreecommitdiff
path: root/scripts/profiling/storage/benchmark-storage.py
diff options
context:
space:
mode:
authordrebs <drebs@leap.se>2016-07-31 10:37:35 -0300
committerdrebs <drebs@leap.se>2016-08-01 21:09:05 -0300
commit3b237bb46743a93feed4bb6f3c839d72fc28df48 (patch)
tree7f09ba9d2954bbc94c659e2af497efb187789c8f /scripts/profiling/storage/benchmark-storage.py
parent027d0b5f40944973807e1a4fc497c496e78b3eeb (diff)
[feat] use couch _all_docs for get_docs() and get_all_docs()
The previous solution would make use of concurrent get's to couch backend in a pool of threads to implement the get_docs() and get_all_docs() CouchDatabase backend methods. This commit replaces those by a simpler implementation use the `_all_docs` couchdb view api. It passes all needed IDs to the view and r etrieves all documents with content in the same request. A comparison between both implementations shows an improvement of at least 15 times for large number of documents. The table below shows the time for different implementations of get_all_docs() for different number of documents and threads versus _all_docs implementation: +-------+-----------------+------------------+-------------+ | | threads | _all_docs | improvement | +-------+-----------------+------------------+-------------+ | 10 | 0.0728030204773 | 0.00782012939453 | 9.3 | | 100 | 0.609349966049 | 0.0377721786499 | 16.1 | | 1000 | 5.86522197723 | 0.370730876923 | 15.8 | | 10000 | 66.1713931561 | 3.61764383316 | 18.3 | +-------+-----------------+------------------+-------------+
Diffstat (limited to 'scripts/profiling/storage/benchmark-storage.py')
0 files changed, 0 insertions, 0 deletions