diff options
author | drebs <drebs@leap.se> | 2016-07-31 10:37:35 -0300 |
---|---|---|
committer | drebs <drebs@leap.se> | 2016-08-01 21:09:05 -0300 |
commit | 3b237bb46743a93feed4bb6f3c839d72fc28df48 (patch) | |
tree | 7f09ba9d2954bbc94c659e2af497efb187789c8f /scripts/profiling/storage/plot.py | |
parent | 027d0b5f40944973807e1a4fc497c496e78b3eeb (diff) |
[feat] use couch _all_docs for get_docs() and get_all_docs()
The previous solution would make use of concurrent get's to couch
backend in a pool of threads to implement the get_docs() and
get_all_docs() CouchDatabase backend methods.
This commit replaces those by a simpler implementation use the
`_all_docs` couchdb view api. It passes all needed IDs to the view and
r etrieves all documents with content in the same request.
A comparison between both implementations shows an improvement of at
least 15 times for large number of documents. The table below shows the
time for different implementations of get_all_docs() for different
number of documents and threads versus _all_docs implementation:
+-------+-----------------+------------------+-------------+
| | threads | _all_docs | improvement |
+-------+-----------------+------------------+-------------+
| 10 | 0.0728030204773 | 0.00782012939453 | 9.3 |
| 100 | 0.609349966049 | 0.0377721786499 | 16.1 |
| 1000 | 5.86522197723 | 0.370730876923 | 15.8 |
| 10000 | 66.1713931561 | 3.61764383316 | 18.3 |
+-------+-----------------+------------------+-------------+
Diffstat (limited to 'scripts/profiling/storage/plot.py')
0 files changed, 0 insertions, 0 deletions