diff options
| author | drebs <drebs@leap.se> | 2016-07-31 10:37:35 -0300 | 
|---|---|---|
| committer | drebs <drebs@leap.se> | 2016-08-01 21:09:05 -0300 | 
| commit | 3b237bb46743a93feed4bb6f3c839d72fc28df48 (patch) | |
| tree | 7f09ba9d2954bbc94c659e2af497efb187789c8f /scripts/profiling/doc_put_memory_usage | |
| parent | 027d0b5f40944973807e1a4fc497c496e78b3eeb (diff) | |
[feat] use couch _all_docs for get_docs() and get_all_docs()
The previous solution would make use of concurrent get's to couch
backend in a pool of threads to implement the get_docs() and
get_all_docs() CouchDatabase backend methods.
This commit replaces those by a simpler implementation use the
`_all_docs` couchdb view api. It passes all needed IDs to the view and
r etrieves all documents with content in the same request.
A comparison between both implementations shows an improvement of at
least 15 times for large number of documents. The table below shows the
time for different implementations of get_all_docs() for different
number of documents and threads versus _all_docs implementation:
  +-------+-----------------+------------------+-------------+
  |       |     threads     |    _all_docs     | improvement |
  +-------+-----------------+------------------+-------------+
  |    10 | 0.0728030204773 | 0.00782012939453 |      9.3    |
  |   100 | 0.609349966049  | 0.0377721786499  |     16.1    |
  |  1000 | 5.86522197723   | 0.370730876923   |     15.8    |
  | 10000 | 66.1713931561   | 3.61764383316    |     18.3    |
  +-------+-----------------+------------------+-------------+
Diffstat (limited to 'scripts/profiling/doc_put_memory_usage')
0 files changed, 0 insertions, 0 deletions
