summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2012-11-14Remove the client ref when it diesPaul J. Davis
We have observed periods of couchjs processes spiking into the hundreds and thousands for short periods of time since the new couch_proc_manager was released. Today I happened to catch one in the act and poked at couch_proc_manager's ets table. There seemed to be a few more couchjs processes with clients than I would have expected so I skimmed the code looking for a place where we didn't clear the client value (which would prevent it from being reused so that it would eventually just timeout). I found a case where if the Pid that checked out the process dies without the OS process dying, we were forgetting to clear the client in the ets table. This patch refactors the two places we return processes into a single function call which clears the OS process client.
2012-11-14Two last bugs in couch_proc_managerPaul J. Davis
* Design doc languages are converted to lists * Make sure to monitor every client correctly
2012-11-14Use a match_spec() instead of a match_pattern()Paul J. Davis
I misread the docs on what was expected for ets:select_receive/3.
2012-11-14Avoid scanning the entire table of procsPaul J. Davis
For large numbers of os processes its possible that we have a slowdown when requesting a new process. The old code matches all possible processes out of the table to find an appropriate candidate. We avoid the issue by using ets:select_reverse to also prefer keeping newer processes and releasing longer lived processes. Length of life is based on the implicit sorting of pids having newer pids sorting larger.
2012-11-14Mark proc as unused in ret_procAdam Kocoloski
Otherwise we'll never reuse it
2012-11-14Trigger a GC when OS process is returnedAdam Kocoloski
2012-11-14Allow processes to idle before exitingPaul J. Davis
When system load exceeds the ability of os_process_soft_limit to keep up with demand we enter a fork-use-kill (FUK) cycle. The constant spawning and destruction os these processes thrashes system resources and causes general instability. This patch changes the behavior from killing each process as its returned to letting it idle for a configurable amount of time (default five minutes) which allows it to be reused by other clients. This way we can avoid adding unnecessary load when demand for couchjs processes exceeds os_process_soft_limit. As a happy benefit this should also allow os_process_soft_limit to be set much lower since the number of processes will now more closely follow actual demand (instead of provisioning for the worst case scenario). Conflicts: apps/couch/src/couch_os_process.erl apps/couch/src/couch_proc_manager.erl Conflicts: apps/couch/src/couch_os_process.erl
2012-11-14Fork new OS processes outside proc_manager loopAdam Kocoloski
2012-11-14Add an index keyed on LRU for faster candidate IDAdam Kocoloski
Our current implementation for closing an LRU DB involves a full scan of a public ets table. This scan blocks all other activity in couch_server and can become a serious bottleneck when the LRU cache hit rate drops too low. In the worst-case all_dbs_active scenario we end up with O(N**2) algorithmic complexity. This patch adds a new index keyed on LRU for faster access to the least recently used databases. It also moves the ets table to a dict on the couch_server heap. The downside is an increased message rate inbound on the couch_server, as clients are no longer allowed to update the LRU data structures without sending a message. BugzID: 12879 Conflicts: apps/couch/src/couch_server.erl
2012-11-14Expose latest=true option in calls to retrieve revisionsBob Dionne
When a call is made to retrieve a specific revision, latest=true will retrieve any descendent leaves instead. This enables the replicator to better keep up with edits that occur whilst it's retrieving revisions BugzID: 14241
2012-11-14Skip ddoc pull, use middleman for validation resetAdam Kocoloski
We were pulling a list of design documents and then ignoring the result when the #db was a partition of a clustered database. Also, the call to fabric:reset_validation_funs/1 can occasionally cause a stray rexi_EXIT message to arrive in the db_updater mailbox (and subsequently kill the server) if a worker fails. I don't think that's desired behavior, though it's a debatable point. This patch spawns a middleman process to act as a sink for those stray messages. BugzID: 13087
2012-11-14Fix spurious declarations of new merge conflictsAdam Kocoloski
This patch also adds extra tests of the key tree merging logic as well as edoc-formatted documentation for the module and a few of the merge functions. Closes COUCHDB-902. Thanks Paul Davis, Bob Dionne, Klaus Trainer. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1065471 13f79535-47bb-0310-9956-ffa450edef68
2012-11-14Support jsonp in externals.Robert Newson
Allow ?callback= for any external that returns json (i.e, uses "json":{} instead of "data":"data". BugzID: 12748
2012-11-14Bump dependenciesRobert Newson
2012-10-23don't delete view groups that fail to openRandall Leeds
...but log a message instead. Fixes COUCHDB-1445.
2012-10-03Support multiarch library paths on Linux.wohali
2012-09-18Bump dependenciesRobert Newson
2012-08-09Don't always require exactly N replies for an mp attachment PUTRobert Newson
It's not safe to assume we require, or will receive, exactly N replies (where N is read from the "n" key of the "cluster" section of the configuaration). This needs proper fabric-ification. This commit will at least allow replication tests with clusters of less than N nodes where the documents have attachments (which triggers the multipart code). BugzID: 14258
2012-08-08Bump dependenciesRobert Newson
2012-08-02Add _replication_statsRobert Newson
2012-06-19Bump couch_replicator to 0.1.2Robert Newson
2012-06-19Don't perform callbacks to couch_replicatorRobert Newson
2012-06-18Fix _replicator dbRobert Newson
2012-06-18Add a dummy replicate/3 for nowAdam Kocoloski
Conflicts: apps/couch/src/couch_rep.erl
2012-06-18Revert "allow replication callback module to be chosen at runtime."Robert Newson
This reverts commit faf9071260147275bbac1633b599e85b4a302e8b.
2012-06-16Fix URL for twigAdam Kocoloski
2012-06-16Fix deprecation warnings for port_compilerAdam Kocoloski
2012-06-16Update lots of dependenciesAdam Kocoloski
2012-06-16Bump rebar to version 2.0.0Adam Kocoloski
2012-04-25Merge pull request #105 from Asquera/fixes/gentoo-buildAdam Kocoloski
Allow building on Gentoo
2012-04-25Fix Build for GentooFelix Gilcher
Modify SConscript so that it recognizes the standard name for the mozjs/spidermonkey lib on gentoo.
2012-04-24WhitespaceBob Dionne
2012-03-20Invoke couch_view_updater with the proper argumentsAdam Kocoloski
This fixes stale=update_after.
2012-03-16Enable Travis buildsRobert Newson
2012-02-22Merge 0.4.xRobert Newson
2012-01-31Fix previous commitRobert Newson
2012-01-31Fix bug in replicator request pipliningPaul J. Davis
A replication with both an HTTP source and target on the same host and port could end up in a dead lock due to ibrowse replication pipelining when attachments are present on the source. The ibrowse http worker would end up forming a multipart/mime body using anonymous reader functions for attachment stubs. When the attachment stub functions are executed it is possible that they end up assigned to the same ibrowse worker. This is a bit of a long path but then end result is equivalent to calling gen_server:call(self(), Args, infinity) from a gen_server callback. A quick work around for users is to set up a DNA alias (possibly in /etc/hosts) or to use a combination of hostname and ip address so that ibrowse assigns the requests to different pools.
2012-01-27WhitespaceBob Dionne
2012-01-24Minor bug getting the reduce_limit settingPaul J. Davis
2012-01-24Minor bug getting the reduce_limit settingPaul J. Davis
2012-01-21Bump dependencies for pre7 releaseRobert Newson
2012-01-18Revert use of spawn_monitorRobert Newson
As Filipe correctly points out, we want the parent to die if the child dies.
2012-01-17Bump to chttpd 1.4.2Robert Newson
2012-01-17Merge pull request #91 from cloudant/13133-attachment-replicationRobert Newson
Fix attachment replication BugzID: 13133
2012-01-16Only set the process dictionary if exit was okRobert Newson
2012-01-16Use spawn_monitor to await process completionRobert Newson
BugzID: 13133
2012-01-10Remove superfluous formattingAdam Kocoloski
2012-01-10Remove unused URLsAdam Kocoloski
2012-01-10Bump fabric and chttpd versionsAdam Kocoloski
The fabric bump fixes some corner cases for read repair. The chttpd bump improves error handling for delayed responses.
2011-12-23Make cURL a soft dependencyPaul J. Davis
This makes sure that we only optionally require the same version of cURL that CouchDB does.