summaryrefslogtreecommitdiff
path: root/src/couchdb/couch_file.erl
AgeCommit message (Collapse)Author
2010-12-10Merged revision 1043524 from trunkFilipe David Borba Manana
Calculate and verify MD5 digests outside of a couch_file server This has a significant positive impact on the performance, both for readers and writers, when there are several requests in parallel acessing the same database or view index file. $ node tests/compare_write_and_read.js --wclients 100 --rclients 200 \ -name1 md5_out -name2 trunk \ -url1 http://localhost:5984/ -url2 http://localhost:5985/ \ --duration 120 run 1) http://graphs.mikeal.couchone.com/#/graph/5c859b3e7d1b9bd0488cfe271105130c run 2) http://graphs.mikeal.couchone.com/#/graph/5c859b3e7d1b9bd0488cfe2711051bba Closes COUCHDB-980 git-svn-id: https://svn.apache.org/repos/asf/couchdb/branches/1.1.x@1044283 13f79535-47bb-0310-9956-ffa450edef68
2010-11-11Micro optimization: when reading an iolist, read up to 8Kbs instead of the 4 ↵Filipe David Borba Manana
bytes length prefix only in the first file read operation. For reads of small terms/iolist (up to 8Kbs), this speeds up the whole operation. No impact on larger terms/iolists. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1033790 13f79535-47bb-0310-9956-ffa450edef68
2010-11-10Micro optimization: faster header reads (read the whole block at once, so ↵Filipe David Borba Manana
far headers are always smaller than a block). Read time is basically reduced to half. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1033714 13f79535-47bb-0310-9956-ffa450edef68
2010-09-25Explicitely close the file descriptor. It doesn't hurt.Filipe David Borba Manana
Also, since we open the file in 'raw' mode, I suspect the file descriptor is not closed when the couch_file server dies. From erl -man file: "The raw option allows faster access to a file, because no Erlang process is needed to handle the file." For me, no Erlang process handling the file means that it's likely the file is not closed by any other process. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1001196 13f79535-47bb-0310-9956-ffa450edef68
2010-09-22Removing duplicated code.Filipe David Borba Manana
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1000156 13f79535-47bb-0310-9956-ffa450edef68
2010-08-18COUCHDB-161 - support Range header for attachments.Robert Newson
Attachments are upgraded to support the Range header on compaction. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@986629 13f79535-47bb-0310-9956-ffa450edef68
2010-07-03Fix deletion to move properly to root of storage, to avoid copying files ↵Damien F. Katz
when view and database dirs are on different mounts. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@960153 13f79535-47bb-0310-9956-ffa450edef68
2010-07-03Fixed intermittent problem deleting files on Windows. Added Juhani ↵Damien F. Katz
Ränkimies to THANKS git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@960150 13f79535-47bb-0310-9956-ffa450edef68
2010-07-01Fix for windows fsync, race conditions in attachments.js and the default ↵Damien F. Katz
shell used when launching windows CouchDB. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@959781 13f79535-47bb-0310-9956-ffa450edef68
2010-06-23Authentication caching, to avoid repeated opening and closing of the users ↵Damien F. Katz
database for each request requiring authentication. COUCHDB-807 git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@957314 13f79535-47bb-0310-9956-ffa450edef68
2010-06-23Make file deletions async to avoid pauses during compaction and db deletion. ↵Damien F. Katz
Fixes COUCHDB-780. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@957303 13f79535-47bb-0310-9956-ffa450edef68
2010-06-15Fix for wrong offset when writing headers and converting them to blocks, ↵Damien F. Katz
only triggered when header is larger than 4k. COUCHDB-800. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@955000 13f79535-47bb-0310-9956-ffa450edef68
2010-06-12more efficient header commits. COUCHDB-767Adam Kocoloski
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@954044 13f79535-47bb-0310-9956-ffa450edef68
2010-06-02trailing whitespace pedantryJan Lehnardt
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@950545 13f79535-47bb-0310-9956-ffa450edef68
2010-05-17use O_APPEND to skip some lseeks, COUCHDB-754Adam Kocoloski
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@945110 13f79535-47bb-0310-9956-ffa450edef68
2010-05-14faster implementation of pread_iolist. Closes COUCHDB-762Adam Kocoloski
Why it's faster: - 2 pread() calls (instead of 3) for MD5-prefixed terms - 1 gen_server:call (instead of 3) in all cases, also guarantees that we don't intersperse pread() calls to different parts of the file when we have concurrent readers. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@944336 13f79535-47bb-0310-9956-ffa450edef68
2010-05-04use crypto:md5 when available. thx fdmanana. Closes COUCHDB-757Adam Kocoloski
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@941033 13f79535-47bb-0310-9956-ffa450edef68
2010-04-02Deterministic/synchronous shutdown code.Damien F. Katz
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@930363 13f79535-47bb-0310-9956-ffa450edef68
2010-03-04reverting damien's latest commit until we can figure out why it's causing ↵John Christopher Anderson
failures git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@918834 13f79535-47bb-0310-9956-ffa450edef68
2010-03-04Changed process tree shutdown to be synchronous, to eliminate spurious test ↵Damien F. Katz
failures caused by processes not shutdown fast enough or at the wrong time. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@918805 13f79535-47bb-0310-9956-ffa450edef68
2009-11-21code improvements from tidier. Patch by Kostis Sagonas. COUCHDB-570Adam Kocoloski
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@882903 13f79535-47bb-0310-9956-ffa450edef68
2009-07-23Fixes etap tests for recent updates.Paul Joseph Davis
Thanks Bob Dionne git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@797053 13f79535-47bb-0310-9956-ffa450edef68
2009-07-17MD5 term checking by using leading bit in 32 bit length to indicate MD5 ↵Damien F. Katz
presence. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@795277 13f79535-47bb-0310-9956-ffa450edef68
2009-07-17Deterministic revids, MD5 checking of documents, added tracking of rev when ↵Damien F. Katz
an attachment is edited to allow attachment level replication. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@795232 13f79535-47bb-0310-9956-ffa450edef68
2009-07-14wheeeeeeeeNoah Slater
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@794057 13f79535-47bb-0310-9956-ffa450edef68
2009-07-06trimmed trailing whitespaceNoah Slater
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@791350 13f79535-47bb-0310-9956-ffa450edef68
2009-05-30Fix for couch_file bug found with etap test suiteDamien F. Katz
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@780300 13f79535-47bb-0310-9956-ffa450edef68
2009-05-25Merging new tail append storage into trunk. Upgrades are automatic, once ↵Damien F. Katz
opened by this version old versions of CouchDB will not be able to open the files. As a precaution, you should back-up your production databases before upgrading. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@778485 13f79535-47bb-0310-9956-ffa450edef68
2009-05-06First cut at _changes api. Update the by_id and by_seq indexes to contain ↵Damien F. Katz
update seq numbers and pointers to bodies on disk, for use in the _changes api. This is a new file version, but the code can continue to serve the old 0.9 version without problems, though certain features in the _changes api will not be able to work. Upgrade to new file version (from 1 to 2) by compacting the file. Also fixed bugs with how the stats api tracks open databases. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@772406 13f79535-47bb-0310-9956-ffa450edef68
2009-04-10Fixes for leaked file handles, with test.Damien F. Katz
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@763858 13f79535-47bb-0310-9956-ffa450edef68
2009-04-09Fix to allow couch_file use in tests without the server being started.Damien F. Katz
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@763833 13f79535-47bb-0310-9956-ffa450edef68
2009-04-09Fix for attachment sparseness bug COUCHDB-220 by giving each attachment it's ↵Damien F. Katz
own stream and calling set_min_buffer instead of ensure_buffer. Also fixed spurious couch_file crash messages by putting the statistics decrement code into a seperate monitoring process. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@763816 13f79535-47bb-0310-9956-ffa450edef68
2009-03-16Fix for problem with btree folds getting incorrect reductions after the ↵Damien F. Katz
first value. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@754982 13f79535-47bb-0310-9956-ffa450edef68
2009-03-03stats metrics renamed: os_open_files -> open_os_files, database_changes -> ↵Jan Lehnardt
database_writes git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@749583 13f79535-47bb-0310-9956-ffa450edef68
2009-02-26Fix and test for file descriptor leaksDamien F. Katz
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@748299 13f79535-47bb-0310-9956-ffa450edef68
2009-02-17Put file ref_counting into it's own module, to make the speed of opening an ↵Damien F. Katz
already open database faster (by not waiting on file pending operations, particularly fsync). git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@745076 13f79535-47bb-0310-9956-ffa450edef68
2009-01-09Added support so clients can detect if a server has potentially lost commits ↵Damien F. Katz
after multiple updates, like during bulk imports and so the replicator can detect lost commits on remote replications. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@733174 13f79535-47bb-0310-9956-ffa450edef68
2008-12-22Fix for leaked file handles when not explicitly closed, added file stats ↵Damien F. Katz
code for checking for leaked file handles, and some refactoring of the view api. git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@728764 13f79535-47bb-0310-9956-ffa450edef68
2008-12-18fix for crash of couch_server when database non-existantDamien F. Katz
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@727832 13f79535-47bb-0310-9956-ffa450edef68
2008-12-16do not fail on heavily loaded systems. wait forever for syscalls to returnJan Lehnardt
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@727050 13f79535-47bb-0310-9956-ffa450edef68
2008-12-12modifications to view server to keep the file descriptor open for the life ↵Damien F. Katz
of the view group. git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@725909 13f79535-47bb-0310-9956-ffa450edef68
2008-10-02HTTPd refactoring. Moved most code out of couch_httpd into associated ↵Damien F. Katz
modules (couch_httpd_view, couch_httpd_db, couch_httpd_misc_handlers). Also a fix to removed previous doc_by_seq index entries on compaction retry. git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@701173 13f79535-47bb-0310-9956-ffa450edef68
2008-08-24fix for occasional crash when attempting to open a database on a heavily ↵Damien F. Katz
loaded server git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@688540 13f79535-47bb-0310-9956-ffa450edef68
2008-08-22Fix for couch_server process crash when databases that don't exist are ↵Damien F. Katz
attempted to be opened. Removed old comments and fixed the AllowRemoteRestart testing option. git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@688129 13f79535-47bb-0310-9956-ffa450edef68
2008-08-20Merge runtimeconfig branch back into trunkJan Lehnardt
git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@687336 13f79535-47bb-0310-9956-ffa450edef68
2008-08-05Fixed bug where we attempted to match a linked port to the current fd, but ↵Damien F. Katz
fail because the fd structure is more than the linked port. git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@682792 13f79535-47bb-0310-9956-ffa450edef68
2008-08-05Added concurrent open db limit and a LRU cache for closing old databases ↵Damien F. Katz
when limit reached (configurable via MaxDbsOpen var in couch.ini). Refactored db update code in couch_db.erl into couch_db_updater.erl. git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@682560 13f79535-47bb-0310-9956-ffa450edef68
2008-07-16Increase more filesystem layer timeouts. Patch by randall <dizzot> leeds ↵Jan Lehnardt
<atface> gmail git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@677373 13f79535-47bb-0310-9956-ffa450edef68
2008-07-10Wait for the OS to flush our newly written data to disk without timing out ↵Jan Lehnardt
too quickly. git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@675780 13f79535-47bb-0310-9956-ffa450edef68
2008-06-12fix for problem when view index header data exceeds 2k.Damien F. Katz
git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@667236 13f79535-47bb-0310-9956-ffa450edef68
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-20 06:02:39 +0000