| Age | Commit message (Collapse) | Author |
|---|
|
|
|
In case the http client loses connection, it has to clear
it's timeout or the reactor will be left in a dirty state
Fixing this solves a problem with some of the tests in Soledad
that were trying to run on a dirty reactor
|
|
|
|
|
|
Two test cases were broken and were implemented here:
The first was that HTTPClient should share the connection between
clients if a pool was not passed explicitly. If you initialize an
HTTPClient without a pool, it will reuse a pool created on the class.
The second was that you should be able to pass to the HTTPCLient a pool
on initialization. Added that possibility and fixed the tests accordingly
|
|
|
|
The connectTimeout parameter of twisted.web.client.Agent only acts on the
connection setup, and the Agent will wait forever for incoming data after the
connection has been established.
This commit adds a timeout for the connection, and will cancel the deferred if
the result has not been received after a certain number of seconds.
|
|
The ability to close cached connections is needed in order to have a clean
reactor when the program ends.
|
|
Altough we specify maxPersistentPerHost, Twisted won't stop opening
connections after that. This limit is used just to keep the size of
persistent connections pool under control. Additional connections will be
made as non persistent. So, if we ask 10000 requests, it will open 10000
connections immediately and leave 10 open after all finished.
For checking this behavior, see getConnection from Twisted source:
http://twistedmatrix.com/trac/browser/tags/releases/twisted-15.2.1/twisted/web/client.py#L1203
I tested this by using http_target from soledad without a local database to
download all encrypted docs from one account with 1700 of them. The
program just hangs and crashes with 1000+ connections and "Too many
files open" warnings.
With this fix, it was able to download normally, respecting the
maxPersistentPerHost as a limiter. :)
|
|
As @meskio pointed out, some cases could need a dedicated pool with
different parameters.
This is a suggested implementation where the pool is reused by default,
creating a dedicated one just if needed/asked. This way we ensure that
resources are under control and special cases are still handled.
|
|
While testing the way that its implemented now, I found out that no check
is being made on certificate attributes against the host.
I found this simple way of creating a BrowserLikePolicyForHTTPS using
a self signed cert and it worked on my test. I used test_https from
Soledad for checking this (which we are fixing on another branch).
Also, we don't want to depend on twisted for other things than
leap.common.http.
|
|
Implements an HTTP client the twisted way, with a focus on pinning the
SSL certs.
* Related: #6506
|
