To facilitate end-to-end testing for such scenarios, I architected a proxy infrastructure; A stripped-down version of which was a Proxy.py – lightweight HTTP proxy server in Python.
❯ pip install --upgrade proxy.py
❯ pip install git+https://github.com/abhinavsingh/proxy.py.git@master
❯ docker run -it -p 8899:8899 --rm abhinavsingh/proxy.py:latest
❯ git clone https://github.com/abhinavsingh/proxy.py.git
❯ cd proxy.py
❯ make container
❯ docker run -it -p 8899:8899 --rm abhinavsingh/proxy.py:latest
❯ brew install https://raw.githubusercontent.com/abhinavsingh/proxy.py/develop/helper/homebrew/stable/proxy.rb
❯ brew install https://raw.githubusercontent.com/abhinavsingh/proxy.py/develop/helper/homebrew/develop/proxy.rb
When proxy.py
is installed using pip
, an executable named proxy
is placed under your $PATH
.
Simply type proxy
on command line to start it with default configuration.
❯ proxy
...[redacted]... - Loaded plugin proxy.http_proxy.HttpProxyPlugin
...[redacted]... - Starting 8 workers
...[redacted]... - Started server on ::1:8899
All the logs above are INFO
level logs, default --log-level
for proxy.py
.
❯ proxy --log-level d
...[redacted]... - Open file descriptor soft limit set to 1024
...[redacted]... - Loaded plugin proxy.http_proxy.HttpProxyPlugin
...[redacted]... - Started 8 workers
...[redacted]... - Started server on ::1:8899
If you are trying to run proxy.py
from source code, there is no binary file named proxy
in the source code.
To override input flags, start docker image as follows. For example, to check proxy.py
the version within Docker image:
Add support for short links in your favorite browsers / applications.
❯ proxy
--plugins proxy.plugin.ShortLinkPlugin
Now you can speed up your daily browsing experience by visiting your favorite website using single character domain names :). This works across all browsers.
Modifies POST request body before sending request to upstream server.
❯ proxy
--plugins proxy.plugin.ModifyPostDataPlugin
By default plugin replaces POST body content with hardcoded b'{"key": "modified"}'
and enforced Content-Type: application/json
.
Verify the same using curl -x localhost:8899 -d '{"key": "value"}' http://httpbin.org/post
{
"args": {},
"data": "{"key": "modified"}",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Content-Length": "19",
"Content-Type": "application/json",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"json": {
"key": "modified"
},
"origin": "1.2.3.4, 5.6.7.8",
"url": "https://httpbin.org/post"
}
Mock responses for your server REST API. Use to test and develop client side applications without need of an actual upstream REST API server.
❯ proxy
--plugins proxy.plugin.ProposedRestApiPlugin
Verify mock API response using curl -x localhost:8899 http://api.example.com/v1/users/
{"count": 2, "next": null, "previous": null, "results": [{"email": "[email protected]", "groups": [], "url": "api.example.com/v1/users/1/", "username": "admin"}, {"email": "[email protected]", "groups": [], "url": "api.example.com/v1/users/2/", "username": "admin"}]}
2019-09-27 12:44:02,212 - INFO - pid:7077 - access_log:1210 - ::1:64792 - GET None:None/v1/users/ - None None - 0 byte
Access log shows None:None
as server ip:port
. None
simply means that the server connection was never made, since response was returned by our plugin.
Now modify ProposedRestApiPlugin
to returns REST API mock responses as expected by your clients.
Redirects all incoming http
requests to custom web server. By default, it redirects client requests to inbuilt web server, also running on 8899
port.
❯ proxy
--enable-web-server
--plugins proxy.plugin.RedirectToCustomServerPlugin
Above 404
response was returned from proxy.py
web server.
Verify the same by inspecting the logs for proxy.py
. Along with the proxy request log, you must also see a http web server request log.
Drops traffic by inspecting upstream host. By default, plugin drops traffic for google.com
and www.google.com
.
❯ proxy
--plugins proxy.plugin.FilterByUpstreamHostPlugin
... [redacted] ...
< HTTP/1.1 418 I'm a tea pot
< Proxy-agent: proxy.py v1.0.0
* no chunk, no close, no size. Assume close to signal end
<
* Closing connection 0
Above 418 I'm a tea pot
is sent by our plugin.
2019-09-24 19:21:37,893 - ERROR - pid:50074 - handle_readables:1347 - HttpProtocolException type raised
Traceback (most recent call last):
... [redacted] ...
2019-09-24 19:21:37,897 - INFO - pid:50074 - access_log:1157 - ::1:49911 - GET None:None/ - None None - 0 bytes
Caches Upstream Server Responses.
❯ proxy
--plugins proxy.plugin.CacheResponsesPlugin
... [redacted] ...
< HTTP/1.1 200 OK
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Date: Wed, 25 Sep 2019 02:24:25 GMT
< Referrer-Policy: no-referrer-when-downgrade
< Server: nginx
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-XSS-Protection: 1; mode=block
< Content-Length: 202
< Connection: keep-alive
<
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"origin": "1.2.3.4, 5.6.7.8",
"url": "https://httpbin.org/get"
}
* Connection #0 to host localhost left intact
... [redacted] ... - GET httpbin.org:80/get - 200 OK - 556 bytes
... [redacted] ... - Cached response at /var/folders/k9/x93q0_xn1ls9zy76m2mf2k_00000gn/T/httpbin.org-1569378301.407512.txt
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Wed, 25 Sep 2019 02:24:25 GMT
Referrer-Policy: no-referrer-when-downgrade
Server: nginx
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Content-Length: 202
Connection: keep-alive
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"origin": "1.2.3.4, 5.6.7.8",
"url": "https://httpbin.org/get"
}
ManInTheMiddlePlugin
Modifies upstream server responses.
Start proxy.py
as:
❯ proxy
--plugins proxy.plugin.ManInTheMiddlePlugin
Verify using curl -v -x localhost:8899 http://google.com
:
... [redacted] ...
< HTTP/1.1 200 OK
< Content-Length: 28
<
* Connection #0 to host localhost left intact
Hello from man in the middle
Response body Hello from man in the middle
is sent by our plugin.
Forward incoming proxy requests to a set of upstream proxy servers.
By default, ProxyPoolPlugin
is hard-coded to use localhost:9000
and localhost:9001
as upstream proxy server.
Let’s start upstream proxies first.
Start proxy.py
on port 9000
and 9001
❯ proxy --port 9000
❯ proxy --port 9001
Now, start proxy.py
with ProxyPoolPlugin
(on default 8899
port):
❯ proxy
--plugins proxy.plugin.ProxyPoolPlugin
Make a curl request via 8899
proxy:
curl -v -x localhost:8899 http://httpbin.org/get
Verify that 8899
proxy forwards requests to upstream proxies by checking respective logs.
Extend in-built Web Server to add Reverse Proxy capabilities.
Start proxy.py
as:
❯ proxy --enable-web-server
--plugins proxy.plugin.ReverseProxyPlugin
With default configuration, ReverseProxyPlugin
plugin is equivalent to following Nginx
config:
location /get {
proxy_pass http://httpbin.org/get
}
Verify using curl -v localhost:8899/get
:
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "localhost",
"User-Agent": "curl/7.64.1"
},
"origin": "1.2.3.4, 5.6.7.8",
"url": "https://localhost/get"
}
Demonstrates inbuilt web server routing using plugin.
Start proxy.py
as:
❯ proxy --enable-web-server
--plugins proxy.plugin.WebServerPlugin
Verify using curl -v localhost:8899/http-route-example
, should return:
HTTP route response
When using multiple plugins, depending upon plugin functionality, it might be worth considering the order in which plugins are passed on the command line.
Plugins are called in the same order as they are passed. Example, say we are using both FilterByUpstreamHostPlugin
andRedirectToCustomServerPlugin
. Idea is to drop all incoming http
requests for google.com
and www.google.com
and redirect other http
requests to our inbuilt web server.
Hence, in this scenario it is important to use FilterByUpstreamHostPlugin
before RedirectToCustomServerPlugin
. If we enable RedirectToCustomServerPlugin
before FilterByUpstreamHostPlugin
, google
requests will also get redirected to inbuilt web server, instead of being dropped.
End-to-End Encryption
By default, proxy.py
uses http
protocol for communication with clients e.g. curl
, browser
. For enabling end-to-end encrypting using tls
/ https
first generate certificates:
make https-certificates
Start proxy.py
as:
❯ proxy
--cert-file https-cert.pem
--key-file https-key.pem
Verify using curl -x https://localhost:8899 --proxy-cacert https-cert.pem https://httpbin.org/get
:
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"origin": "1.2.3.4, 5.6.7.8",
"url": "https://httpbin.org/get"
}
By default, proxy.py
will not decrypt https
traffic between client and server. To enable TLS interception first generate root CA certificates:
❯ make ca-certificates
Lets also enable CacheResponsePlugin
so that we can verify decrypted response from the server. Start proxy.py
as:
❯ proxy
--plugins proxy.plugin.CacheResponsesPlugin
--ca-key-file ca-key.pem
--ca-cert-file ca-cert.pem
--ca-signing-key-file ca-signing-key.pem
:note: MacOS users also need to pass explicit CA file path needed for validation of peer certificates. See –ca-file flag.
Verify TLS interception using curl
❯ curl -v -x localhost:8899 --cacert ca-cert.pem https://httpbin.org/get
* issuer: C=US; ST=CA; L=SanFrancisco; O=proxy.py; OU=CA; CN=Proxy PY CA; [email protected]
* SSL certificate verify ok.
> GET /get HTTP/1.1
... [redacted] ...
< Connection: keep-alive
<
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"origin": "1.2.3.4, 5.6.7.8",
"url": "https://httpbin.org/get"
}
The issuer
line confirms that response was intercepted.
Also verify the contents of cached response file. Get path to the cache file from proxy.py
logs.
❯ cat /path/to/your/tmp/directory/httpbin.org-1569452863.924174.txt
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Wed, 25 Sep 2019 23:07:05 GMT
Referrer-Policy: no-referrer-when-downgrade
Server: nginx
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Content-Length: 202
Connection: keep-alive
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"origin": "1.2.3.4, 5.6.7.8",
"url": "https://httpbin.org/get"
}
Viola!!! If you remove CA flags, encrypted data will be found in the cached file instead of plain text.
Now use CA flags with other plugin examples to see them work with https
traffic.
Proxy Over SSH Tunnel
Requires paramiko
to work. See requirements-tunnel.txt
|
+------------+ | +----------+
| LOCAL | | | REMOTE |
| HOST | <== SSH ==== :8900 == | SERVER |
+------------+ | +----------+
:8899 proxy.py |
|
FIREWALL
(allow tcp/22)
Proxy HTTP(s) requests made on a remote
server through proxy.py
server running on localhost
.
- Requested
remote
port is forwarded over the SSH connection. proxy.py
running on thelocalhost
handles and responds toremote
proxy requests.
localhost
MUST have SSH access to theremote
serverremote
server MUST be configured to proxy HTTP(s) requests through the forwarded port number e.g.:8900
.remote
andlocalhost
ports CAN be same e.g.:8899
.:8900
is chosen in ascii art for differentiation purposes.
Start proxy.py
as:
❯ # On localhost
❯ proxy --enable-tunnel
--tunnel-username username
--tunnel-hostname ip.address.or.domain.name
--tunnel-port 22
--tunnel-remote-host 127.0.0.1
--tunnel-remote-port 8899
Make a HTTP proxy request on remote
server and verify that response contains public IP address of localhost
as origin:
❯ # On remote
❯ curl -x 127.0.0.1:8899 http://httpbin.org/get
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"origin": "x.x.x.x, y.y.y.y",
"url": "https://httpbin.org/get"
}
Also, verify that proxy.py
logs on localhost
contains remote
IP as client IP.
access_log:328 - remote:52067 - GET httpbin.org:80
|
+------------+ | +----------+
| LOCAL | | | REMOTE |
| HOST | === SSH =====> | SERVER |
+------------+ | +----------+
| :8899 proxy.py
|
FIREWALL
(allow tcp/22)
Start proxy.py
in embedded mode with default configuration by using proxy.main
method. Example:
import proxy
if __name__ == '__main__':
proxy.main()
Customize startup flags by passing list of input arguments:
import proxy
if __name__ == '__main__':
proxy.main([
'--hostname', '::1',
'--port', '8899'
])
or, customize startup flags by passing them as kwargs:
import ipaddress
import proxy
if __name__ == '__main__':
proxy.main(
hostname=ipaddress.IPv6Address('::1'),
port=8899
)
Note that:
- Calling
main
is simply equivalent to startingproxy.py
from command line. main
will block untilproxy.py
shuts down
Non-blocking Mode
Start proxy.py
in non-blocking embedded mode with default configuration by using start
method: Example:
import proxy
if __name__ == '__main__':
with proxy.start([]):
# ... your logic here ...
Note that:
start
is similar tomain
, exceptstart
won’t block.start
is a context manager. It will startproxy.py
when called and will shut it down once scope ends.- Just like
main
, startup flags withstart
method can be customized by either passing flags as list of input arguments e.g.start(['--port', '8899'])
or by using passing flags as kwargs e.g.start(port=8899)
.
To setup and teardown proxy.py
for your Python unittest classes, simply use proxy.TestCase
instead of unittest.TestCase
. Example:
import proxy
class TestProxyPyEmbedded(proxy.TestCase):
def test_my_application_with_proxy(self) -> None:
self.assertTrue(True)
Note that:
proxy.TestCase
overridesunittest.TestCase.run()
method to setup and teardownproxy.py
.proxy.py
server will listen on a random available port on the system. This random port is available asself.PROXY_PORT
within your test cases.- Only a single worker is started by default (
--num-workers 1
) for faster setup and teardown. - Most importantly,
proxy.TestCase
also ensuresproxy.py
server is up and running before proceeding with execution of tests. By default,proxy.TestCase
will wait for10 seconds
forproxy.py
server to start, upon failure aTimeoutError
exception will be raised.
To override default startup flags, define a PROXY_PY_STARTUP_FLAGS
variable in your test class. Example:
class TestProxyPyEmbedded(TestCase):
PROXY_PY_STARTUP_FLAGS = [
'--num-workers', '1',
'--enable-web-server',
]
def test_my_application_with_proxy(self) -> None:
self.assertTrue(True)
See test_embed.py for full working example.
If for some reasons you are unable to directly use proxy.TestCase
, then simply override unittest.TestCase.run
yourself to setup and teardown proxy.py
. Example:
import unittest
import proxy
class TestProxyPyEmbedded(unittest.TestCase):
def test_my_application_with_proxy(self) -> None:
self.assertTrue(True)
def run(self, result: Optional[unittest.TestResult] = None) -> Any:
with proxy.start([
'--num-workers', '1',
'--port', '... random port ...']):
super().run(result)
or simply setup / teardown proxy.py
within setUpClass
and teardownClass
class methods.
Plugin Developer and Contributor Guide
As you might have guessed by now, in proxy.py
everything is a plugin.
- We enabled proxy server plugins using
--plugins
flag. All the plugin examples were implementingHttpProxyBasePlugin
. See documentation of HttpProxyBasePlugin for available lifecycle hooks. UseHttpProxyBasePlugin
to modify behavior of http(s) proxy protocol between client and upstream server. Example, FilterByUpstreamHostPlugin. - We also enabled inbuilt web server using
--enable-web-server
. Inbuilt web server implementsHttpProtocolHandlerPlugin
plugin. See documentation of HttpProtocolHandlerPlugin for available lifecycle hooks. UseHttpProtocolHandlerPlugin
to add new features for http(s) clients. Example, HttpWebServerPlugin. - There also is a
--disable-http-proxy
flag. It disables inbuilt proxy server. Use this flag with--enable-web-server
flag to runproxy.py
as a programmable http(s) server. HttpProxyPlugin also implementsHttpProtocolHandlerPlugin
.
- HttpProtocolHandler thread is started with the accepted TcpClientConnection.
HttpProtocolHandler
is responsible for parsing incoming client request and invokingHttpProtocolHandlerPlugin
lifecycle hooks. HttpProxyPlugin
which implementsHttpProtocolHandlerPlugin
also has its own plugin mechanism. Its responsibility is to establish connection between client and upstream TcpServerConnection and invokeHttpProxyBasePlugin
lifecycle hooks.HttpProtocolHandler
threads are started by Acceptor processes.--num-workers
Acceptor
processes are started by AcceptorPool on start-up.AcceptorPool
listens on server socket and pass the handler toAcceptor
processes. Workers are responsible for accepting new client connections and startingHttpProtocolHandler
thread.
Contributors must start proxy.py
from source to verify and develop new features / fixes.
See Run proxy.py from command line using repo source for details.
Pre-commit hook ensures lint checking and tests execution.
cd /path/to/proxy.py
ln -s $(PWD)/git-pre-commit .git/hooks/pre-commit
Every pull request is tested using GitHub actions.
See GitHub workflow for list of tests.
Attempts to create an IPv4 connection, then IPv6 and finally a dual stack connection to provided address.
>>> conn = new_socket_connection(('httpbin.org', 80))
>>> ...[ use connection ]...
>>> conn.close()
socket_connection
is a convenient decorator + context manager around new_socket_connection
which ensures conn.close
is implicit.
As a context manager:
>>> with socket_connection(('httpbin.org', 80)) as conn:
>>> ... [ use connection ] ...
As a decorator:
>>> @socket_connection(('httpbin.org', 80))
>>> def my_api_call(conn, *args, **kwargs):
>>> ... [ use connection ] ...
>>> build_http_request(b'GET', b'/')
b'GET / HTTP/1.1rnrn'
>>>
Generate HTTP GET request with headers
>>> build_http_request(b'GET', b'/',
headers={b'Connection': b'close'})
b'GET / HTTP/1.1rnConnection: closernrn'
>>>
>>> import json
>>> build_http_request(b'POST', b'/form',
headers={b'Content-type': b'application/json'},
body=proxy.bytes_(json.dumps({'email': '[email protected]'})))
b'POST /form HTTP/1.1rnContent-type: application/jsonrnrn{"email": "[email protected]"}'
build_http_response(
status_code: int,
protocol_version: bytes = HTTP_1_1,
reason: Optional[bytes] = None,
headers: Optional[Dict[bytes, bytes]] = None,
body: Optional[bytes] = None) -> bytes
gen_private_key(
key_path: str,
password: str,
bits: int = 2048,
timeout: int = 10) -> bool
gen_public_key(
public_key_path: str,
private_key_path: str,
private_key_password: str,
subject: str,
alt_subj_names: Optional[List[str]] = None,
extended_key_usage: Optional[str] = None,
validity_in_days: int = 365,
timeout: int = 10) -> bool
remove_passphrase(
key_in_path: str,
password: str,
key_out_path: str,
timeout: int = 10) -> bool
gen_csr(
csr_path: str,
key_path: str,
password: str,
crt_path: str,
timeout: int = 10) -> bool
sign_csr(
csr_path: str,
crt_path: str,
ca_key_path: str,
ca_key_password: str,
ca_crt_path: str,
serial: str,
alt_subj_names: Optional[List[str]] = None,
extended_key_usage: Optional[str] = None,
validity_in_days: int = 365,
timeout: int = 10) -> bool
See pki.py and test_pki.py for usage examples
CLI Usage
Use proxy.common.pki
module for:
- Generation of public and private keys
- Generating CSR requests
- Signing CSR requests using custom CA.
python -m proxy.common.pki -h
usage: pki.py [-h] [--password PASSWORD] [--private-key-path PRIVATE_KEY_PATH]
[--public-key-path PUBLIC_KEY_PATH] [--subject SUBJECT]
action
proxy.py v2.1.2 : PKI Utility
positional arguments:
action Valid actions: remove_passphrase, gen_private_key,
gen_public_key, gen_csr, sign_csr
optional arguments:
-h, --help show this help message and exit
--password PASSWORD Password to use for encryption. Default: proxy.py
--private-key-path PRIVATE_KEY_PATH
Private key path
--public-key-path PUBLIC_KEY_PATH
Public key path
--subject SUBJECT Subject to use for public key generation. Default:
/CN=example.com
Browse through internal class hierarchy and documentation using pydoc3
. Example:
❯ pydoc3 proxy
PACKAGE CONTENTS
__main__
common (package)
core (package)
http (package)
main
FILE
/Users/abhinav/Dev/proxy.py/proxy/__init__.py
Pre v2.x, proxy.py
used to spawn new threads for handling client requests.
Starting v2.x, proxy.py
added support for threadless execution of client requests using asyncio
.
In future, threadless execution will be the default mode.
Till then if you are interested in trying it out, start proxy.py
with --threadless
flag.
proxy.py
is strictly typed and uses Python typing
annotations. Example:
>>> my_strings : List[str] = []
>>> #############^^^^^^^^^#####
Hence a Python version that understands typing annotations is required. Make sure you are using Python 3.6+
.
Verify the version before running proxy.py
:
❯ python --version
All typing
annotations can be replaced with comment-only
annotations. Example:
>>> my_strings = [] # List[str]
>>> ################^^^^^^^^^^^
It will enable proxy.py
to run on Python pre-3.6
, even on 2.7
. However, as all future versions of Python will support typing
annotations, this has not been considered.
Make sure plugin modules are discoverable by adding them to PYTHONPATH
. Example:
PYTHONPATH=/path/to/my/app proxy --plugins my_app.proxyPlugin
...[redacted]... - Loaded plugin proxy.HttpProxyPlugin
...[redacted]... - Loaded plugin my_app.proxyPlugin
OR, simply pass fully-qualified path as parameter, e.g.
proxy --plugins /path/to/my/app/my_app.proxyPlugin
Make sure proxy.py
is listening on correct network interface. Try following flags:
- For IPv6
--hostname ::
- For IPv4
--hostname 0.0.0.0
Most likely it’s a browser integration issue with system keychain.
- First verify that basic auth is working using
curl
curl -v -x username:password@localhost:8899 https://httpbin.org/get
- See this thread for further details.
It’s a compatibility issue with vpnkit
.
See moby/vpnkit exhausts docker resources and Connection refused: The proxy could not connect for some background.
A starter fluentd.conf template is available.
- Copy this configuration file as
proxy.py.conf
under/etc/google-fluentd/config.d/
- Update
path
field to log file path as used with--log-file
flag. By default/tmp/proxy.log
path is tailed. - Reload
google-fluentd
:sudo service google-fluentd restart
Now proxy.py
logs can be browsed using GCE log viewer.
proxy.py
is made to handle thousands of connections per second without any socket leaks.
- Make use of
--open-file-limit
flag to customizeulimit -n
. - Make sure to adjust
--backlog
flag for higher concurrency.
If nothing helps, open an issue with requests per second
sent and output of following debug script:
❯ ./helper/monitor_open_files.sh <proxy-py-pid>
Sometimes you may see None:None
in access logs. It simply means that an upstream server connection was never established i.e. upstream_host=None
, upstream_port=None
.
There can be several reasons for no upstream connection, few obvious ones include:
- Client established a connection but never completed the request.
- A plugin returned a response prematurely, avoiding connection to upstream server.
❯ proxy -h
usage: proxy [-h] [--backlog BACKLOG] [--basic-auth BASIC_AUTH]
[--ca-key-file CA_KEY_FILE] [--ca-cert-dir CA_CERT_DIR]
[--ca-cert-file CA_CERT_FILE]
[--ca-signing-key-file CA_SIGNING_KEY_FILE]
[--cert-file CERT_FILE]
[--client-recvbuf-size CLIENT_RECVBUF_SIZE]
[--devtools-ws-path DEVTOOLS_WS_PATH]
[--disable-headers DISABLE_HEADERS] [--disable-http-proxy]
[--enable-dashboard] [--enable-devtools] [--enable-events]
[--enable-static-server] [--enable-web-server]
[--hostname HOSTNAME] [--key-file KEY_FILE]
[--log-level LOG_LEVEL] [--log-file LOG_FILE]
[--log-format LOG_FORMAT] [--num-workers NUM_WORKERS]
[--open-file-limit OPEN_FILE_LIMIT] [--pac-file PAC_FILE]
[--pac-file-url-path PAC_FILE_URL_PATH]
[--pid-file PID_FILE] [--plugins PLUGINS] [--port PORT]
[--server-recvbuf-size SERVER_RECVBUF_SIZE]
[--static-server-dir STATIC_SERVER_DIR] [--threadless]
[--timeout TIMEOUT] [--version]
proxy.py v2.1.2
optional arguments:
-h, --help show this help message and exit
--backlog BACKLOG Default: 100. Maximum number of pending connections to
proxy server
--basic-auth BASIC_AUTH
Default: No authentication. Specify colon separated
user:password to enable basic authentication.
--ca-key-file CA_KEY_FILE
Default: None. CA key to use for signing dynamically
generated HTTPS certificates. If used, must also pass
--ca-cert-file and --ca-signing-key-file
--ca-cert-dir CA_CERT_DIR
Default: ~/.proxy.py. Directory to store dynamically
generated certificates. Also see --ca-key-file, --ca-
cert-file and --ca-signing-key-file
--ca-cert-file CA_CERT_FILE
Default: None. Signing certificate to use for signing
dynamically generated HTTPS certificates. If used,
must also pass --ca-key-file and --ca-signing-key-file
--ca-file CA_FILE Default: None. Provide path to custom CA file for peer
certificate validation. Specially useful on MacOS.
--ca-signing-key-file CA_SIGNING_KEY_FILE
Default: None. CA signing key to use for dynamic
generation of HTTPS certificates. If used, must also
pass --ca-key-file and --ca-cert-file
--cert-file CERT_FILE
Default: None. Server certificate to enable end-to-end
TLS encryption with clients. If used, must also pass
--key-file.
--client-recvbuf-size CLIENT_RECVBUF_SIZE
Default: 1 MB. Maximum amount of data received from
the client in a single recv() operation. Bump this
value for faster uploads at the expense of increased
RAM.
--devtools-ws-path DEVTOOLS_WS_PATH
Default: /devtools. Only applicable if --enable-
devtools is used.
--disable-headers DISABLE_HEADERS
Default: None. Comma separated list of headers to
remove before dispatching client request to upstream
server.
--disable-http-proxy Default: False. Whether to disable
proxy.HttpProxyPlugin.
--enable-dashboard Default: False. Enables proxy.py dashboard.
--enable-devtools Default: False. Enables integration with Chrome
Devtool Frontend. Also see --devtools-ws-path.
--enable-events Default: False. Enables core to dispatch lifecycle
events. Plugins can be used to subscribe for core
events.
--enable-static-server
Default: False. Enable inbuilt static file server.
Optionally, also use --static-server-dir to serve
static content from custom directory. By default,
static file server serves out of installed proxy.py
python module folder.
--enable-web-server Default: False. Whether to enable
proxy.HttpWebServerPlugin.
--hostname HOSTNAME Default: ::1. Server IP address.
--key-file KEY_FILE Default: None. Server key file to enable end-to-end
TLS encryption with clients. If used, must also pass
--cert-file.
--log-level LOG_LEVEL
Valid options: DEBUG, INFO (default), WARNING, ERROR,
CRITICAL. Both upper and lowercase values are allowed.
You may also simply use the leading character e.g.
--log-level d
--log-file LOG_FILE Default: sys.stdout. Log file destination.
--log-format LOG_FORMAT
Log format for Python logger.
--num-workers NUM_WORKERS
Defaults to number of CPU cores.
--open-file-limit OPEN_FILE_LIMIT
Default: 1024. Maximum number of files (TCP
connections) that proxy.py can open concurrently.
--pac-file PAC_FILE A file (Proxy Auto Configuration) or string to serve
when the server receives a direct file request. Using
this option enables proxy.HttpWebServerPlugin.
--pac-file-url-path PAC_FILE_URL_PATH
Default: /. Web server path to serve the PAC file.
--pid-file PID_FILE Default: None. Save parent process ID to a file.
--plugins PLUGINS Comma separated plugins
--port PORT Default: 8899. Server port.
--server-recvbuf-size SERVER_RECVBUF_SIZE
Default: 1 MB. Maximum amount of data received from
the server in a single recv() operation. Bump this
value for faster downloads at the expense of increased
RAM.
--static-server-dir STATIC_SERVER_DIR
Default: "public" folder in directory where proxy.py
is placed. This option is only applicable when static
server is also enabled. See --enable-static-server.
--threadless Default: False. When disabled a new thread is spawned
to handle each client connection.
--timeout TIMEOUT Default: 10. Number of seconds after which an inactive
connection must be dropped. Inactivity is defined by
no data sent or received by the client.
--version, -v Prints proxy.py version.
Proxy.py not working? Report at:
https://github.com/abhinavsingh/proxy.py/issues/new
- No longer
a single file module. - Added support for threadless execution.
- Added dashboard app.
- Added support for unit testing.
Python3
only.- Deprecated support for
Python 2.x.
- Deprecated support for
- Added support multi core accept.
- Added plugin support.
- Single file.
- Single threaded server.
For detailed changelog refer to release PRs or commit history.
Blog post: https://abhinavsingh.com/proxy-py-a-lightweight-single-file-http-proxy-server-in-python/
Github page: https://github.com/abhinavsingh/proxy.py