How to Catch All Http Traffic (Local Proxy)

How to catch all HTTP traffic (local proxy)

There's an HTTP Proxy in WEBrick (part of Ruby stdlib) and here's an implementation example.

If you like living on the edge there's also em-proxy by Ilya Grigorik.

This post by Ilya implies that it does seem to need some tweaking to solve your problem.

How do I monitor all incoming http requests?

Guys found the perfect way to monitor ALL traffic that is flowing locally between requests from my machine to my machine:

1. Install Wireshark

2. When you need to capture traffic that is flowing from a localhost to a localhost then you will struggle to use wireshark as this only monitors incoming traffic on the network card. The way to do this is to add a route to windows that will force all traffic through a gateway and this be captured on the network interface.

   To do this, add a route with <ip address> <gateway>:

    cmd> route add 192.168.20.30 192.168.20.1
   

3. Then run a capture on wireshark (make sure you select the interface that has bytes flowing through it) Then filter.

The newly added routes will come up in black. (as they are local addresses)

How to capture HTTP traffic from a Lua process on Windows?

Well you have to find out how to set proxy for Lua executable.

Have in mind, though, that even if set up the Lua executable to respect WinHTTP/WinINET proxy settings these settings are per user and the chances are your service does not run under your user account. So the first thing to try is changing the WinINET proxy for the service user account to Fiddler and restarting the service.

You can also configure Fiddler as a reverse proxy and capture the inbound traffic to that local process that gets the GET requests. That might be simpler to do. Here is a how to - http://docs.telerik.com/fiddler/Configure-Fiddler/Tasks/UseFiddlerAsReverseProxy

how do I Sniff/Listen HTTP on my local machine?

There is no way to use TIdHTTPServer or TIdHTTPProxyServer to sniff HTTP traffic without altering browser/client settings to connect to it. That is what Fiddler does - alters WinInet's proxy settings temporarily (IE uses WinInet, so it picks up the alterations transparently). What you are asking for requires lower-level sniffing of the network subsystem, such as what WinPCap/Ethereal does.

How to get a web server to send outbound http requests through local fiddler proxy?

Short answer: you can't...

...unless your web application is written to open a connection to a Proxy server and route requests through that connection (e.g. connect to a remote proxy, and then send requests through it).

Typically what developers do is just dump the Web Request/Response to a debug file to inspect during development (or to debug on a live server, by enabling it with a flag at runtime).

Fiddler is a "proxy" service/server. When you are using it normally to debug browser requests, your Browser is configured to connect to a Proxy server. That is, it will send all web requests through your fiddler's local server (I think it's localhost:8888 if i remember from my Windows days of using Fiddler) which in turn makes a connection to your local web server that you are debugging.

You can read more about Proxies at Wikipedia.

In that picture above, your local web server would be Alice. Meaning, Alice would need to be configured to connect to a proxy server and then make web requests through it.

EDIT:

(for the "I really need this" crowd)

If you really want to modify your web server to send requests through a proxy, there are a few Go packages already written to help you. GoProxy is one such package.

How to listen to GET and POST requests for all connections using GO

You can either easily build a proxy with httputil - example here.

Or you might want to use gopacket as described here.

---

Related Topics