Using Web Debugging Proxies
Thankfully, all major modern browsers provide debugging tools that allow you to inspect network traffic, and 3rd party tools like Fiddler and Charles not only allow you to see network requests, but offer expanded capabilities to interact with your site.
We'll explore both types of tools.
Browser-Based Traffic Sniffing
As I mentioned, every major browser has built-in debugging tools. These include:
- Internet Explorer's F12 Developer Tools
- Firefox's Web Developer Tools and the Firebug add-on
- Chrome's Developer Tools
- Opera's Dragonfly
- Safari's Web Inspector
Each set has their own unique capabilities, but each has the ability to collect network traffic. If we look at the following images, you can see that while the UIs may vary, the data collected and displayed is very similar:
The end result is a list of the browser's network requests involved with downloading our pages' assets or data. The networking tool is able to intercept these requests to show you important data such as:
Fiddler will take the request for your URI and replace it with a local file.
- The type of request (GET, POST, etc.)
- What's being requested
- The URI
- The status
- The size
- How long it took to fulfill the request
and if it's been minified:
In this case, the file hasn't been minified, and I can follow-up with the developer to determine if it makes sense to do so. Granted, it's a 2K file, but every byte matters and this information allows me to better optimize my site.
By hovering over the timelines, we get a dialog that gives us a breakdown of how the request progressed:
Part of that was because it was blocked from loading for 760ms. If this turned out to be a pervasive issue, you could look into using a script loader like RequireJS to better manage script loading and dependencies.
Because dynamic apps are so pervasive, being able to inspect XHR calls is vital. Previously, you saw a ton of network requests, and trying to filter through all of them to find your XHR calls isn't efficient. Because of this, most tools let you choose which types of requests you'd like displayed. Here I'm filtering by XHR requests so I can evaluate the request and response:
By drilling down into the request, I can evaluate important details about the request, such as the headers and status, request method, cookies and most importantly, the response that was returned:
HTML was returned in thise case, but the response could be anything including text, JSON or XML. The great thing is that I'm able to inspect it fully in case I run into any trouble.
Cookies are incredibly useful, and since we use them extensively, having an easy way to inspect their values makes life easier. Developer tools make it easy to do that by showing you which cookies were sent and received:
If you've ever done server-side development without client-side tools, you'll know why this is so awesome.
Overall, the great thing about this is that the capability is right in your browser, making it incredibly convenient to popup the debugger and check things out. Sometimes, though, you need a little more horsepower.
3rd Party HTTP Proxy Tools
HTTP Proxy applications like Fiddler and the Charles Web Debugging Proxy are the big brothers of browser-based network traffic sniffers. Not only can they intercept network requests from the browser but also other applications on your machine, making them much more versatile for debugging. They also tend to offer richer features such as:
- Bandwith throttling
- Autoresponders for specific requests
- SSL proxying
- Plugin ecosystem
- Customizable scripts
- Recording and replay of testing scenarios
I extensively use the Windows-based, incredibly feature-rich Fiddler (it's freeware!). It's also used heavily inside of Microsoft because of it's robust feature-set. The developer of Fiddler, Eric Lawrence, previously worked at Microsoft and still maintains the application.
If we look at the UI, you'll see similarities in the output to what we saw in the browser tools. All of the network requests show up along with key information about the requests.
And by drilling into a request, I can see extensive detail about it, including the minified source of the jQuery library:
Much of that information can be pulled back via the browser-based tools, but what happens when you want to see if a specific library is blowing up your site? You can definitely swap out libraries and troubleshoot. A better route would be to build a Fiddler AutoResponder that intercepts your request and replaces the production library with one of your choice. Think about that for a second. Fiddler will take the request for your URI and replace it with a local file. Let's check it out.
First, I need to identify the URI I want to replace. In this case, I see that my blog's theme is running jQuery v1.2.6. That's insane, but before I drop it in and wreak havoc on my site, I'd like to see if jQuery v1.8.3 works as expected.
I click on the entry for jQuery v1.2.6. In the right column of Fiddler, I select the tab "AutoResponder" and check "Enable automatic responses". Kicking off the responder is as simple as dragging the URI into the rule editor. You'll notice that it starts the rule by comparing the URI. If it matches, it'll respond with an event of your choice.
Since I want to test out jQuery 1.8.3, I want the rule to swap out the production version with a local copy of jQuery that I have on my computer.
I save the rule and re-load my page. The end result is that while the URI might look the same, inspecting the results verifies that jQuery v1.8.3 was in fact injected, allowing me to test this on the fly without making any changes to the site:
From a debugging perspective, I can't stress how useful this is, especially when you're trying to nail down a bug in older versions of a framework or library.
My good friend Jonathan Sampson made a great screencast on using this feature.
Network latency can be a killer, especially for single-page apps.
- Stress testing
- Security auditing
- Traffic diffing to compare two traffic profiles
By itself, Fiddler has a TON of features - too many to describe in this post. That's why there's a 330 page book on how to take full advantage of it. It's only $10 and will help you learn the ins-and-out of this great tool.
OSX and Linux
If you're on OSX or Linux, the best option is the Charles Web Debugging Proxy. It's a great and well-supported app, and while commercial, it is worth every penny. I've looked for good alternatives that focused on web development, and Charles really stood out.
The interface is similar to Fiddler, but it offers two different ways of looking at network traffic:
The style is entirely up to you. I tend to lean towards the structured view because it feels a little more organized, but it is a little more work to find out where a specific URI is.
Like Fiddler, Charles also offers an autoresponder capability. It's called "Map Local...", and you get to it by right-mouse clicking on a specific URI. This allows you to choose a local file to work with.
When I reload the page, I'll now have jQuery v1.2.6 replaced by the local copy of jQuery v1.9 that was on my computer.
Another great feature of Charles is the ability to throttle your network requests to simulate specific bandwidth speeds. I remember the days of 56k modems and their blazing speeds, so using this brings back fond memories (um, right):
Charles can also work on Windows since it offers a complete cross-platform UI.
Which Tool to Use
I use all of these tools all of the time because I test on every major browser. Having this capability really makes troubleshooting easier. Naturally, choosing whether to use a browser-based sniffer or a hard-core app-based proxy depends entirely on your debugging needs.
If you just have to inspect some traffic and check results, a browser-based sniffer will most likely suit you perfectly.
On the other hand, if you need granular control of how URIs respond or want the flexibility to create custom test scripts, then a tool like Fiddler or Charles is where you need to go. The great thing is that we have solid choices to help us do this, especially as the complexity of our projects increase.