Introduction to HTTP debuggers
31 Oct 2021 » MSA
In many blog posts, I have explained how to use the browser’s developer tools to debug or test implementations. In my day-to-day work, whenever I need to check an HTTP call or a cookie, I use these tools. I have used them so much, that I do not have to even look at the keyboard to press CTRL+Shift+I. However, there are some situations where you need more than what the browser can offer. Enter the HTTP debuggers.
What is an HTTP debugger?
An HTTP debugger is a tool that is designed to play with the HTTP traffic that goes through it. It is not just a proxy that acts as an intermediary between a client and a server. While it can act as a plain proxy, it has many more features baked in: you can review the HTTP traffic, manipulate it, replay it… This means that you need to understand very well the HTTP protocol and the data it transfers.
I have only worked with two of such tools: Fiddler and Charles. However, a quick Internet search shows that there are many more. I have mainly used Charles and I think it is the best tool of the two. Its license is affordable, but if money is an issue, Fiddler, which has a free option, is also a good tool.
I want to clarify one aspect of HTTP debuggers. They require some expertise, sometimes a license, maybe your IT department does not like them… making them cumbersome to use. When I started working at Adobe, I used Charles almost daily, as the browsers did not offer any developer tools. However, thanks to these tools, I rarely use Charles nowadays. What I am trying to say is that you should not try to boil the ocean; if you can use only your browser, do so.
Use Cases
If you read the features of any HTTP debugger, you will get a long list of terms that you may not even understand. It is up to you to make the connection between these features and your problems. Although frustrating, you need to understand that the developers of these debuggers do not know how they will be used. All they can do is list their features and how to use them.
That being said, if you are reading this post, it is probably because you work with Adobe tools and you need to debug their implementations. Since I have used HTTP debuggers on multiple occasions, in this post I want to share the use cases I have come across, where these tools have become handy. My hope is that you will end up understanding when these tools are useful in real-world scenarios related to Adobe tools. If you have other use cases, please, let me know in the comments, so I can add them.
Debugging mobile implementations
I am not an expert in mobile devices but, from what I can tell, there are important limitations with what you can do on a mobile device. The consequence is that it is very difficult or impossible to debug a native app with an Adobe SDK or a mobile website with Adobe tools. I know that, at least in the past, you could connect your iOS Safari with your MacOS Safari, but I do not know what features this connection offers.
In my opinion, the best solution to debug any of these two cases is to run an HTTP debugger on your laptop and configure your mobile device to use your laptop as an HTTP proxy. Then, you can play with your app or mobile web and see all the HTTP calls on your laptop. There are still some considerations that you need to be aware of:
- Both laptops and mobiles device must be on the same network.
- Not all apps will honour the HTTP proxy settings. If yours is one of those, tell the developers to fix it.
- Configuring the proxy in your mobile device may require a bit of expertise.
- For HTTPS traffic, you will have to add some additional, non-trivial configuration to your mobile device.
Even with these shortcomings, it is the best way to debug the calls to Adobe tools on a mobile device.
Although I will not go into any more details, this use case can easily be adapted to server-side implementations.
Switch tag manager environment
When you modify your tag manager configuration, you want to test the changes in the production website without publishing your changes. If you are using Adobe Launch, you can use the Launch and DTM Switch or the Adobe Experience Platform Debugger. Both tools offer a feature to switch between environments.
However, it is not always possible to use a browser extension. The most notable example is, as explained before, mobile devices. Other examples could be obscure browsers that does not support WebKit extensions or if your IT policy do not allow you to install these extensions. In these situations, using an HTTP debugger with an URL mapping feature would do the trick. You just need to configure this tool so that, when it receives a request to the tag manager’s production URL, it actually fetches the development URL.
The same applies to cases where you do not use tag managers. In other words, if your website loads directly any of the Adobe JavaScript libraries (at.js, VisitorAPI.js…) and you need to test a new version. You can use an HTTP debugger to swap the published code with a local version of the file. This is actually how I worked when I became an Adobe Analytics consultant, as we were still using s_code.js. I had different versions of this JavaScript file on my hard disk and used Charles to test them against my client’s website.
Accurate testing
If you are a tester, you know that under no circumstances should you publish any new code without first testing it. Never.
So, if you are tasked with testing a new configuration or new code in Adobe Launch, you could use one of the browser extensions mentioned earlier. However, there is one important detail that you should know. These browser extensions are written in JavaScript and they play with your HTML. Although they work, the experience you get using them is never going to be exactly what your end-users will get. Remember, they will not have these extensions.
Ideally, you would want to test a setup, which is as close as possible to your end-user environments. Again, an HTTP debugger is the solution. In fact, this is just an extension of the previous use case. You could have a browser with no extensions and configure the HTTP debugger as described above, to fetch a development or staging library.
There is even an additional benefit. Browser extensions will inevitably add a delay to the load time. If you are also evaluating the performance, an HTTP debugger will barely add any noticeable delay.
Inject HTML
The final use case that I have come across is when you need to inject HTML to test new features. The best example is when you are migrating to Adobe Launch from another tag manager or no tag manager at all. In this scenario, your production website will not even have the Launch code, so browser extensions will not work. You can use the rewrite functionality of the HTTP debugger to search for <head> in the HTML and append the top-of-the-page code, and search for </body> and prepend the bottom-of-the-page code.
Also, if you are waiting for new elements in the data layer to be deployed, you could use this same functionality to inject them while you add additional tag manager rules that require the new data.
How to use Charles
My initial plan was to write a single post on Charles. However, as I thought and wrote more, I realised that there was a lot of information to share. So, I have broken up my initial ideas into multiple partes. In future posts, I will explain how to configure Charles to achieve the previous use cases.
Image by rawpixel.com