Server-side Adobe Analytics

10 Jun 2018 » Analytics Tips , Server Side

Today I will explain how to implement server-side Adobe Analytics. If you have been reading my last few posts, this is a continuation of the series on server-side digital marketing. I assume you are familiar with Adobe Analytics implementations using JavaScript.

Capturing data

When implementing server-side Adobe Analytics, you must perform the same actions you would do with the AppMeasurement JavaScript library in the browser. This library does a lot of the heavy lifting for you, so many data points are captured automatically. However, when going server-side, you need to add all the code necessary to capture the same data points, before forwarding them to Adobe Analytics servers.

  • Uniquely identify the browser. This is a two step process:
    • Generate an ID for the browser. The most common method is to generate this ID server-side and store it in a cookie. I would recommend an expiration of 2 years, refreshed with every page view. For increased security, this cookie can be an HTTP-only cookie.
    • Generate the ECID. Follow the instructions I explained in the post server-side ECID. In the section “Storage of data”, the “Unique ID” will be the ID from the previous point. By now, you have an ECID for the browser.
  • Capture browser details. Adobe Analytics generates the “Visitor Profile > Technology” reports based on information automatically captured by the AppMeasurement library. This information is not mandatory for a correct functioning, but if you use these reports, you now need to add the code in the browser to capture this information and send it to your servers. Here is a list of parameters you may want to capture:
    • Browser height and width
    • Colour depth
    • Are cookies enabled?
    • Is JavaScript enabled?
    • Is Java enabled?
    • JavaScript version
    • Time-zone of the browser
    • Screen resolution
  • Capture HTTP headers. There is some additional data you need to capture, this time coming from the HTTP connection.
    • IP address. This is not an HTTP header, but it will come in the HTTP connection.
    • X-Forwarded-For header. You should evaluate whether you want to honour this header. If you do, it should override the IP address.
    • Accept-Language header. Language of the browser.
    • User-Agent header. String identifying the browser.
  • Adobe Analytics data. Finally, you need to get the Analytics information: page name, eVars, props, events, links… It is completely implementation dependant how you capture all the data. In general, it will be a combination of information you have in the CMS and activity happening in the browser. If you have a data layer, this is the information you are looking for.

Data Insertion API

Once you have all the data you need for your server-side Adobe Analytics implementation, you can start sending it. We will be using the Data Insertion API to send the data. There is nothing special about this API, it is the same one used by the JavaScript Analytics implementation. Therefore, nothing prevents you from calling it from your servers in place of the browser.

Before proceeding, do you have Regional Data Collection (RDC) enabled? Although it is not strictly needed for server-side analytics, you will get many benefits from it. If you do not know what I am talking about, check your collection servers. If they are something like xyz.2o7.net, then you are still using the old configuration. On the other hand, collection servers like abc.sc.omtrdc.net are using RDC. Most, if not all, implementations of the last 4 years should be in RDC. The reason I am mentioning RDC is because I had a customer, who had not transitioned to RDC, but wanted to implement server-side Adobe Analytics. They had a lot of problems with it, especially, many unsuccessful calls. They transitioned to RDC and all these problems went away.

The Data Insertion API has two modes: GET and POST, named after the most common HTTP methods.

XML-based POST method

With this method, you generate an XML file and submit it using an HTTP POST method. The URL to send the data is https://[namespace].sc.omtrdc.net/b/ss//6. You must replace [namespace] with your own value, so the server is exactly the same you use in the client-side Analytics implementation. By the way, there is no mistake in the double ‘//’ towards the end of the URL.

The XML content is has the following structure:

<?xml version="1.0" encoding="UTF-8"?>
<request>
    <sc_xml_ver>1.0</sc_xml_ver>
    <!-- Insert here all Analytics tags -->
</request>

The list of XML supported tags can be found here. The table in the link shows you the XML tag with the corresponding JavaScript variable, so, if you have already implemented Adobe Analytics in JavaScript, you can easily migrate to server-side Adobe Analytics. Take special note of the restrictions described in the previous; for example, you must include one of <visitorID>, <marketingCloudVisitorID> or <IPaddress>.

Since it is usually better to learn by examples, let’s analyse the following XML:

<?xml version="1.0" encoding="UTF-8"?>
<request>
    <sc_xml_ver>1.0</sc_xml_ver>
    <browserHeight>768</browserHeight>
    <browserWidth>1024</browserWidth>
    <imsregion>6</imsregion>
    <marketingCloudVisitorID>36800664351771965062969965047613395449</marketingCloudVisitorID>
    <reportSuiteID>myrsid</reportSuiteID>
    <ipaddress>192.168.0.10</ipaddress>
    <pageName>Home - POST</pageName>
    <pageURL>http://test.domain.com/index-post.html</pageURL>
    <userAgent>Mozilla/5.0 (Android 4.4; Mobile; rv:41.0) Gecko/41.0 Firefox/41.0</userAgent>
</request>

I am pretty sure you understand all the parameters, but just in case:

  • The browser window is 1024×768 pixels in size
  • The <imsregion> is the value of dcs_region, received in the response of the server-side ECID. In this case, 6 for Europe.
  • The <marketingCloudVisitorID> is the d_mid parameter, also from the server-side ECID call.
  • The report suite ID is “myrsid”.
  • The IP address of the browser making the request is 192.168.0.10. Review section “Capturing data” to see how the get the IP address.
  • The page name viewed in the browser has a name of “Home - POST” and its URL is “http://test.domain.com/index-post.html”.
  • The browser is Firefox, version 41, running under Android 4.4. This data is extracted from the User-Agent: “Mozilla/5.0 (Android 4.4; Mobile; rv:41.0) Gecko/41.0 Firefox/41.0”. As with the IP address, this value comes from the “Capturing data” section.

If everything went well, you will get the following response:

<?xml version="1.0" encoding="UTF-8"?>
<status>SUCCESS</status>

This response is useful to confirm that the insertion has been successful. You do not get anything similar with the GET method, so this is why, in general, the POST method is preferred.

You can also look into the official tutorial: https://github.com/AdobeDocs/analytics-1.4-apis/tree/master/docs/data-insertion-api, with various examples.

GET method

The GET method is exactly what the AppMeasurement JavaScript library uses. We are just going to generate it from server-side, with a couple of tweaks. Instead of creating an XML file and send it to Adobe Analytics’ collection servers, all the data is stuffed in the URL, as query string parameters. This URL has the format:

https://[namespace].sc.omtrdc.net/b/ss/[rsid]/0?key1=val1&key2=val2...

The [namespace] and the [rsid] are placeholders for your namespace (the same as in the POST method) and the report suite ID respectively. The keys will be, as usual, information from the browser, the current page, user interactions, custom variables (eVars, props, events), etc. The accepted keys are documented here. The link is the same as with the POST method and there is a 1:1 correspondence between most XML tags and most query string parameters. Remember to URL-encode all values.

To make matters even simpler, you can analyse an existing JavaScript Analytics implementation as it sends the data and reproduce it server side. I will refer you to my post How to debug an Adobe Analytics implementation with instructions on how to see how the AppMeasurment library sends the Analytics data.

When making the request, you must add two additional HTTP headers:

  • X-Forwarded-For, with the IP address of the browser; otherwise, Adobe Analytics will use your server’s public IP address.
  • User-Agent, with the browser’s value.

Let’s analyse an example:

GET /b/ss/myrsid/0?bh=768&bw=1024&aamlh=6&mid=36800664351771965062969965047613395449&aamb=j8Odv6LonN4r3an7LhD3WZrU1bUpAkFkkiY1ncBR96t2PTI&pageName=Home%20-%20GET&g=http%3a%2f%2ftest.domain.com%2findex-get.html HTTP/1.1
Host: [namespace].sc.omtrdc.net
Accept: */*
User-Agent: Mozilla/5.0 (Android 4.4; Mobile; rv:41.0) Gecko/41.0 Firefox/41.0
X-Forwarded-For: 192.168.0.10

If you take a closer look, you will notice that this example is very similar to the example for the POST method above. The main difference is the aamb parameter, which comes from the d_blob parameter of the ECID response, which has not correspondence in XML.

[UPDATE 27/08/2019] You can also use a POST method based on these parameters. Instead of passing in the body an XML document, you send the URL-encoded query string parameters. This is why I sometimes mention 3 methods, not just 2.

Sample codes of server-side Adobe Analytics

As with the server-side ECID, I have created a couple of scripts to show how these methods work:

I assume that you already have ecid.sh from my previous post and it works. Further instructions:

  1. Download the files and store the in them same directory as ecid.sh.
  2. Rename them, changing the extension from .txt to .sh.
  3. Edit them and set the collection server and report suite ID.
  4. If necessary, give the scripts executable permissions: chmod a+x aa-post.sh aa-get.sh.
  5. Make sure ecid.sh has already run and generated the ecid.json and cookies.txt files.
  6. Run the desired script 
    $ ./aa-get.sh
$ ./aa-post.sh

Feel free to modify these files until you get what you are looking for. Let me know in the comments if you run into any issues.

Related Posts