The audit is a process for verification of the numbers that you report to your advertisers. Audits can be performed in a number of different ways.
* Server-based audits examine data that is available at the server, most importantly traffic logs and web logs. An auditing organization will prowl through the logs to check for various kinds of impressions that should not be reported. This investigation will include an examination of the parameters you use to run your traffic analysis programs. Auditors may insert software in the web server that causes independent logs, totally under the auditor's control, to be created.
* Panel-based audits measure the surfing behavior of a sample panel of users, and attempt to project that statistically to the entire Web population
* Browser-based audits attempt to confirm actual ad displays. For example, an applet can be attached to an ad or to a page; the applet will report when the ad is actually displayed on some user's browser.
Larger consumer sites like Yahoo and Amazon.com, and their advertisers, use panel-based audits, and the numbers are sometimes front-page news. Smaller consumer sites and B2B sites generally don't have the volume for panel-based audits to be statistically significant, and rely mostly on server-based audits. Browser-based audits are a newer technique and are not heavily used.
How good are the different techniques? Jim Spaeth, President of the Advertising Research Foundation, tells of comparisons where on site X a server-based procedure showed 15% of the traffic shown by a panel-based audit, but on a second site Y the order of the methods was reversed, with the server-based procedure showing 300% of the traffic that the panel-based audit did. "This kind of result gives people chills down the back of the neck," Mr. Spaeth said. He also noted that different procedures of the same class also tend to produce different numbers.
Figure 1, from a comparison of three different measures of traffic on Yahoo in 1999, shows graphically how different techniques may differ; the Figure was originally published on TheStandard.com.
When your CFO faces an audit, it's always perfectly clear what's required. If there are problems, the auditor can explain exactly what they are. Try this: Ask your CFO if it would be surprising to have an audit performed by two different highly respected firms on the same business at the same time and get wildly different results. You already know what the answer is: some variant of "that shouldn't happen." That's because accounting firms and standards bodies have agreed on rules for audits that cover almost any question that could be asked. Yet, despite the apparent simplicity of the data that need to be analyzed and the fact that the Web is all technology all the time, the standards just aren't there yet.
This may be partly because there is no single recognized standards body as there is for financial accounting (within a country). However, that situation is starting to change as voluntary or ad hoc organizations put in the work to develop standards. One such organization is FAST, which stands for Future of Advertising Stakeholders. FAST has developed a number of draft standards for how and what to measure, and some are being adopted voluntarily. However there is no legal or even quasi-legal pressure to make sites, software manufacturers or auditing firms adhere to them.
One promising attempt to level the playing field is the planned September launch of Audit Central, a web site that will publish audit reports that have been made publicly available by the sites that were audited. The site is run by ABC Interactive, BPA International, and Engage I/Pro. These competing audit firms have a clear interest in improving the quality of audits and public recognition of their value. The site is scheduled to begin with approximately 600 reports, all from companies that have agreed to make their reports public.
One of the simplest measures that any site wants to know is how many different individuals - "unique visitors" in industry parlance - visit the site. FAST's draft standard on Metrics and Methodology suggests "three acceptable methods for identifying unique users: unique registration, unique cookies and unique IP address with heuristic." Of these, it suggests that unique registration is the best, "Sites that register visits should have no problem determining the page requests that belong to the same visitor. A site must use 100% registration in order to use this method validly."
Next is the use of unique cookies. If a unique cookie is dropped on every browser, the user can be uniquely identified even without any personal information. The third method calls for the use of IP addresses. However, IP addresses are only an approximate match to actual users. As FAST states, "It must be noted that IP addresses can and often do represent more than one user, so this measure does not necessarily represent the number of people reached. It should also be noted that dynamically assigned IP addresses impact the accuracy of this methodology."
Few websites require registration before showing any pages at all to a user, so the most practical way to track individuals uniquely is with cookies. (True some small percentage of users block cookies, but because there are so few they become largely irrelevant to the discussion). The standard doesn't say that a site has to drop cookies, only that if it doesn't it must have another way to count visitors.
If the site doesn't drop unique cookies then visitor calculations have to be done by making educated guesses based on IP address. Such guesses could take into account the time period between two pages served to the same IP address, the click trail as revealed in the referrer field, and other items. In the latter category are cookies that may be dropped by the web server without the website taking explicit action; Microsoft's IIS in particular can end up dropping quite a few. Any particular traffic program can use any of these means to count visitors, but there is no one best way to do so.
* Server-based audits examine data that is available at the server, most importantly traffic logs and web logs. An auditing organization will prowl through the logs to check for various kinds of impressions that should not be reported. This investigation will include an examination of the parameters you use to run your traffic analysis programs. Auditors may insert software in the web server that causes independent logs, totally under the auditor's control, to be created.
* Panel-based audits measure the surfing behavior of a sample panel of users, and attempt to project that statistically to the entire Web population
* Browser-based audits attempt to confirm actual ad displays. For example, an applet can be attached to an ad or to a page; the applet will report when the ad is actually displayed on some user's browser.
Larger consumer sites like Yahoo and Amazon.com, and their advertisers, use panel-based audits, and the numbers are sometimes front-page news. Smaller consumer sites and B2B sites generally don't have the volume for panel-based audits to be statistically significant, and rely mostly on server-based audits. Browser-based audits are a newer technique and are not heavily used.
How good are the different techniques? Jim Spaeth, President of the Advertising Research Foundation, tells of comparisons where on site X a server-based procedure showed 15% of the traffic shown by a panel-based audit, but on a second site Y the order of the methods was reversed, with the server-based procedure showing 300% of the traffic that the panel-based audit did. "This kind of result gives people chills down the back of the neck," Mr. Spaeth said. He also noted that different procedures of the same class also tend to produce different numbers.
Figure 1, from a comparison of three different measures of traffic on Yahoo in 1999, shows graphically how different techniques may differ; the Figure was originally published on TheStandard.com.
When your CFO faces an audit, it's always perfectly clear what's required. If there are problems, the auditor can explain exactly what they are. Try this: Ask your CFO if it would be surprising to have an audit performed by two different highly respected firms on the same business at the same time and get wildly different results. You already know what the answer is: some variant of "that shouldn't happen." That's because accounting firms and standards bodies have agreed on rules for audits that cover almost any question that could be asked. Yet, despite the apparent simplicity of the data that need to be analyzed and the fact that the Web is all technology all the time, the standards just aren't there yet.
This may be partly because there is no single recognized standards body as there is for financial accounting (within a country). However, that situation is starting to change as voluntary or ad hoc organizations put in the work to develop standards. One such organization is FAST, which stands for Future of Advertising Stakeholders. FAST has developed a number of draft standards for how and what to measure, and some are being adopted voluntarily. However there is no legal or even quasi-legal pressure to make sites, software manufacturers or auditing firms adhere to them.
One promising attempt to level the playing field is the planned September launch of Audit Central, a web site that will publish audit reports that have been made publicly available by the sites that were audited. The site is run by ABC Interactive, BPA International, and Engage I/Pro. These competing audit firms have a clear interest in improving the quality of audits and public recognition of their value. The site is scheduled to begin with approximately 600 reports, all from companies that have agreed to make their reports public.
One of the simplest measures that any site wants to know is how many different individuals - "unique visitors" in industry parlance - visit the site. FAST's draft standard on Metrics and Methodology suggests "three acceptable methods for identifying unique users: unique registration, unique cookies and unique IP address with heuristic." Of these, it suggests that unique registration is the best, "Sites that register visits should have no problem determining the page requests that belong to the same visitor. A site must use 100% registration in order to use this method validly."
Next is the use of unique cookies. If a unique cookie is dropped on every browser, the user can be uniquely identified even without any personal information. The third method calls for the use of IP addresses. However, IP addresses are only an approximate match to actual users. As FAST states, "It must be noted that IP addresses can and often do represent more than one user, so this measure does not necessarily represent the number of people reached. It should also be noted that dynamically assigned IP addresses impact the accuracy of this methodology."
Few websites require registration before showing any pages at all to a user, so the most practical way to track individuals uniquely is with cookies. (True some small percentage of users block cookies, but because there are so few they become largely irrelevant to the discussion). The standard doesn't say that a site has to drop cookies, only that if it doesn't it must have another way to count visitors.
If the site doesn't drop unique cookies then visitor calculations have to be done by making educated guesses based on IP address. Such guesses could take into account the time period between two pages served to the same IP address, the click trail as revealed in the referrer field, and other items. In the latter category are cookies that may be dropped by the web server without the website taking explicit action; Microsoft's IIS in particular can end up dropping quite a few. Any particular traffic program can use any of these means to count visitors, but there is no one best way to do so.
No comments:
Post a Comment