Synthetic Monitoring
Previously, we spoke about the many reasons that monitoring could be enabled and the many types of monitoring. This article will be a two part piece that lays the groundwork for what the use cases are for each of the following monitoring types; Synthetic vs. RUM/RUA. Let’s start with Synthetic monitoring and its use cases.
Synthetic is not a bad thing
This needs to be clearly stated: the two use cases for RUM vs Synthetic are very different and one should not be vilified over another. Don’t let very different RUM vendors make this claim! The two are complementary as you will read later.
Here’s my working definition of Synthetic monitoring agents: These are known [location, ISP, CPU- standardized], instrumented [agents/browsers outfitted with analytics-gathering], fixed-bandwidth agents that are dedicated to doing repeated measurements. These agents are typically high bandwidth, but could also be DSL, but they are tasked to specifically run pre-determined scripts from their locations against target sites, on a periodic basis.
These distinctions of what makes a synthetic monitoring agent are critical in understanding the differences between these and RUM.
The case for Synthetic
Synthetic agents playback scripts that tell them specific things to do, over and over again, from a very clean, consistent environment.
About Scripting
These scripts are written by either following a transaction and recording it from within a real browser or having a scripting tool that embeds a browser within it and you create your script by navigating through the embedded browser.
These scripts do something as simple as getting the Homepage, or a single object on the page (or just the base HTML), to conducting a complete login and checkout transaction. When sent to the agents to execute, these scripts provide a known set of actions and can be counted on to be consistent for execution or availability of the pages they include. The script MUST be tweaked to know when to stop capturing all the data on the page and then move on to the next step or stop the measuring. Thus: scripts can be very temperamental (complex ones break all the time) and not all websites work well with scripts (like Flash-based UI ones).
The agents running these scripts do NOTHING else but run these customized scripts from their respective locations.
These agents will run these scripts at a known frequency (e.g. hourly, every 30, 15, 10, 5 minutes, etc.) or they will run these scripts within a certain period (e.g. 3 times per hour). The point being is that you can expect a certain number of measurements each day, hour, week, etc.
You can see that agents and their scripts provide a type of “clean-room” kind of reliable measuring over time.
Operations-oriented people use synthetic agents to know fundamentally if their service-application is alive (Up) or not and performing within tolerances. They may not care about page performance and only that the web/application servers are responding for a select page (e.g. the login page). RUM cannot do this for them because nothing is actively pinging their applications/servers; Synthetic can.
Web Performance & Marketing/SEO people care about performance but want to understand on a long- term basis what performance normally looks like, both as a baseline and as new releases are put into place. Knowing this from a known set of high bandwidth (backbone) agents is useful to them and they can see if their performance is affected at all with large amounts of bandwidth. Agents with mobile network or lower bandwidth DSL connections could also provide additional data points.
Note: if I told you that a DSL customer had a 9 second page load time, you might attribute the problem to the DSL speed. But if I told you that this same page loaded in 9 seconds from the same location with a very high bandwidth site, then you’d know that this may NOT be about bandwidth but page design or 3rd-party content slowing the page down.
One final thing about synthetic agents: they vary in capability widely. Some are real and very current browsers, while others are emulated browsers that may only reflect some parts of the performance and not others. Some providers lag behind the current browser crop because of the lag in instrumenting the browsers and then deploying them as agents.
Other emulated “browsers” miss the actual execution path of the DOM, or don’t execute Async JavaScripts, so may not call down all the page elements in the same order (or at all) as a real browser. Some limited ones only do GET calls for all content and don’t execute JS or CSS, just fetch and measure these objects.
So there is one particular place that Synthetic can do over RUM-it can measure API call performance. Since an API call does not typically fire an embedded RUM beacon, RUM cannot help unless it can somehow fire/rum whenever a particular API is being requested. With Synthetic, you can script a very precise action and API call that can be repeatedly called. With RUM, you must fire a beacon to get the data-if it never fires, you never have the data.
So synthetic has its place, even in the world of modern RUM. Next week, read what RUM/RUA has to offer and why you might use one over the other?