Last month I presented at The Quirks Event in Brooklyn and was honestly blown away by the response I received. To all of you who reached out at the show and after, please accept my sincere heartfelt “Thank you!” Considering the response, I thought it made sense to share the highlights of the presentation here for those unable to attend. I also want to thank Quirks for organizing the event. It was truly one of the best MR events I’ve ever attended and strongly recommend everyone make it a point to attend next year.

A 30-minute presentation is a lot of cover in a single blog so I’ve decided to break it up into 3 which I’ll release each week. Thanks for reading and enjoy!

Surprise! Sample companies buy quite a bit of sample from each other on a regular basis. You didn’t really think a single panel could supply 1000 census rep completes in a small DMA did you? As a result, we have become good at working with each other to cut through the noise and create an effective partnership. I’m peeling back the curtain here in hopes that our full-service market research clients and end-clients can integrate into their daily process creating a smoother interaction for all of us.

In the presentation, we discussed best practices for vetting suppliers and how to evaluate if you’re dealing with a reputable company with quality sample. The second and third sections had an operational focus on how we manage our partners to yield accurate bids and maximize success in field.

When evaluating new suppliers our focus is on their panels and their methods for vetting their panelists. We also smell test new suppliers. New sample companies (there are hundreds now) pitch us daily and most of them don’t “smell” very good to be honest. Rome wasn’t built in a day and neither are panels. What we’re seeing is a larger and larger amount of domestic and overseas firms who have figured out that they can simply buy sample from a sample exchange and pass it off as their own. If you’re approached by a young company who claims to have millions of panelists in dozens of markets globally, they probably don’t. We’ve been building the Precision Sample panel now for over 7 years and we’ve just now gotten north of 9 million panelists in 22 markets.

This matters because sample firms, like every other online business, have always been under attack. In the 2000’s it was “click farms” or shops overseas in places like India, Vietnam, China, Russia and more with real people taking surveys impersonating US citizens, B2B & IT professionals. That evolved into those groups creating bots to take surveys and the next evolution is what you see on the next slide.

Ultimate Survey Bot is a free, consumer product that anyone can install on their PCs, point to their panel account and it will take all surveys available in that account, qualifying for a good portion of them. We’ve been tracking this product for a while and have reverse engineered it to prevent it from accessing our client surveys. I can tell you that it is broadly in use and installed on many computers in the US and abroad and that it is quite difficult to detect and even good at giving consistent answers to surveys. Mind you I said consistent, not accurate. We find a decent portion of the traffic we receive from other suppliers and exchanges is comprised of this bot specifically. There are also a few other free consumer survey bots available.

I figured there would be quite a bit of tension in the room after I shared this and you may be tense right now too if our friend the Ultimate Survey Bot is new to you. I heard a huge sigh of relief and a nervous laugh when I showed this slide. Let’s take a minute to decompress together….

Ok, ready?

Great… The good news here is that reputable sample companies are aware of this and other bots and have been actively staying one step ahead of them for the past few years. Here’s how we do it.

At Precision Sample we’ve found that a single validation strategy is insufficient to keep not only the bots, but also undesirable groups out of our client’s surveys. We’ve classified these into four pillars or buckets. There are multiple techniques in each of these buckets and our platform, Quality Sentinel, has these and some additional special sauce layered in. When you’re evaluating new suppliers, focus on systems they have in place to validate their panelists. If they can’t speak to the details on the following page in an informed manner and show you some real results for how many panelists they screen out using these techniques then you’re better off looking elsewhere. Through the above article, we can recommend you the latest dresses.in a variety of lengths, colors and styles for every occasion from your favorite brands.

I’ll quickly summarize what each of these pillars accomplish

• Identity Validation is the process of comparing panelist supplied PII (personally identifiable information – name, address, DOB) against 3^rd party databases. We’ve found that the real key is to require an exact match on date of birth. Reason being that it’s easy to pull a name and postal address from the internet, more difficult to get the date of birth.
• Digital Fingerprinting – by now I think we all know what digital fingerprinting is so I won’t go into the details but it’s important for deduplication. 5 years ago, digital fingerprinting was the holy grail for unique sample, unfortunately it’s not today. Have you noticed that all browsers have some form of “Private Browsing” session feature on them now? Guess what, that defeats digital fingerprinting by masking the settings of your computer. The cheats have figured this out by now so it isn’t very effective at keeping them out but is effective at keeping honest participants out who happened to get into the same survey twice.
• IP Geo Validation – the great thing about overseas groups and bots is that they are limited to entering surveys from proxy IP addresses. An IP address has a physical location that it is tied to. We have postal address on our panelists and compare the location of the IP address to the postal address. If we find they are not in an acceptable radius of each other, we stop the survey session. This check is applied at every interaction we have with our panelists and it’s an incredibly effective and powerful tool.
• CAPTCHA – our last line of defense is to require a CAPTCHA prior to each survey. Good CAPTCHAs require a human to complete them. We use reCAPTCHA. It’s provided by Google and free. If your supplier does not have CAPTCHA on their end, we recommend placing one at the start of your survey.

We look for our partners to have 3 of the 4 of the above validation measures in place for developed markets and if it’s a B2B or IT study, best to have all 4.

Next up are bidding best practices and strategies to maintain healthy relationships and generate accurate feasibility. See you next week!

–Don Golden