Monitoring Prices
On Bot-Aware Websites

Check out This Article

Why Our Clients Hire Us

For almost a decade Fortune 100 companies and highly successful manufacturers have hired eEnforce for their ability to find the real identities of sellers and permanently stop them from infringing upon their trademarks on Amazon, eBay, Walmart & Google.

roadmap

If you’re a loyal fan of Twitter, and at the same time you’re interested in eCommerce, then you’re certainly not bypassed by the news of Telfar bags being sold out in a matter of seconds a few days ago. You may think that this only concerns fashion lovers, but there is more story to it. Why would some bag cause such a backlash all over the Internet? It turned out that the whole collection didn’t go to buyers, but resellers instead.

How was that possible? Well, it seems that resellers used bot (non-human) technology, which makes the whole story not quite as simple as it seems.

So, what is bot traffic, bot-aware websites, and how eCommerce businesses can monitor prices in such an environment?

Let’s shed some more light on it.

How bot-driven traffic is used in eCommerce?

One of the massive changes that has taken the eCommerce industry by storm is definitely bot-driven traffic. Monitoring websites that are constantly investing resources into blocking automated bot traffic is far away from being an easy task. In the eCommerce environment this kind of monitoring is mostly related to product prices. This has been the case for more than a decade now, since Amazon helped pioneer this practice.

In the case of brick-and-mortar stores price changes are usually happening on a weekly basis. That takes a lot of work, so it’s understandable why it can’t be done more frequently. In the eCommerce world, though, prices change even a few times a day. This process is done with ease with the help of algorithms. These algorithms gather all different types of data – competitors pricing data, price and sales forecast, inventory levels, etc.

To stay in the game, eCommerce players use different techniques including bots, in order to not lag behind their rivals. In practice, this means that they have only a few minutes during which they need to become aware of any competitors price change.

You may think that this is exaggeration and that a few minutes or half an hour can’t make such a difference. However, even this short period may seem like a lifetime during most popular shopping periods such as Christmas, or Black Friday. The businesses who are not priced efficiently will become aware of it almost immediately – by the decrease in sales.

Disguised as humans

Monitoring prices has become one of main focuses in the eCommerce business. Of course, competition is also very aware of this, so everyone is trying to find ways to stop it.

That’s when bot-driven traffic made its appearance.

Bots are meant to work as humans would when they visit a certain website. That way, websites would think that they receive real, human traffic, while instead the bots are scraping the website and getting all the information needed. Nowadays, websites are investing a lot of time and money into blocking automated bot traffic.

In eCommerce the use of bots may seem like a vicious circle. Companies are trying to find good examples and use them themselves while aiming to “break through” the competitors defense systems at the same time. However, not all companies are skilled enough to do this on their own. For those less-savvy ones, there are third party solutions, such as Price2Spy, that can be beneficial.

Bot-aware techniques

One of the ways that websites developed to protect their data from competitors was CAPTCHA. CAPTCHA stands for “Completely Automated Public Turing test to tell Computers and Humans Apart”. It’s a string of numbers and letters that bots aren’t able to read, but humans are. Therefore, it’s clear that if competitors want to evade this solution, they need to find a way for their computer to act like a human.

Different techniques are used. For instance, some programs are acting like humans would while visiting a website. Therefore, they don’t go directly for the price information, but instead roam around the page as any other potential customer would.

 

As users, we’re all familiar with CAPTCHA. It’s a box where we usually need to complete some very basic test – such as inserting letters or numbers, finding objects, etc. Solving it manually it’s not an issue, but doing it automatically is more harder than it seems. In fact, for many websites this is an insurmountable obstacle, but Price2Spy has found a way to overcome it.

However, the techniques used for CAPTCHA solving require a much more complex infrastructure. This means that a website which requires this approach will, in fact, be using Stealth technology. After a lot of hard and dedicated work, Price2Spy was able to develop this technique. To put it simply, system administrators find it impossible to distinguish regular traffic from the traffic generated by Price2Spy. That allows Price2Spy to monitor sites like Amazon, Google Shopping, HomeDepot etc. Pretty impressive, right?

As you can see, both sides are almost on a monthly basis developing new plans of action. If we tell you that reportedly, non-human traffic makes up more than half of the whole Internet traffic, you get a picture of what a big deal it is. Monitoring prices on bot-aware websites turned into a real cat-and-mouse game.

That game often leaves companies with a very rude surprise: the technology that they were using to monitor prices on a certain website suddenly stopped working.

Similar thing has happened recently with Amazon. They made an important upgrade to its anti-bot detection system, but Price2Spy managed to resolve this issue.

Wrapping it up

As you can tell, this is a never ending game, where both sides try to make a smarter move which will set them apart from the competition. With all the tech improvements happening on a daily basis, this can hardly be an easy task. eCommerce businesses are maybe in the hardest position. With price changes happening so frequently, it’s almost impossible to stay on top of everything without the right help.

Price monitoring tools now often offer price monitoring on bot-aware websites. Since this is a complicated task, it’s important that you entrust it to someone experienced.

Price2Spy offers you a 30 day free trial, so that you have enough time to test even the most complicated solutions.

What has been your experience with monitoring prices on bot-aware websites?

What has been your experience with monitoring prices on bot-aware websites? We would like to hear more about it in the comments below!