RE: How does google know my bot is a bot? 07-03-2016, 02:43 PM
#11
Anyways, Google is probably changing there bot detection techniques every day! So its probably a combination of many things.
| How does google know my bot is a bot? filter_list | |
(07-03-2016, 09:02 AM)Eclipse Wrote:(06-12-2016, 01:26 PM)Stocking Wrote: Google does more user checking than a user agent alone. However, if it's something such as a search, just use Google's API. APIs exist for more than one Google service, in fact most of them, so if it's something that can use an API, in this case, it should.
As far as what else it does, I'm not sure. I myself have never been able to quite "crack the code" on what Google does. Perhaps it's worth a Google.
Google's search API is deprecated and it was rate limited anyway.
![[Image: BXqGARG.png]](https://i.imgur.com/BXqGARG.png)
(07-09-2016, 12:57 AM)Stocking Wrote:(07-03-2016, 09:02 AM)Eclipse Wrote:(06-12-2016, 01:26 PM)Stocking Wrote: Google does more user checking than a user agent alone. However, if it's something such as a search, just use Google's API. APIs exist for more than one Google service, in fact most of them, so if it's something that can use an API, in this case, it should.
As far as what else it does, I'm not sure. I myself have never been able to quite "crack the code" on what Google does. Perhaps it's worth a Google.
Google's search API is deprecated and it was rate limited anyway.
and it's not hard to bypass rate limiting on, well, anything.
(07-09-2016, 08:34 AM)Eclipse Wrote:(07-09-2016, 12:57 AM)Stocking Wrote:(07-03-2016, 09:02 AM)Eclipse Wrote: Google's search API is deprecated and it was rate limited anyway.
and it's not hard to bypass rate limiting on, well, anything.
I'd disagree. Google uses machine learning algorithms and it's vast amount of data to map how a bot looks and behaves and then serves anyone that fits the bill a captcha. It even flags up your real browser of you behave like a bot. Feel free to attempt it.
![[Image: BXqGARG.png]](https://i.imgur.com/BXqGARG.png)
(07-09-2016, 11:50 PM)Stocking Wrote:(07-09-2016, 08:34 AM)Eclipse Wrote:(07-09-2016, 12:57 AM)Stocking Wrote: and it's not hard to bypass rate limiting on, well, anything.
I'd disagree. Google uses machine learning algorithms and it's vast amount of data to map how a bot looks and behaves and then serves anyone that fits the bill a captcha. It even flags up your real browser of you behave like a bot. Feel free to attempt it.
Homemade proxies are a thing. I was referring to the APIs. The only reason you wouldn't be able to bypass the rate limiting now is because you would use the same API key, however, if you have multiple already, you're good to go.
(07-10-2016, 12:56 PM)Eclipse Wrote:(07-09-2016, 11:50 PM)Stocking Wrote:(07-09-2016, 08:34 AM)Eclipse Wrote: I'd disagree. Google uses machine learning algorithms and it's vast amount of data to map how a bot looks and behaves and then serves anyone that fits the bill a captcha. It even flags up your real browser of you behave like a bot. Feel free to attempt it.
Homemade proxies are a thing. I was referring to the APIs. The only reason you wouldn't be able to bypass the rate limiting now is because you would use the same API key, however, if you have multiple already, you're good to go.
No idea how homemade proxies come into this.
![[Image: BXqGARG.png]](https://i.imgur.com/BXqGARG.png)