greetings, fellow internet!

welcome to digitalgangster.com, the coolest community for people who hack the gibson and make bajillions of dollars off of online marketing. this version of the forum is in view only mode for archival purposes. the new forum can be found here.

ASGPHE,
ytcracker, original digital gangster
follow me on twitter
fan me on facebook
Results 1 to 15 of 15

Thread: Can google block you for too many search queries?

  1. #1

    Default Can google block you for too many search queries?

    Similar to how yahoo will ip ban and give you a Http status code of 999 - chill the fuck out. Will google IP ban you or send you a captcha if you do too many google queries.... like say, 200 in a day, 500, 1k, 2k, etc. Or just do too many too fast. This has to do with a multi-threaded app which can potentially churn out a lot of queries.

  2. #2
    Audio's Avatar
    Join Date
    Jun 2007
    Location
    banks back room
    Posts
    15,911

    Default

    yep, google does hit you with a captcha after a ton of queries too fast
    im not sure about daily limits, but there is def rate limiting
    Coder for hire. Anything you need, outstanding prices and good return time.
    Contact: Skype - GetMoneyHo, AIM - theorangebox

  3. #3

    Join Date
    Jan 2008
    Location
    mile marker 0
    Posts
    336

    Default

    yeah, google throws up a similar page that says you are conducting too many searches and to protect the users of google they are temporarily stopping results from being displayed.

  4. #4

    Join Date
    Apr 2009
    Location
    Serbia
    Posts
    624

    Default

    Quote Originally Posted by Audio View Post
    yep, google does hit you with a captcha after a ton of queries too fast
    im not sure about daily limits, but there is def rate limiting
    this and I'm pretty sure if you search certain types of strings they do it even quicker.

  5. #5

    Default

    Also, once you complete the captcha... does it keep asking for it over and over or does it sort of give you some more chances to ask for more queries before another captcha? .....IIRC last time I was using tor (at least 2 years ago), it seemed to me like I had to enter that captcha more than once.

  6. #6

    Default

    Quote Originally Posted by TurnOfTide View Post
    Also, once you complete the captcha... does it keep asking for it over and over or does it sort of give you some more chances to ask for more queries before another captcha? .....IIRC last time I was using tor (at least 2 years ago), it seemed to me like I had to enter that captcha more than once.
    It depends on how the level of aggressiveness and the specific queries. Sometimes, you'll get one that lasts for a couple of pages of 10, sometimes you get hit once per page.

    Use 100 results per page to save money on decaptcher.

  7. #7
    admin KOiN's Avatar
    Join Date
    Jul 2005
    Posts
    29,773

    Default

    Quote Originally Posted by Audio View Post
    yep, google does hit you with a captcha after a ton of queries too fast
    PM me with forum questions or concerns and I will do my best to help

  8. #8

    Join Date
    Jul 2005
    Location
    south FL.
    Posts
    3,406

    Default

    Use proxies, rotate thru google hosts, if possible stray from using any inurl: , intext: , intitle:, and add a decent pause between pages... other than that, use bing/yahoo.

    Code:
    www.google.com
    216.239.51.100
    216.239.51.101
    216.239.51.104
    216.239.51.107
    216.239.51.115
    216.239.51.184
    216.239.51.19
    216.239.51.44
    216.239.51.81
    216.239.51.83
    216.239.51.84
    216.239.51.99
    216.239.59.100
    216.239.59.101
    216.239.59.102
    216.239.59.103
    216.239.59.104
    216.239.59.107
    216.239.59.115
    216.239.59.147
    216.239.59.17
    216.239.59.18
    216.239.59.184
    216.239.59.19
    216.239.59.44
    216.239.59.81
    216.239.59.83
    216.239.59.84
    216.239.59.99
    64.233.161.100
    64.233.161.101
    64.233.161.102
    64.233.161.104
    64.233.161.107
    64.233.161.115
    64.233.161.147
    64.233.161.18
    64.233.161.184
    64.233.161.19
    64.233.161.44
    64.233.161.81
    64.233.161.83
    64.233.161.84
    64.233.161.91
    64.233.161.99
    64.233.167.104
    64.233.167.107
    64.233.167.115
    64.233.167.147
    64.233.167.18
    64.233.167.184
    64.233.167.19
    64.233.167.44
    64.233.167.81
    64.233.167.83
    64.233.167.84
    64.233.167.91
    64.233.167.99
    64.233.169.104
    64.233.169.107
    64.233.169.115
    64.233.169.184
    64.233.169.19
    64.233.169.44
    64.233.169.81
    64.233.169.83
    64.233.169.84
    64.233.169.99
    64.233.171.100
    64.233.171.101
    64.233.171.102
    64.233.171.104
    64.233.171.107
    64.233.171.115
    64.233.171.147
    64.233.171.18
    64.233.171.184
    64.233.171.19
    64.233.171.44
    64.233.171.81
    64.233.171.83
    64.233.171.84
    64.233.171.99
    64.233.179.100
    64.233.179.101
    64.233.179.104
    64.233.179.107
    64.233.179.115
    64.233.179.184
    64.233.179.19
    64.233.179.44
    64.233.179.81
    64.233.179.83
    64.233.179.84
    64.233.179.91
    64.233.179.93
    64.233.179.99
    64.233.183.103
    64.233.183.104
    64.233.183.107
    64.233.183.115
    64.233.183.147
    64.233.183.17
    64.233.183.18
    64.233.183.184
    64.233.183.19
    64.233.183.44
    64.233.183.81
    64.233.183.83
    64.233.183.84
    64.233.183.91
    64.233.183.93
    64.233.183.99
    64.233.185.100
    64.233.185.101
    64.233.185.104
    64.233.185.107
    64.233.185.115
    64.233.185.184
    64.233.185.189
    64.233.185.19
    64.233.185.44
    64.233.185.81
    64.233.185.83
    64.233.185.84
    64.233.185.99
    64.233.187.104
    64.233.187.107
    64.233.187.115
    64.233.187.184
    64.233.187.19
    64.233.187.44
    64.233.187.81
    64.233.187.83
    64.233.187.84
    64.233.187.91
    64.233.187.93
    64.233.187.99
    64.233.189.104
    64.233.189.107
    64.233.189.115
    64.233.189.44
    64.233.189.81
    64.233.189.83
    64.233.189.84
    66.102.1.104
    66.102.1.107
    66.102.1.115
    66.102.1.184
    66.102.1.44
    66.102.1.81
    66.102.1.83
    66.102.1.84
    66.102.1.91
    66.102.1.93
    66.102.9.100
    66.102.9.101
    66.102.9.102
    66.102.9.104
    66.102.9.107
    66.102.9.115
    66.102.9.147
    66.102.9.18
    66.102.9.184
    66.102.9.81
    66.102.9.83
    66.102.9.84
    66.102.9.99
    66.249.81.100
    66.249.81.101
    66.249.81.102
    66.249.81.104
    66.249.81.107
    66.249.81.115
    66.249.81.147
    66.249.81.18
    66.249.81.19
    66.249.81.44
    66.249.81.81
    66.249.81.83
    66.249.81.84
    66.249.81.99
    66.249.83.100
    66.249.83.101
    66.249.83.104
    66.249.83.107
    66.249.83.115
    66.249.83.184
    66.249.83.19
    66.249.83.44
    66.249.83.81
    66.249.83.83
    66.249.83.84
    66.249.83.99
    66.249.89.104
    66.249.89.107
    66.249.89.115
    66.249.89.184
    66.249.89.19
    66.249.89.44
    66.249.89.81
    66.249.89.83
    66.249.89.84
    66.249.89.99
    66.249.91.104
    66.249.91.107
    66.249.91.115
    66.249.91.147
    66.249.91.18
    66.249.91.184
    66.249.91.19
    66.249.91.44
    66.249.91.81
    66.249.91.83
    66.249.91.84
    66.249.91.91
    66.249.91.93
    66.249.91.99
    66.249.93.100
    66.249.93.101
    66.249.93.104
    66.249.93.107
    66.249.93.115
    66.249.93.184
    66.249.93.19
    66.249.93.44
    66.249.93.81
    66.249.93.83
    66.249.93.84
    66.249.93.91
    66.249.93.93
    66.249.93.99
    72.14.205.100
    72.14.205.101
    72.14.205.102
    72.14.205.104
    72.14.205.107
    72.14.205.115
    72.14.205.147
    72.14.205.184
    72.14.205.189
    72.14.205.19
    72.14.205.44
    72.14.205.81
    72.14.205.83
    72.14.205.84
    72.14.205.99
    72.14.207.100
    72.14.207.101
    72.14.207.104
    72.14.207.107
    72.14.207.115
    72.14.207.184
    72.14.207.19
    72.14.207.44
    72.14.207.81
    72.14.207.83
    72.14.207.84
    72.14.207.91
    72.14.207.99
    72.14.209.101
    72.14.209.104
    72.14.209.107
    72.14.209.115
    72.14.209.184
    72.14.209.19
    72.14.209.44
    72.14.209.81
    72.14.209.83
    72.14.209.84
    72.14.209.99
    72.14.211.100
    72.14.211.101
    72.14.211.104
    72.14.211.107
    72.14.211.115
    72.14.211.184
    72.14.211.19
    72.14.211.44
    72.14.211.81
    72.14.211.83
    72.14.211.84
    72.14.211.99
    72.14.215.101
    72.14.215.104
    72.14.215.107
    72.14.215.115
    72.14.215.184
    72.14.215.19
    72.14.215.44
    72.14.215.81
    72.14.215.83
    72.14.215.84
    72.14.215.99
    72.14.217.101
    72.14.217.102
    72.14.217.104
    72.14.217.107
    72.14.217.115
    72.14.217.147
    72.14.217.184
    72.14.217.19
    72.14.217.44
    72.14.217.81
    72.14.217.83
    72.14.217.84
    72.14.217.99
    72.14.219.100
    72.14.219.104
    72.14.219.107
    72.14.219.115
    72.14.219.44
    72.14.219.81
    72.14.219.83
    72.14.219.84
    72.14.221.101
    72.14.221.102
    72.14.221.104
    72.14.221.107
    72.14.221.115
    72.14.221.147
    72.14.221.18
    72.14.221.184
    72.14.221.19
    72.14.221.44
    72.14.221.81
    72.14.221.83
    72.14.221.84
    72.14.221.91
    72.14.221.93
    72.14.221.99
    72.14.223.100
    72.14.223.101
    72.14.223.102
    72.14.223.104
    72.14.223.107
    72.14.223.115
    72.14.223.147
    72.14.223.18
    72.14.223.184
    72.14.223.19
    72.14.223.44
    72.14.223.81
    72.14.223.83
    72.14.223.84
    72.14.223.99
    72.14.235.100
    72.14.235.101
    72.14.235.102
    72.14.235.104
    72.14.235.107
    72.14.235.115
    72.14.235.147
    72.14.235.18
    72.14.235.184
    72.14.235.19
    72.14.235.44
    72.14.235.81
    72.14.235.83
    72.14.235.84
    72.14.235.99
    72.14.253.100
    72.14.253.101
    72.14.253.102
    72.14.253.104
    72.14.253.107
    72.14.253.115
    72.14.253.147
    72.14.253.18
    72.14.253.184
    72.14.253.19
    72.14.253.44
    72.14.253.81
    72.14.253.83
    72.14.253.84
    72.14.253.99

  9. #9

    Default

    If you're using your own OCR, go after certain localizations of Google, as the captchas will be [0-9] only.

    I know this works on the Russian and Ukranian Googles, although I'm not sure about the others.
    Busy traveling. Your inquiry is very important to us. Please hold and it will be answered in the order received.

  10. #10

    Default

    Run Hrefer without proxies and you'll see G bans IPs pretty fast. You can even hit the block manually by using some of the search terms pointed out by tric, seen it happen running certain FF plugins as well.

  11. #11
    xtldx's Avatar
    Join Date
    Apr 2007
    Location
    My Casa
    Posts
    2,144

    Default

    I have a project that scrapes google maps for mileage, and I sent about 10-15k queries a day. In addition all the computers here search google hundreds of times a day. I've only see google limit us a few times in the past 5 years, but I would assume I'm pumping another 10k search queries to them a day. All coming from the same IP. However, google could have whitelisted me as far as I know. Its all legitimate search traffic (except the map scraping).

  12. #12
    lol im band
    Join Date
    Feb 2010
    Posts
    28

    Default

    ...

  13. #13

    Default

    What if you use google's Web API?
    http://www.caliban.org/ruby/APIs_Ref...earch_requests

    Does anyone know if this is a better method?

  14. #14

    Join Date
    Jun 2009
    Location
    neptune
    Posts
    1,072

    Default

    rotate through user agents, and set it up with decaptcher and you should be good.

  15. #15

    Default

    Anyone have some rough estimates of how many queries (basic, no inurl or intext) can be done per minute? per hour?

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •