Added delay to TSearch.py to make the script more human

[wjamoe]

My list contains 600 items, ms will block after about 30 results.


I added a time.sleep(60) command in TSearch.py (and had to import time) to fool the (b i n g) search engine.

Each input line takes 60 seconds, it makes the script more human.....

So I don't have to enter lost of  subselections (600=20x30 input items) 



See both changes in the code below between the line with ======== on it

Code: Python [Select]

1. #-------------------------------------------------------------------------------
2. # imports we need
3. # include top level directory (needed to import from helper below)
4. import sys
5. import os
6. #201302 wjamoe added import time ===================================
7. import time
8. #=============================================================

Code: Python [Select]

1. def DoSearch(self,id,label,search):
2.         # this function does the main work -- take a search term
3.         # return a list of results
4.         results = []
5.  
6.         #initialize result with engine name and id
7.         baseresult = {"engine":self.get_searchenginename()}
8.         if id!="":
9.             baseresult["id"]=id
10.         if label!="":
11.             baseresult["label"]=label
12.  
13.         # ok grab a web page using our search terms
14.         content = self.SearchWebPageGrabContents(search)
15.  
16.         # and now parse results
17.         addresults = self.ParseWebPageContentIntoResults(search,content,baseresult)
18.         #201302 wjamoe added delay of 60 seconds =============================
19.         time.sleep(60)
20.         #=============================================================
21.         # add
22.         results.extend(addresults)
23.         # return
24.         return results

The testrun for 2 items takes 120 seconds 

wiem

[Ath]

Disclaimer: I don't know how to program in python (at all, yet)
Couldn't you keep start-time, fire 5 or 10 searches, then wait until 60 seconds have passed since start-time, and fire another set of searches? Would take much less waiting. But it would also look/feel less human-like searching...

[wjamoe]

That would require to reshuffle code, and will take more time to program and a lot more testing to find out what the heuristics are within the search engines. The engines probably all differ in that respect to identify robots and alike.
Current solution suffices for me, sorry...

[mouser]

I'm just thrilled that someone other than me is using this tool.