需要某網站資料抓回來統計,結果當然沒我想的那麼簡單, 程式跑一跑就撈回來
光這裡就卡好久, 後來觀察cookie 從 cf_clearance 找到這篇文章
目前只能做到半自動, 使用firefox 建立session後
如圖複製cURL指令,產生bat檔 ,再執行bat,將3x 頁面抓回來解析
import re
inputstr = input("paste here:")
inputstr = inputstr.replace("AgencyDepartment=^%^E4^%^B8^%^AD^%^E5^%^B8^%^82", "AgencyDepartment=中市")
inputstr = inputstr.replace("--compressed","")
inputstr = inputstr.replace("-H \"Accept-Encoding: gzip, deflate, br, zstd\"","")
runFile = open("run.bat","w")
for i in range(1,38):
fileName = str(i)+".html"
command = re.split(r'Index\/\d', inputstr)[0] + "Index/" + str(i) + re.split(r'Index\/\d', inputstr)[1] + " -o " + fileName
runFile.write(command)
runFile.write("\n")
runFile.write("timeout 2")
runFile.write("\n")
runFile.close()