-sharpcoding:utf-8
from collections import Counter
import requests
from bs4 import BeautifulSoup
from urllib import request
import urllib
url = "https://www.baidu.com.cn/s?wd=" + urllib.parse.quote("python3") + "&pn=" -sharp wordpn..
headers = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36",
"Connection": "keep-alive",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"}
r=requests.get(url,headers=headers)
soup = BeautifulSoup(r.text.replace("<b>", "").replace("</b>", ""),"lxml")
url_list=soup.select("div.result > h3 > a")
for i in url_list:
print(i.getText())
the following are the first four results of searching python3, in the browser. When I use the above python3 program to take the top ten results, the results are inconsistent. I have tried many keywords. Which friend can tell me the reason? is it the problem of my program or what Baidu has done? thank you!