as in the title, write a simple function test to generate a soup object from the URL using Python requests and BeautifulSoup, (see the example below). If you call this function directly in the main thread, everything will be fine, but if you call this function in a child thread, you will have log information such as encoding error: input conversion failed due to input error, bytes 0x95 0x50 0x22 0x20
, and it will be output directly to stdout, and you will not be able to catch it without throwing an exception.
is there any way to solve this problem?
here is an example that can be reproduced.
from threading import Thread
import requests
from bs4 import BeautifulSoup
def test():
r = requests.get("http://zhuanlan.sina.com.cn/")
soup = BeautifulSoup(r.content,"lxml")
print("test")
test()
print("test")
t = Thread(target=test)
t.start()
t.join()
the output is as follows
test
test
encoding error : input conversion failed due to input error, bytes 0x95 0x50 0x22 0x20
encoding error : input conversion failed due to input error, bytes 0x95 0x50 0x22 0x20
encoding error : input conversion failed due to input error, bytes 0x95 0x50 0x22 0x20