Performance problems in batch Import of MongoDB with Java

I am using Java to implement a function of bulk importing data into MongoDB.

my idea is to read the target content, convert it to a Document object, put it into List, and then import it into the library through the insertMany method of MongoDB Driver. The code is as follows:

clipboard.png

but now a performance problem has been found:

:26K+30MB+10

I locate the problem by myself and find that the time-consuming code is a block of circular statements, but I don"t know any other ways to implement it. I also ask you Daniel to give me some advice.

Jan.23,2022

list is wrapped with concurrency-safe vector or Collections.synchronizedList (List list), and then added in chunks and multithreaded. This kind of task is easy to divide, such as handing over to one thread every 2k.

MySQL Query : SELECT * FROM `codeshelper`.`v9_news` WHERE status=99 AND catid='6' ORDER BY rand() LIMIT 5
MySQL Error : Disk full (/tmp/#sql-temptable-64f5-1e5f5fe-4582c.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
MySQL Errno : 1021
Message : Disk full (/tmp/#sql-temptable-64f5-1e5f5fe-4582c.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Need Help?