What should the vuejs, front end do if it saves a large amount of data?
upload financial bills. The front end parses nearly 300000 data when uploading files. When you click the upload button, you need to send these 300000 pieces of data to the backend.
now you will report Message length exceeded maximum allowed length. when uploading, and the browser is very slow when parsing files. Is there any good solution?
onSelectOriginalFile(result) {
this.notifySuccessParsing();
this.data = UploadDataParser.parseShipmentCheck(result);//
}
submit() {
ReconciliationResource.import(this.data);//
}
it's simple to leave the parsing to the back end, and the front end will upload the whole file. Js is not suitable for CPU-intensive tasks, you use for or while to parse files is very CPU, unless you use WebAssembly (but I have not tried), and the process is Synchronize, so you will feel stuck.
so the solution is to pass the file to the back end, which is parsed in C or other more appropriate language, and then page back to the front end or something.
if you must burn the customer's CPU at the front end, you can consider the following things:
- change the client, instead of using a normal browser, use
Electron ;
- try
WebAssembly parse the file;
- optimize the file structure and speed up parsing. If you insist on using an ordinary browser as a client, you can first convert the file to json format (specifically write a tool or something, I don't know what your original format is), and then the front end
JSON.parse , but it will take a long time to parse if the file is too large.
in addition, I have never encountered the problem that the information is too large to be sent at once, but it should not be very troublesome. After all, it is no problem for the network disk to send large files. Check this hhh yourself. All I can think of now is that I can stream upload using WebSocket .
javascript itself is single-threaded, suitable for IO operations, not suitable for intensive computing work.
however, it is not very reliable to do everything in the backend, because the backend resources are limited.
to solve the stutter problem, try worker
File upload to the backend for fast processing
