Receive a large amount of data from hardware devices in real time, and what are the recommended storage methods for large data?

problem description

there are hundreds of devices, and each device produces some real-time data every second. These data will be analyzed and used later. What is the best way to store these data?

the environmental background of the problems and what methods you have tried

previously stored in mysql, Aliyun"s mysql storage is also limited or upgraded. The key is that even if the upgrade data is large, the speed is slow. Is there any better way to store this data, which is convenient for storage and analysis later? If saving to a file

Go java python php

Jul.05,2022

MongoDB can or can consider

if data loss is allowed, consider using ELK collection and storage.

mysql stores relational data, such as device ID, time, etc.
specific data is recommended to be stored in Aliyun's OSS, which ensures that the query is convenient, and the MYSQL load is not in the order of magnitude of

million per second. It is recommended to use hbase for storage
since you do not require real-time analysis, you can upload batch data with local cache in the device
if you need to use message queue. kafka

Next: Centos in VMware mounts the shared directory of win10. How to solve the case-insensitive problem?

MySQL Query : SELECT * FROM `codeshelper`.`v9_news` WHERE status=99 AND catid='6' ORDER BY rand() LIMIT 5
MySQL Error : Disk full (/tmp/#sql-temptable-64f5-1b36f6e-2c04c.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
MySQL Errno : 1021
Message : Disk full (/tmp/#sql-temptable-64f5-1b36f6e-2c04c.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Need Help?