problem description

customer table users has the following fields:

• there are 5 sales groups. Each sales group imports at least one batch of 100000 items of data into the table every month, and can only import the data of its own sales group
• within the same sales group, the mobile phone numbers of customers with normal status (state=0 or 1) cannot be repeated. Repeat discarding
• customer"s mobile phone number can be repeated among different sales groups
• the number of normal customers per sales group is estimated to be within 10 million

requires accurate and fast import into users table

methods and problems you have tried

• currently, the data in the users table is synchronized to redis according to the sales group ID, and the mobile phone number is saved with the set collection

SADD users:1 13100010001 13100010002 13100010003
SADD users:2 13100020001 13100020002 13100020003
SADD users:3 13100030001 13100030002 13100030003

• when importing, first write the excel table data to redis as file ID (assumed here is 8), and get the set collection mobiles:8
• then make the difference with the original set collection of the sales group users:1 to get the difference diff:1 .
• finally write the difference to mysql

SADD mobiles:8 13100010001 13100010004
SDIFFSTORE diff:1 mobiles:8 users:1

[ problem ] there are several places where the management side can add or delete users table, coupled with unknown reasons, resulting in data inconsistency between mysql and redis

what result do you expect?

is there a better way to import data into mysql accurately and quickly?