has encountered a difficult problem recently.
I use redis for message queuing. Each queue contains a task id,
the simple process is as follows:
1. Read id, consumption from list
2. Read the details of this task from the database according to id,.
the data structure is as follows: for example,
ID=1 URL:abc.com
ID=2 URL:abd.com
ID=3 URL:ab3.com
.
I need to do a function now.
controls the access interval for each url"s domain name. For example, abc.com and efg.com are included in the url in the task
I need to control the consumption of one task containing abc.com every 5 seconds and one task containing efg.com every 10 seconds
what if you can accomplish this function while taking into account the performance?
< hr >add a condition: distributed, millions of queues, dozens of servers
< hr > come back after a short time to look at this problem. There are many articles on the redis delay queue.
the professional message queuing rabbitmq also has tutorials in this area.
-0-
it took me a year to find the right answer. It was disgusting.
answer: https://codeshelper.com/q/10.