How to correctly use redis queue to deal with php second kill concurrency problem?

Baidu found this way of thinking:
needs a queue, a queue of panic buying results and a queue of inventory. In the case of high concurrency, first put the user into the queue, use a thread loop to remove a user from the queue, and judge whether the user is already snapping up the result queue, if so, he has snapped up, otherwise the inventory is not snapped up, and the inventory is reduced by 1. Write to the database and put the user into the result queue.

"first put the user into the queue and use a thread loop to extract a user from the queue."
how to "loop with a thread", I don"t understand how to start and when to start this "thread"?

is it true that every time the user draws a lottery, the access to the main program immediately starts the "thread" to cycle through the queue data?

or is it that the user clicks the lottery button and triggers the queuing operation?
then, there will be an independent process started by cli on the server. In the case of subscribing to the listening queue for a long time, if there is any data, the user queue will be dequeued, and then the order will be issued.

queuing and dequeuing should be called by two different programs, right? So how can you successfully join the team and get out of the queue as soon as the user visits the lottery, and how to arrange these invocation actions at the code level?


the following solutions are recommended not to be used. RabbitMQ is easy to install and easy to use, which is much more direct than redis queue, and you don't have to consider most of the problems.

"loop processing with one thread", I don't know how to start, when to start this "thread"?

  • my plan is to first write a program that specifically processes the data in the queue and call this program every minute with a scheduled task (linux's crontab).

this program uses blPop or brPop to block access to list data (the difference between these two methods can be seen in the redis document). The reason for blocking is that in case there is no data in this minute and the data comes in 30 seconds later, you have to wait another 30 seconds to process. I set the maximum time for blocking for 59 seconds. At the same time, in order to be compatible with a large amount of data, the program will cycle to read data from list, each time the data is read in a blocking way, so the blocking time of each time will dynamically change according to the length of time the current program is running. In theory, if the business is not complex, the program will not run more than 60 seconds at a time, which will meet the requirements, and if it exceeds 60 seconds, new threads will be automatically added to execute the next loop.

you don't understand the next question. You know the big process. Let me tell you the details of this process

I use redis's list, and other answers mention using locks. If it's your solution, you don't need to use locks at all, because list has solved the concurrency problem for you.

as long as the user clicks "grab", you put the user's information lPush or rPush into a list, and you will return an int, that is, after this operation, how many pieces of data are in the list. This int is thread-safe, and no matter how high the concurrency is, it will not cause the int to be out of date for this user, so you can judge whether the int has exceeded the inventory. Directly tell the front end that the user missed the second kill, if not, let the front end wait for the result of the rush purchase. (there is no need to remove users who are out of stock from list at this time. It is recommended that the inventory number be queried before the second kill and put into redis, and then do not modify the inventory number of redis, because this inventory number is specially used to compare with the length of list)

the next steps are based on different business requirements. If you next ask the user to fill in supplementary information, it is the simplest: write an API to receive the supplementary information from the user, query the ranking of the user in list, and compare it with the inventory to see if he is really killed in seconds, and then do the storage operation.

if there is no need for the user to operate next, write an interface that blocks polling, as you did to answer your first question above.


Redis queue can set the queue length. When a user connects to the server and the Redis queue length is + 1, note that the data structure of the queue is first-in, first-out. When the queue length reaches the length you set, Redis is not allowed to join the queue.


when there is high concurrency, you can use locks to control panic buying or lottery draw. If you get the lock, you can make purchases. If you do not get the lock, you will return immediately. As you said, people who rush to buy will join the queue for panic buying. There is no sequence in it. You can't wait for the first come first. There are tens of millions of people who rush to buy. Do you want to build a queue of tens of millions of people?. Besides, I don't know how long the customer will have to wait until the queue is newly added.


$ttl = 4;
$random = mt_rand(1,1000).'-'.gettimeofday(true).'-'.mt_rand(1,1000);

$lock = fasle;
while (!$lock) {
    $lock = $redis->set('lock', $random, array('nx', 'ex' => $ttl));
}

if ($redis->get('goods.num') <= 0) {
    echo ("");
    //
    if ($redis->get('lock') == $random) {
        $redis->del('lock');
    }
    return false;
}

$redis->decr('goods.num');
echo ("");
//
if ($redis->get('lock') == $random) {
    $redis->del('lock');
}
return true;

Source http://coffeephp.com/articles.


I would also like to know the answer


the second kill doesn't need to be so complicated, it only needs a unique lock to increase atoms, and then judge whether it exceeds the maximum quantity according to the number of rush purchases in feedback. If it exceeds, it will indicate that there is no quantity.


if it is hundreds of post times per second, my idea is:

  1. insert first, including millisecond timestamps, or directly use self-incrementing ID
  2. then sort to determine which one is inserted. If it exceeds the total quantity, the rush purchase fails, and the rush purchase is changed to finished.

use a thread loop, I don't know how to do it, and when to start this thread
  1. this can be executed before the second kill starts, and the execution process runs through the whole second kill process, which can be several processes or a process running all the time. This is the process of leaving the team. If you join the queue, it is the customary mode of PHP, which automatically starts the process and throws data into the queue one request at a time.
  2. then we need to understand what the meaning of the queue is and what the meaning of redis is. Redis is used to carry and send, through a counter, first query there is inventory on the implementation of the queue, inventory deduction on the direct return. Redis carries a layer of backward queue data is much smaller, the essence of the queue is to help DB carry and use concurrently, so that DB transactions to implement all serialization, to avoid lock contention to reduce DB performance.

that's what I did.
for example, you have 1000 second kill products
that's the design,

everybody go to the page waiting for the second kill, click the second kill button, and then ajax requests an api interface. The core code of this interface is

.
$number = $redis->incr($key);
if ($number > 1000) {
    return '';
} else {
    $_SESSION['flag'] = 1;
    return redirect('');
}

in this way, the person who gets it can slowly fill in the information on the next page, and the person who doesn't get it can give up on the current page.

MySQL Query : SELECT * FROM `codeshelper`.`v9_news` WHERE status=99 AND catid='6' ORDER BY rand() LIMIT 5
MySQL Error : Disk full (/tmp/#sql-temptable-64f5-1ea42a0-1cb4.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
MySQL Errno : 1021
Message : Disk full (/tmp/#sql-temptable-64f5-1ea42a0-1cb4.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Need Help?