RabbitMQ Message Acknowledgement Mechanism
When using RabbitMQ, we can use message persistence to solve the problem of message loss due to abnormal server crash, but we also encounter a problem, when the message publisher sends the message out, does the message reach the broker proxy server correctly or not? Without special configuration, by default the publish operation will not return any information to the producer, that is, by default our producer does not know whether the message has reached the broker correctly, if the message has been lost before reaching the broker, the persistence operation will not solve this problem, because the message has not reached the proxy server, how do you persist, so how to solve this problem?
RabbitMQ provides us with two ways to do this.
implemented through the amqp transaction mechanism, which is the solution provided at the AMQP protocol level.
This is achieved by setting the channel to confirm mode.
There are three methods related to the transaction mechanism in RabbitMQ: txSelect(), txCommit() and txRollback(). txSelect is used to set the current channel into transaction mode, txCommit is used to commit the transaction, and txRollback is used to roll back the transaction.
Four additional steps with matters.
client sends Tx.
broker sends Tx.Select-Ok (followed by publish)
client sends Tx.
broker sends Tx.Commit-Ok
Using a transactional mechanism can severely reduce RabbitMQ's message throughput, so it is common to use an acknowledgement mode.
Implementation principle of confirm mode on the Producer side
The producer sets the channel to confirm mode. Once the channel enters confirm mode, all messages posted on the channel are assigned a unique ID (Correlation Id starting at 1). Once the message has been delivered to all matching queues, the broker sends an acknowledgement to the producer (containing the unique ID of the message), which lets the producer know that the message has reached the destination queue correctly.
The great thing about the confirm pattern is that it is asynchronous. Once a message is published, the producer application can continue sending the next message while waiting for the channel to return an acknowledgement.
After the channel is set to confirm mode, all subsequent messages that are published will be confirmed (i.e. ack) or nacked once. But there is no guarantee on how quickly a message will be confirmed, and the same message will not be both confirmed and nacked.
A channel that is already in transaction mode cannot be set to confirm mode, i.e. the two modes cannot coexist.
Message acknowledgement (Consumer side)
The broker will CONFIRM the message in the following cases.
basicRecover: is that messages that are not routed successfully can be resent to the queue using recovery.
basicReject: is the receiving end to tell the server that I refuse to receive this message, do not process, you can set whether to put back in the queue or throw away, and can only once reject a message, the official website has a clear statement can not be bulk rejection of messages, in order to solve the bulk rejection of messages to have a basicNack.
basicNack: N messages can be rejected at once. The client can set the multiple parameter of the basicNack method to true, and the server will reject all unacknowledged messages with the specified delivery_tag (tag is a 64-bit long value, the maximum value is 9223372036854775807).
If it takes a few seconds to process a message, you might think that if a consumer server, network, or network card fails and hangs while the message is being processed, then maybe that message or task being processed is not completed and the message and task is lost.
To ensure that messages or tasks are not lost, RabbitMQ supports message acknowledgement-ACK. The ACK mechanism is that the consumer side receives a message from RabbitMQ and finishes processing it and gives feedback to RabbitMQ, which receives the feedback before removing this message from the queue. If a consumer hangs while processing a message (network instability, server exception, website failure, etc. resulting in a channel, connection closure, or TCP connection loss, etc.), then he will not have ACK feedback and RabbitMQ will assume that the message was not consumed properly and will put this message back into the queue. If there are other consumers online at the same time, RabbitMQ will immediately push this message to this online consumer. This mechanism ensures that no messages or tasks are lost in the event of a consumer server failure.
If the consumer server hangs when RabbitMQ sends a message to a consumer, the message will not have a timeout; even if a message takes a very long time to process, it will not cause the message to time out. This way the messages are never removed from the RabbitMQ server. A message is removed from the RabbitMQ server data only when the consumer correctly sends an ACK confirmation feedback and RabbitMQ acknowledges receipt.
The ACK acknowledgement mechanism for messages is turned on by default.
There are 2 modes of message acknowledgement, automatic and manual.
In auto-acknowledgement mode, the message is considered to be successfully sent immediately after it is sent; this mode allows for higher throughput, but is indeed insecure. If the consumer's TCP connection or channel is closed before successful delivery, the message sent by the server will be lost. Another thing to consider when using automatic confirmation mode is consumer overload. Manual acknowledgement mode can use prefetch to limit the number of outstanding ("in-progress") sends on the channel. However, in the case of automatic confirmation, there is no such restriction.
Prefetch count values in the range of 100 to 300 typically provide the best throughput and do not run a significant risk of overwhelming the consumer.
Forgot to confirm
Forgetting to return a confirmation message via basicAck is a common error. This error is very serious and will cause messages to be returned to the RabbitMQ server after the consumer client exits or shuts down, which will fill up the RabbitMQ server memory and RabbitMQ will not actively delete these returned messages. To monitor for such errors, you can use the rabbitmqctl messages_unacknowledged command to print out the relevant information.
When the autoAck parameter is set to false, for the RabbitMQ server, the messages in the queue are split into two parts: those that are waiting to be cast to the consumer, and those that have been cast to the consumer but have not yet received the consumer's ack. If RabbitMQ never receives an acknowledgement signal from a consumer and the consumer consuming this message has disconnected, RabbitMQ schedules the message to re-enter the queue and wait to be cast to the next consumer, which may of course still be the same consumer.
RabbitMQ does not set an expiration time for unacknowledged messages; it determines whether the message needs to be recast to the consumer based solely on whether the consumer connection consuming the message has been disconnected, and is designed this way because RabbitMQ allows consumers to consume a message for as long as they want.
If the message consumption fails, you can also call Basic.Reject or Basic.Nack to reject the current message instead of acknowledging it, if it is simply rejected then the message will be lost and the corresponding requeue parameter needs to be set to true, then RabbitMQ will re-queue this message so that it can be sent to the next subscribed consumer. If the requeue parameter is set to false, RabbitMQ immediately removes the message from the queue and does not send it to a new consumer.