In computer science, message queues and mailboxes are software-engineering components typically used for inter-process communication (IPC), or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content. Group communication systems provide similar kinds of functionality. The message queue paradigm is a sibling of the publisher/subscriber pattern, and is typically one part of a larger message-oriented middleware system. Most messaging systems support both the publisher/subscriber and message queue models in their API, e.g. Java Message Service (JMS). Competing Consumers pattern enables multiple concurrent consumers to process messages on the same message queue.
Remit and ownership Message queues implement an asynchronous communication pattern between two or more processes/threads whereby the sending and receiving party do not need to interact with the message queue at the same time. Messages placed onto the queue are stored until the recipient retrieves them. Message queues have implicit or explicit limits on the size of data that may be transmitted in a single message and the number of messages that may remain outstanding on the queue.
Remit Many implementations of message queues function internally within an operating system or within an application. Such queues exist for the purposes of that system only. Other implementations allow the passing of messages between different computer systems, potentially connecting multiple applications and multiple operating systems. These message queuing systems typically provide resilience functionality to ensure that messages do not get "lost" in the event of a system failure. Examples of commercial implementations of this kind of message queuing software (also known as message-oriented middleware) include IBM MQ (formerly MQ Series) and Oracle Advanced Queuing (AQ). There is a Java standard called Java Message Service, which has several proprietary and free software implementations. Real-time operating systems (RTOSes) such as VxWorks and QNX encourage the use of message queuing as the primary inter-process or inter-thread communication mechanism. This can result in integration between message passing and CPU scheduling. Early examples of commercial RTOSes that encouraged a message-queue basis to inter-thread communication also include VRTX and pSOS+, both of which date to the early 1980s. The Erlang programming language uses processes to provide concurrency; these processes communicate asynchronously using message queuing.
Ownership The message queue software can be either proprietary, open source or a mix of both. It is then run either on premise in private servers or on external cloud servers (message queuing service).
Proprietary options have the longest history, and include products from the inception of message queuing, such as IBM MQ, and those tied to specific operating systems, such as Microsoft Message Queuing (MSMQ). Cloud service providers also provide their proprietary solutions such as Amazon Simple Queue Service (SQS), StormMQ, Solace, and IBM MQ. Open source choices of messaging middleware systems includes Apache ActiveMQ, Apache Kafka, Apache Qpid, Apache RocketMQ, JBoss Messaging, RabbitMQ, Sun Open Message Queue, and Tarantool. Examples on hardware-based messaging middleware vendors are Solace, Apigee, and IBM MQ.
Usage In a typical message-queueing implementation, a system administrator installs and configures message-queueing software (a queue manager or broker), and defines a named message queue. Or they register with a message queuing service. An application then registers a software routine that "listens" for messages placed onto the queue. Second and subsequent applications may connect to the queue and transfer a message onto it. The queue-manager software stores the messages until a receiving application connects and then calls the registered software routine. The receiving application then processes the message in an appropriate manner. There are often numerous options as to the exact semantics of message passing, including:
Durability – messages may be kept in memory, written to disk, or even committed to a DBMS if the need for reliability indicates a more resource-intensive solution. Security policies – which applications should have access to these messages? Message purging policies – queues or messages may have a "time to live". Message filtering – some systems support filtering data so that a subscriber may only see messages matching some pre-specified criteria of interest. Delivery policies – do we need to guarantee that a message is delivered at least once, or no more than once? Routing policies – in a system with many queue servers, what servers should receive a message or a queue's messages? Batching policies – should messages be delivered immediately? Or should the system wait a bit and try to deliver many messages at once? Queuing criteria – when should a message be considered "enqueued"? When one queue has it? Or when it has been forwarded to at least one remote queue? Or to all queues? Receipt notification – A publisher may need to know when some or all subscribers have received a message. These are all considerations that can have substantial effects on transaction semantics, system reliability, and system efficiency.
Standards and protocols Historically, message queuing has used proprietary, closed protocols, restricting the ability for different operating systems or programming languages to interact in a heterogeneous set of environments. An early attempt to make message queuing more ubiquitous was Sun Microsystems' JMS specification, which provided a Java-only abstraction of a client API. This allowed Java developers to switch between providers of message queuing in a fashion similar to that of developers using SQL databases. In practice, given the diversity of message queuing techniques and scenarios, this wasn't always as practical as it could be. Three standards have emerged which are used in open source message queue implementations:
… excerpt ends here. Continue reading the full article.
