This is a demonstration of using both ZeroMQ and Mosquitto in order to illustrate passing messages from producers to consumers whilst additionally sending log messages from both components to a logging daemon.

• The source-code can be checked out from our subversion repository by issuing:

svn co http://svn.grimore.org/zeroSquitto

• zeromq
• mosquitto
• automake
• autoconf

The package can be built on both macOS / OSX and Linux. With the required packages installed, the software package can be compiled by issuing the commands:

./configure
make

and then installed with:

make install

In order to keep the system clean, to uninstall the programs, one would issue from the directory:

make uninstall

Assuming that Homebrew is installed, the following command will install the required packages:

brew install zeromq mosquitto automake autoconf

The build environment generates the following programs:

• zeroSquitto - the combined ZeroMQ and Mosquitto server.
• zeromqClient - a ZeroMQ client.
• zeromqServer - a ZeroMQ server.
• mosquittoClient - a Mosquitto / MQTT client.
• log-s - a logging server.

log-s is the logging component designed to accept messages over TCP from the mosquittoClient and zeromqClient components. It is the first component that has to be started, for instance, using the command:

log-s -n LOG-S -a 127.0.0.1 -p 2500

where:

• LOG-S is a descriptive name.
• 127.0.0.1 is the address to listen on.
• 2500 is the port to listen on.

log-s is designed as a TCP server using sockets with the additional benefit of sending the log messages both to the console and to syslog on *nix systems.

syslog has been chosen as the log destination because syslog represents the standard way for daemons to report messages with the additional benefit of being compatible with many tools that are able to extract and aggregate data.

log-s is capable of accepting messages from the components:

• zeroSquitto - the combined ZeroMQ and Mosquitto server.
• zeromqClient - a ZeroMQ client.
• mosquittoClient - a Mosquitto / MQTT client.

zeroSquitto is the combined ZeroMQ and Mosquitto server with the additional capability to send messages to the logging server log-s. The program can be started by issuing the command:

zeroSquitto -e tcp://*:1026 -a 127.0.0.1 -p 1883 --log-address 127.0.0.1 \
    --log-port 2500

where:

• tcp://*:1026

  - represents a ZeroMQ endpoint to listen on.

• 127.0.0.1 - represents the Mosquitto listening address.
• 1883 - represents the Mosquitto listening port.
• the second 127.0.0.1 represents the address of the logging component

log-s.

• 2500 represents the port of the logging component log-s.

zeromqClient is a ZeroMQ client - it is designed to send the message PULL to a ZeroMQ server, at which point the ZeroMQ server will respond with the data.

The program can be run by issuing the command:

zeromqClient tcp://127.0.0.1:1026 127.0.0.1 2500

where:

• tcp://127.0.0.1:1026

  represents a ZeroMQ endpoint to connect to.

• 127.0.0.1 is the address of a log-s server.
• 2500 is the port of a log-s server.

When started, zeromqClient will retrieve messages from the ZeroMQ server and additionally send log messages to the logging server log-s.

zeromqServer is a ZeroMQ server, similar to the zeromqClient program but will not send messages to log-s. zeromqServer is designed to work together with zeromqClient and does not require log-s to be started. In order to start zeromqServer, the following command can be issued:

zeromqServer tcp://*:1027/

where:

• ''tcp://*:1027/''

  is the ZeroMQ endpoint to listen on.

Conversely, zeromqClient can be started with:

zeromqClient tcp://127.0.0.1:1027 127.0.0.1 2500

in order to make zeromqClient and zeromqServer exchange messages through ZeroMQ.

mosquittoClient is a Mosquitto client / subscriber. Even though the the "subscriber" program is capable to read messages, mosquittoClient has been written to allow logging messages to the logging component log-s.

mosquittoClient can be started by issuing the command:

mosquittoClient 127.0.0.1 1883 temperature 127.0.0.1 2500

where:

• 127.0.0.1 is the address of the Mosquitto broker
• 1883 is the port of the Mosquitto broker
• the second 127.0.0.1 represents the address of the logging component

log-s.

• 2500 is the port of a log-s server.

zeromqServer and zeromqClient will work together and exchange messages through ZeroMQ. Additionally zeromqClient is able to send messages to log-s.

                    +---------+
                    | ZeroMQ  |
                    |         |
                    |         |
                    |         |
  +--------------+  |         |  +--------------+
  | zeromqServer +<------------->+ zeromqClient |
  +--------------+  |         |  +-------+------+
                    |         |          |
                    +---------+          |
                                         |
                    +---------+          |
                    |  LOG-S  +<---------+
                    +---------+

On the other hand, running zeroSquitto, zeromqClient, mosquittoClient and log-s will make zeroSquitto communicate with zeromqClient over ZeroMQ and mosquittoClient over Mosquitto.

                         +---------+
     +--------------+    | ZeroMQ  |
     | zeromqClient +<------------------------+
     +-------+------+    |         |          |
             |           +---------+          |
             |                                |
             |                                |
             +------------+                   |
                          v                   v
                     +----+----+      +-------+------+
                     |  LOG-S  +------|  zeroSquitto  |
                     +----+----+      +-------+------+
                          ^                   ^
           +--------------+                   |
           |                                  |
           |            +-----------+         |
           |            | Mosquitto |         |
  +--------+--------+   |           |         |
  | mosquittoClient +<------------------------+
  +-----------------+   |           |
                        +-----------+

All components use POSIX threads for performance and the ability to be interrupted whenever the program has to be terminated. For instance, the combined ZeroMQ and Mosquitto component zeroSquitto uses three separate threads: one for pushing messages to ZeroMQ, one for publishing to Mosquitto and a third and separate thread that sends messages to log-s.

Queues and circular queues are used to enqueue data that is then consumed by various threads. For instance, zeroSquitto will generate temperature readings and push the result onto two separate queues: one for ZeroMQ and another for Mosquitto. Whenever messages have to be logged, zeroSquitto enqueues the message onto a logging queue that is then processed by the logging thread and sent to a log-s server.

The non-blocking design of the code ensures that CPU usage is kept at a minimum whilst still allowing the program to be responsive.

Although both ZeroMQ and Mosquitto have APIs for C/C++, the project does not warrant the low-level capabilities of the C/C++ programming languages. For instance, most of the code deals with allocating buffers and processing or formatting messages to be sent over the network which is a task that can be repetitive and also extremely error-prone. It would be far more suitable to use a high-level programming language that contains the necessary constructs and could achieve the same semantics faster and with the additional benefit of code safety.

It seems that ZeroMQ and Mosquitto are not made to be mixed: ZeroMQ is made for posting data that is stored and can the be retrieved, whilst Mosquitto publishes data that can be lost in case there is no bound subscriber. Whilst Mosquitto seems to be suitable in situations where loss of data is acceptable, ZeroMQ seems by far the best choice when the data is not expendable. On the other hand, ZeroMQ has no concept of IP addresses (due to the different types of endpoints) - in fact, the documentation indicates that security concerns should be mitigated on a higher layer (ie: firewall) - this is unfortunate since the networking layer is rather complex and it would be highly desirable to be able to identify clients.