OpenDLV and Libcluon
OpenDLV is a software framework mainly for Internet-of-things (IoT) and robotics applications. It pioneered the microservice-based architecture for autonomous systems, and has since its first release in 2015 grown in terms of available software services, number of users, and developers. The long-term goal of OpenDLV is to provide a large collection of microservices and supporting software, useful for many different types of projects. The microservices are running as small embedded software components with minimalistic footprint, capable of operating a large variety of devices and robots. With this goal in mind, the OpenDLV framework also aims to standardize data logging and communication between microservices. The standardized communication involves both low-level data from sensors and commands to actuators, but also high-level data connected to artificial intelligence and machine learning.
Note! A microservice is a small independent piece of software with a clearly defined input and output. Microservices can form a larger software system similarly to as how jigsaw puzzle pieces form a large picture. In a typical autonomous system there might be more than ten or twenty such services, each responsible for a specific hardware interface or internal logic. In general, each individual microservice runs without any knowledge of what other microservices are running, simply waiting for the expected input to produce the given output. Therefore, it is up to the engineer to design the sofware deployment in a way so that each microservice receives its needed input.
The microservice pattern is inherently modular by design and OpenDLV already includes many microservices for interfacing with sensors and actuators, as well as algorithms for path-planning, decision-making and control. The OpenDLV community is actively striving to increase functionality continuously, by adding more algorithms and tools over time, involving both physical hardware systems, virtual systems (simulation), and a mix between the two.
In addition, OpenDLV uses modern DevOps software engineering principles, to allow high quality software that is easy to both develope and deploy to devices of various architectures. The big benefit of building a large software framework according to a microservice architecture is that it inherently prevents so called technical debt as services never grow out of their own usage domain. Also they allow for efficient deployment and over-the-air updates (OTA) as only parts of the system needs to be redeployed on updates.
There are in general two ways of sharing data between microservices in OpenDLV, through the Ethernet network stack, typically via UDP multicast, and through shared memory. This tutorial will introduce the most basic case of Ethernet communication and in the same process introduce the microservice concept.
Ethernet: UDP and TCP
Before starting to learn how OpenDLV shares data between microservices, it is good to have a basic understanding of how data is sent over an Ethernet network, and some related terms. When picturing a set of microservices forming a larger system, for example running inside an autonomous robot, one can picture the communication either to be one-to-one between microservices, or one-to-many where one service, a producer, creates the data to then pass it on to several recipients, the consumers. Almost all microservices in OpenDLV are designed with a one-to-many mindset, and this is a special case that can be achieved with Ethernet communication.
For Ethernet there are in general two types of communication, UDP and TCP. In both concepts there is a server, or host, and a client. TCP is more formalized and heavy-weight, where the server and clients form strong link between themselves where each party knows if all sent data has been received and the status of the communication link. In UDP, the connections are less formalized and the communication is therefore more light-weight. The drawback of UDP is that there is no way for the connected parties to know if all data was received, or if the link is, for example, congested. For UDP there is also a possibility for broadcast and multicast of messages, the one-to-many concept as mentioned above, while TCP is strictly communication between two end-points.
Libcluon, a middleware for data transmission
As indicated above, OpenDLV is much about communication between different parts of a IoT or robotic system, either between software components (microservices) or to and from hardware components. Since this is such an important part of the system it is important to formalize it across each component and largely between software versions. For OpenDLV, the library Libcluon is used as a so called middleware to implement such formalized communication protocol. Libcluon is a minimalistic header-only library that is easy to integrate into each microservice, and its core functionality is kept rather constant over time reducing the risk of bugs and errors. Since all OpenDLV microservices uses the same library for communication, it is ensured that communication will always work in a unified way. The middleware supports both TCP and UDP connections, as well as shared memory as will be discussed in more detail in a later tutorial.
In general, the data sent internally in a system of microservices is of rather small volume, where low latency is a priority. Therefore it makes sense to use the one-to-many approach as offered by UDP, to simplify the integration of software components. This also allows a software deployment over several computers. However, in many cases it is a good idea to partition the communication so that microservices can form walled-in sub-systems. In Libcluon this is done by favoring multicast rather than broadcast, where multicast messages can be sent and received within what Libcluon refers to as conferences.
There are cases when UDP multicast is not suitable to use however, and that is when the data volume is to large to be sent through UDP datagrams. In some cases, TCP could be used instead if a one-to-one communication link across computers is needed. However, shared memory might be a better option since it allows for low latency transmission of very large volumes of data, but only within one computer (not between computers).
Using Libcluon to create a distributed system
This tutorial will continue with the software that was developed is the previous tutorial, with the prime checker software, but now the purpose is to step-by-step turn it into a microservice.
First, open a terminal and go to the data/mytest folder as:
cd data/mytest
wget command, according to:
wget https://raw.github.com/chrberger/libcluon/gh-pages/headeronly/cluon-complete.hpp
Then, open the source code to the program (gedit helloworld.cpp) and include
and use the Libcluon header-only library according to:
#include <iostream>
#include "cluon-complete.hpp"
#include "prime-checker.hpp"
int32_t main(int32_t, char **) {
PrimeChecker pc;
std::cout << "Hello world = " << pc.isPrime(43) << std::endl;
cluon::UDPSender sender{"127.0.0.1", 1234};
sender.send("Hello UDP world!");
return 0;
}
The first change is that the Libcluon library is included as
#include "cluon-complete.hpp". The included file is the entire Libcluon
library (header-only) and introduces all its functions. It is a formal
requirement that each OpenDLV microservice includes the library, to allow for
communication within the system.
The next change is that a UDP sender was introduced
cluon::UDPSender sender{"127.0.0.1", 1234};. As seen previously
cluon::UDPSender is a C++ class, and sender is an object created from this.
The arguments to the object, the string 127.0.0.1 and the integer 1234 is
used internally by the object when constructed.
All C++ classes can define a special member function called the constructor
where logic needed to form the object is located, and how the given constructor
arguments are used within the formed object. In the PrimeChecker class, no
arguments were required and therefore no constructor was needed.
However, the UDPSender class needs the arguments and could not be created
without. The meaning of the arguments in this case is that 127.0.0.1 is the IP
address and 1234 is the UDP port to where the sender should send. Note that
127.0.0.1 is a very special IP address that always refers to the local
computer, or localhost. In this case that means that the program will only
establish connections to other programs running on the same computer.
Lastly, the code sender.send("Hello UDP world!") was added, where send is a
method within the class UDPSender (and therefore also the object sender).
The method makes the object send the message "Hello UDP world!".
Save, close, and build the program (cd build and make). This time it will
take a bit more time to build, as Libcluon is also included into the program.
The program that you just compiled works as a client, it connects to an network
(IP) address and UDP port and sends a message. In order to test it there needs
to be a UDP server listening on that port, in order for the client to establish
a connection. There is a simple tool in Linux systems called Netcat (nc), that
can be used as a server when testing the program. The tool works for both TCP
and UDP and can act as both server and client. In this case Netcat should act as
a server and listen to incoming traffic, so that your program can send messages
to it. First, start by installing Netcat on your computer. In the terminal type
the following commands:
Note! If you are using OpenDLV Desktop, this is already included.
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install netcat
Then, open an additional (second) terminal from where you can run the Netcat server. Start Netcat with the following command:
nc -u -l -p 1234
-u indicates that Netcat should communicate over UDP, -l
that it should act as a server and listen for incoming data, and -p that it
should open port 1234 for incoming connections and data. In the original
(first) terminal, still in the build folder, start your program as:
./helloworld
Tip! When starting Netcat as a server it will be open indefinitely and wait for incoming data. In order to stop it you press Ctrl+c. This is the standard way to close a running program from the Linux terminal.
The next step is to extend the program to also receive data. The code for this
will be a bit more advanced, as receiving data also involves the aspect of
reacting to whatever data that comes in. Libcluon is a modern C++ library, and
therefore it naturally uses so called lambda functions to encapsulate logic
into a code structure that can be moved between functions. In essence, a lambda
function is a function assigned into a variable. Open the helloworld.cpp file
through the old terminal, either by doing gedit ../helloworld.cpp or move to
the mytest folder with cd .. and then open the file with
gedit helloworld.cpp. Add the following content:
#include <chrono>
#include <iostream>
#include "cluon-complete.hpp"
#include "prime-checker.hpp"
int32_t main(int32_t, char **) {
PrimeChecker pc;
std::cout << "Hello world = " << pc.isPrime(43) << std::endl;
cluon::UDPSender sender{"127.0.0.1", 1234};
sender.send("Hello UDP world!");
cluon::UDPReceiver receiver("0.0.0.0", 1235,
[](std::string &&data, std::string &&,
std::chrono::system_clock::time_point &&) noexcept {
std::cout << "Received " << data.size() << " bytes." << std::endl;
});
while (receiver.isRunning()) {
std::this_thread::sleep_for(std::chrono::duration<double>(1.0));
}
return 0;
}
On the first row, the new standard library chronos was included. This library
include functions related to time and time-based objects.
Tip! Below #include <iostream> a vertical space was added to make the code more readable. Always think about the reader when you write your code, because you are never only writing the code for yourself, anyone should easily be able to read your code.
Then the receiver part was added as the class cluon::UDPReceiver instantiated
into the object receiver with three constructor arguments; a string, an
integer, and a lambda function. The special IP address 0.0.0.0 means that the
receiver will accept connections from all client IPs, and the UDP port 1235
refers to where the receiver (server) will listen for incoming data. The next
four rows forms the lambda function, starting with its own three function
arguments, namely two strings and one timestamp. Of these three, only the first
one is used (named), and that is the incoming data. The second argument is IP
of the sender, and the third is the time when the message arrived. Then, after
the arguments there is a section that constitutes the function body, in this
case printing some information about the received data.
A large benefit of C++ and one of the reasons why it is suitable for software
that runs on IoT, robots, and other embedded systems is that it allows the
programmer to get very close to the computer hardware such as the memory. For
example, the && notation above is specific to C++ and relates to memory
access. One ampersand refers to the specific address in memory where data is
stored (a text string in this example), and when two ampersands are used it
instead refers to the address of that address. This can be exemplified by
having a piece of paper that states the address of someone's house, and then
another piece of paper that gives the location of the first piece of paper.
This way of using a reference to a reference is very specific to programming
APIs such as for Libcluon, and at this points it mainly servers as an example of
how versatile C++ is when telling the computer exactly how the program should
work according to the computer's capabilities. Note that the lambda function
needs to be written exactly as it is stated in the example, apart from the
function body of course, otherwise Libcluon would not be able to use it as
expected internally and would give an error.
This type of lambda function, that is given as input to another function, is often referred to as a callback since it is used to call back to us whenever something happens inside the library. In this case, the lambda function is called every time an external client is sending a message to the receiver (server).
Finally, a while loop as added to the code. As stated on the same row, the
condition for the loop to run is as long as the UDP receiver is active. This
will be the case until the program is stopped, for example using Ctrl+c. Then
inside the loop the code
std::this_thread::sleep_for(std::chrono::duration<double>(1.0)) is used from
the chrono library, and it will put the running program thread to sleep for
one second. This is a very important line to avoid the program going into a
state where it would run the loop at an uncontrolled speed, maxing out the CPUs
power. The reason why the loop is needed at all is that the receiver object
enters its own program thread where it stays to wait for incoming data. While
it waits for data you need to make sure that that the program does not stop,
hence the loop that runs for as long as the receiver is up and running.
Now, go ahead and save and close the file. Then, as before, make sure you are in
the build folder and run make to compile. You will directly notice that the
compiler suddenly give a linking error. Compilation is always done in three
steps; preprocessing, compiling, and then linking. The linking step is the part
of the building where different chunks of machine code are merged together into
a single program, if compiling a program that uses external pre-compiled
libraries. In this case, Libcluon itself is not a pre-compiled library, instead
a header-only library, but Libcluon uses a few standardized libraries
internally, and one is the library that handles program threads. To fix the
error, you need to allow the needed library in CMakeLists.txt. The same needs
for all pre-compiled libraries, and it is common that OpenDLV microservices use
a few different external libraries to be able to carry out its dedicated task.
Open the CMakeLists.txt file and change the following:
cmake_minimum_required(VERSION 3.2)
project(helloworld)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -Wextra")
find_package(Threads REQUIRED)
add_executable(${PROJECT_NAME} ${CMAKE_CURRENT_SOURCE_DIR}/helloworld.cpp
${CMAKE_CURRENT_SOURCE_DIR}/prime-checker.cpp)
target_link_libraries(${PROJECT_NAME} Threads::Threads)
enable_testing()
add_executable(test-prime-checker test-prime-checker.cpp
${CMAKE_CURRENT_SOURCE_DIR}/prime-checker.cpp)
add_test(NAME test-prime-checker COMMAND test-prime-checker)
There are two new rows in the file, the first one
find_package(Threads REQUIRED) tells CMake to find the Threads package and
that it is required to proceed with the compilation. Then the next new line is
target_link_libraries(${PROJECT_NAME} Threads::Threads) and that instructs
CMake to prepare the actual linking (using g++ under the hood as discussed
before). With this standard library linked to our compiled executable program
threads will be supported, and therefore the cluon::UDPReceiver class which
uses a thread internally to wait for incoming data. Now save and close the file.
Once again, try to compile and run your program (in the build folder run make,
make test, and ./helloworld).
It should now work, and the program will start by outputting Hello world = 1
then it will silently send the UDP message Hello UDP world! to port 1234
on the local machine as before, and finally it will start a UDP server on port
1235 and wait for any message until the user closes the program with Ctrl+c
To test that the server portion of the program works, we can once again use Netcat. Let the program run in the first terminal, and then use the second terminal to run the Netcat command, this time acting as a sender, according to:
echo "Hi there!" | nc -u 127.0.0.1 1235
echo command is to print the text Hi there!, which is then
followed by the pipe symbol |. The pipe symbol makes the output of the
program in front of the pipe turn into input to the program after the pipe, in
this case Netcat. The Netcat command is this time also used with the flag -u
meaning UDP mode, but this time there is no -l flag to start a server, so
instead Netcat act as a client connecting to the IP address 127.0.0.1, the
special address for the local host, and the UDP port 1235, the port that was
specified for the receiver inside the C++ program. This means that nc will
connect to the UDP server on the local machine listening to the UDP port 1235
and send whatever data given as input, in this case Hi there! which is then
received by the C++ program. The size of this message in bytes is then printed
to the terminal.
Turning to UDP multicast
Now when the basic send and receive functions of the program works it is time
to go ahead and make it into a microservice that is able to communicate
with other microservices on the same communication link. Then, it is no longer
desirable to have a single receiver and one or more senders, but each part of
the system should be able to act as both at the same time. The answer to this
is to use UDP broadcast or unicast, as several senders then can exists on the
same communication link. The program that you did so far is already prepared
for this, as it contained both a part for sending and receiving, but before on
different UDP ports (since two servers with the same port never can exist on
one-to-one communication). Now it is time to turn to UDP multicast, and
therefore also remove the need to use Netcat for testing the program. Open the
helloworld.cpp, either by doing gedit ../helloworld.cpp or move to the
mytest folder with cd .. and then open the file with gedit helloworld.cpp.
Add the following content:
#include <chrono>
#include <iostream>
#include "cluon-complete.hpp"
#include "prime-checker.hpp"
int32_t main(int32_t, char **) {
PrimeChecker pc;
std::cout << "Hello world = " << pc.isPrime(43) << std::endl;
cluon::UDPSender sender{"225.0.0.111", 1236};
sender.send("Hello UDP world!");
std::this_thread::sleep_for(std::chrono::duration<double>(3.0));
cluon::UDPReceiver receiver("225.0.0.111", 1236,
[](std::string &&data, std::string &&,
std::chrono::system_clock::time_point &&) noexcept {
std::cout << "Received " << data.size() << " bytes." << std::endl;
});
while (receiver.isRunning()) {
std::this_thread::sleep_for(std::chrono::duration<double>(1.0));
}
return 0;
}
1236, (2) both the sender and the receiver are now sending to
the special IP address 225.0.0.111 which is inside the range of multicast
addresses, and (3) there is a new delay of three seconds between the sender
and the receiver. Now, when the multicast address and the port is the same the
sender and the receiver should be able to connect and exchange data. Save and
close the file, and then compile again using make inside the build folder.
The next step is to start two instances of the same program to test that
messages can be sent in-between. However, from the design of the program, it is
clear that one program needs to be started first, then you need to wait three
seconds before the receiver is started. Then you can start the second program in
the second terminal and the message Hello UDP world! should be visible in the
first. Note that the message sent from the first terminal is never received
anywhere as there at that time is no active receiver. Go ahead and test the
program in two terminals, in both terminals making sure to be be located in the
build folder and run the ./helloworld command. Note that you do not need to
compile the program before running it in the second terminal. When you are done,
the programs can be exited by pressing Ctrl+c.
Here it might also be a good idea to reflect on that the multicast messages are actually sent over the entire local network, so the two programs could be stated on any two different computers on the network, and the same results would be achieved. This is great for a robotic system with many embedded computers, for example.
Introducing Libcluon messages
The next step is to introduce a more standardized way of sending data. What was done so far, using strings such as Hi there! is not a very organized way of sending data. Libcluon comes to the rescue here, including a standard way of sending and receiving various types of low-level data. This includes both ways of forming data inside the program, but also ways to transmit the data efficiently.
In order to begin creating these data structures it is easier to install Libcluon on the system, to access the needed tools. Run the following commands to install Libcluon:
Note! If you are using OpenDLV Desktop, this is already included.
sudo add-apt-repository -y ppa:chrberger/libcluon
sudo apt-get update
sudo apt-get install libcluon
The next step is to define the messages that should be sent between the Libcluon
programs. This is done within an odvd file. In your source code folder
(data/mytest) create a new file called messages.odvd by using gedit:
message MyTestMessage1 [id = 2001] {
uint16 myValue [id = 1];
}
The file format is defined by Libcluon, where each message is defined as a
message with a name, in this case MyTestMessage1, and an id, in this case
2001. Then the message includes one or more data fields, in this case a
- (2 bytes) unsigned integer called
myValuewith id1. Note that the
The next step is to turn the odvd file into code that can be included into the
C++ program. In Libcluon there is a tool called the cluon messages compiler
that is used only for this purpose. Use the following command in your terminal:
cluon-msc --cpp --out=messages.hpp messages.odvd
messages.odvd file as input and creates the C++
file messages.hpp as output. The output file is a header file that easily can
be included into a program.
If now running ls inside the mytest folder one can see the new file
messages.hpp. If opening that file in the editor one will first see the text
THIS IS AN AUTO-GENERATED FILE: DO NOT MODIFY AS CHANGES MIGHT BE OVERWRITTEN
meaning that this file is volatile and should always be created through the
odvd file. Even if possible, it would be inefficient and risky to rely on
manual compilation of the odvd file, as it would then not be guaranteed that
the C++ file strictly follows the specification inside the odvd file.
Therefore, a better solution needs to be pursued, so continue by again removing
the messages.hpp file according to:
rm messages.hpp
CMakeLists.txt file for editing
(gedit CMakeLists.txt) and add the following:
cmake_minimum_required(VERSION 3.2)
project(helloworld)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -Wextra")
find_package(Threads REQUIRED)
add_custom_command(OUTPUT ${CMAKE_BINARY_DIR}/messages.hpp
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMAND cluon-msc --cpp --out=${CMAKE_BINARY_DIR}/messages.hpp
${CMAKE_CURRENT_SOURCE_DIR}/messages.odvd
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/messages.odvd)
include_directories(SYSTEM ${CMAKE_BINARY_DIR})
add_executable(${PROJECT_NAME} ${CMAKE_CURRENT_SOURCE_DIR}/helloworld.cpp
${CMAKE_CURRENT_SOURCE_DIR}/prime-checker.cpp ${CMAKE_BINARY_DIR}/messages.hpp)
target_link_libraries(${PROJECT_NAME} Threads::Threads)
enable_testing()
add_executable(test-prime-checker test-prime-checker.cpp
${CMAKE_CURRENT_SOURCE_DIR}/prime-checker.cpp)
add_test(NAME test-prime-checker COMMAND test-prime-checker)
The new part is the add_custom_command section, where the cluon-msc command
will be run every time the program is compiled. Specifically, the first line of
the section means that if another part of our build process would need a file
called ${CMAKE_BINARY_DIR}/messages.hpp, then the following lines are a
recipe on how that file can be generated. The WORKING_DIRECTORY part states
that any generated file should be created inside the build folder rather than
in the source folder, as we do not want to keep the generated C++ file after the
compilation is done as we have the odvd file. Then, the COMMAND section
explains how the file is created, using the cluon-msc as demonstrated before,
and finally the DEPENDS section states that the odvd file is required in
order for the corresponding hpp file to be created.
Then, the row include_directories(SYSTEM ${CMAKE_BINARY_DIR}) is added to
tell g++ that include (hpp) files may also be present inside the build
folder, and not only in the mytest folder as before. Finally, in the
add_executable section, the file ${CMAKE_BINARY_DIR}/messages.hpp is added
as a requirement for building the executable. Note that the build process will
start at the add_executable and then notice that the file
${CMAKE_BINARY_DIR}/messages.hpp needs special care to be acquired (not only
read from disk), and then the add_custom_command section is considered.
Save and close the file. Then once more compile the program with cd build and
make. Now it should by default also compile the messages.hpp file. If
running the ls command now, inside the build folder, then there should be a
file called messages.hpp. It is much better to temporarily keep this file
inside the build folder and have it recreated for every compilation, and only
have the source odvd file in the mytest folder. In that way we avoid the
risk of keeping out-of-sync files that could result in bugs and errors in our
compiled programs.
The next step is to start using the message MyTestMessage1 inside the C++
program. In order to do so, change the helloworld.cpp file according to:
#include <chrono>
#include <iostream>
#include "cluon-complete.hpp"
#include "prime-checker.hpp"
#include "messages.hpp"
int32_t main(int32_t, char **) {
PrimeChecker pc;
std::cout << "Hello world = " << pc.isPrime(43) << std::endl;
cluon::UDPReceiver receiver("225.0.0.111", 1238,
[](std::string &&data, std::string &&,
std::chrono::system_clock::time_point &&) noexcept {
std::stringstream sstr{data};
cluon::FromProtoVisitor decoder;
decoder.decodeFrom(sstr);
MyTestMessage1 receivedMsg;
receivedMsg.accept(decoder);
PrimeChecker pc;
std::cout << receivedMsg.myValue() << " is"
<< (pc.isPrime(receivedMsg.myValue()) ? " " : " not ")
<< "a prime." << std::endl;
});
while (receiver.isRunning()) {
cluon::UDPSender sender{"225.0.0.111", 1238};
uint16_t value;
std::cout << "Enter a number to check:" << std::endl;
std::cin >> value;
MyTestMessage1 msg;
msg.myValue(value);
cluon::ToProtoVisitor encoder;
msg.accept(encoder);
std::string data{encoder.encodedData()};
sender.send(std::move(data));
}
return 0;
}
Note: This example is simply to show the details on how the communication works, but in a later example a much less verbose version is presented.
The first change is that the messages.hpp file now is included as well, as
discussed above this contains the C++ code corresponding to the message
described in the odvd file. Then note that the UDP ports for both the sender
and receiver are changed to 1238, this is done so that the different versions
of the program would not interfere with each other if running on a larger local
network. Next, the line sender.send("Hello UDP world!") is removed and
replaced by several lines of code, now inside the while loop instead of the
sleep function, with the purpose of constructing a MyTestMessage1 data message
from user input. The user input is stored in a variable value of type
uint16_t, and filled with data using the std::cin command which is the
antagonist of std::cout. The std::cin command stops the program and waits
for user input, and any such input will be read into the variable value. Then,
the MyTestMessage1 object is created with the name msg. Inside the object,
there is a method called myValue() which is used to set a value to the
internal variable also named myValue. All of these names are drawn directly
from the specification in the odvd file, so please take a moment to review
that file. The three following lines, involving the cluon::ToProtoVisitor
encodes the message msg into a std::string named data, and that string is
finally sent using the send() method as used in the previous example.
The lambda function inside receiver is also changed so that it can receive
messages coded from MyTestMessage1. The line
std::cout << "Received " << data.size() << " bytes." << std::endl is removed.
Instead an std::stringstream called sstr is created to turn the received
data into a stream, a C++ concept that represents data of unknown length.
Here, the length is actually well known, but the cluon::FromProtoVisitor
object decoder requires a stream from where it decodes data. The decoded
data is then turned into the MyTestMessage1 object receivedMsg, with the
data field myValue filled according to the received data. After decoding the
message, a PrimeChecker object called pc is created, and that is used in
order to determine if the received value is a prime number or not. The part
(pc.isPrime(receivedMsg.myValue()) ? "" : "not") is a special short form
if statement available in some programming languages. The condition is followed
by ? and if true the part before : is returned, otherwise the part after.
In this case, if the value receivedMsg.myValue() is a prime, then an empty
string is returned, and if it is not then the sting not is returned instead.
Tip! If you get error messages in the terminal it can look very strange and be difficult to read if not used to it. A recommendation is to always scroll up to the top of the errors and begin by fixing the first error. Most often an error results in many follow-up errors, and the compiler is not very helpful in explaining that. In addition, try to fix just one problem at a time, because often the fix to one problem can solve many of other.
Now save and close the file, and then build the program (make inside the
build folder). As before, two instances of the program is needed in order to
test both sending and receiving. When running (./helloworld) the first
instance however, you will, as expected, get the following output:
Hello world = 1
Enter a number to check:
build folder in a second terminal and run the program once again. The same
output will be shown, and this time when inputting a value and pressing enter,
the program still running in the first terminal should react. Finally, try to
open a third terminal and redo the same experiment.
Supported message types
The message used in the program above only included a single field of type
uint16. There are of course many other fields to choose from to build
messages, and any number of fields can be used. See a full list of available
field types below. The only restriction when working with messages are that the
total size of the message should not be larger than the maximum size allowed by
UDP messages. This should only be a problem if including raw binary data as a
string field. If including such raw binary data it is important to make sure
that sizes never exceed the maximum UDP size limit. For messages larger than
this limit, for example for image frames, the best option would be to instead
use shared memory for data transfer between microservices, with the restriction
that such microservies need to be deployed on the same physical machine.
Note! This section should not be added in any of your source files. It is only intended as a reference.
message MyTestMessage1 [id = 2001] {
uint16 myValue [id = 1];
}
message MyTestMessage2 [id = 2002] {
bool myValue1 [id = 1];
uint8 myValue2 [id = 2];
int8 myValue3 [id = 3];
uint16 myValue4 [id = 4];
int16 myValue5 [id = 5];
uint32 myValue6 [id = 6];
int32 myValue7 [id = 7];
uint64 myValue8 [id = 8];
int64 myValue9 [id = 9];
float myValue10 [id = 10];
double myValue11 [id = 11];
string myValue12 [id = 12];
MyTestMessage1 myValue13 [id = 13];
}
The OpenDLV standard message set
For OpenDLV there is a standard Libcluon messages set, referred to as the OpenDLV standard message set, primarily used for any OpenDLV microservice. The benefit of such shared set is that if two algorithms, encapsulated into microservices, share the same inputs and outputs they can be easily interchangeable, without affecting the overall system. This is especially important when building logic for autonomous systems, but less important when working with pure logging applications.
In addition to the standard messages, microservices might of course define their own internal messages, or even dump raw data to disk for logging purposes.
Introducing the libcluon OD4 session
As the last part of this tutorial, and the finale of our prime checker program,
the Libcluon OD4 session will now be introduced. The OD4 session replaces much
of the complicated details shown in the previous example, for the purpose of
providing a simple-to-use microservice interface. To use the OD4 session, change
the code in helloworld.cpp into the following:
#include <chrono>
#include <iostream>
#include "cluon-complete.hpp"
#include "prime-checker.hpp"
#include "messages.hpp"
int32_t main(int32_t, char **) {
PrimeChecker pc;
std::cout << "Hello world = " << pc.isPrime(43) << std::endl;
cluon::OD4Session od4(111,
[](cluon::data::Envelope &&envelope) noexcept {
if (envelope.dataType() == 2001) {
MyTestMessage1 receivedMsg = cluon::extractMessage<MyTestMessage1>(std::move(envelope));
PrimeChecker pc;
std::cout << receivedMsg.myValue() << " is"
<< (pc.isPrime(receivedMsg.myValue()) ? " " : " not ")
<< "a prime." << std::endl;
}
});
while (od4.isRunning()) {
uint16_t value;
std::cout << "Enter a number to check:" << std::endl;
std::cin >> value;
MyTestMessage1 msg;
msg.myValue(value);
od4.send(msg);
}
return 0;
}
As seen in the code, the OD4 session handles all the encoding and decoding of
messages internally, as well as all the details regarding the UDP sender and
receiver. Still, inside the OD4 session, everything is very similar to what was
done in the last version. The cluon::OD4Session object od4 is created with
two constructor arguments, first an id referred to as the conference id in
Libcluon, and the second a lambda function reacting to incoming data. In
practice the conference id is the last number of the multicast address as seen
in the previous code version, but the conceptual understanding should be that of
a real-world conference where everyone in the same room are able to communicate,
in this case referring to microservices. The lambda function
is a callback that is called every time a message, here called envelope, is
received. The envelope contains a message, as well as some meta information
such as timestamps, the type id of the contained message, and a user-specified
if referred to as the sender stamp. In the example above, the first step
inside the lambda function is to check the type id of the envelope to determine
if the content is of type MyTestMessage1. If so, then the message is extracted
from the envelope, and then the PrimeChecker object is created and used as
before. Then, the second change is how a message is created and sent, with the
OD4 session only consisting of three lines of code, the first creating the
message, the second setting the value of the internal fields, and third sending
it using the od4 object. Finally, the while loop was slightly adjusted to
instead use the OD4 session for its condition rather than the UDP receiver as
before.
Finally, save and close the file. Now compile it and test it with three terminals as before. The function should be exactly the same, even if the OD4 session simplified the code greatly.
Conclusion
In the past two tutorials you worked with the prime checker as a way to first learn basic C++ skills and how to compile and formally test code. Then in this tutorial the concept of microservices was introduced, and explained from the perspective of communication and deployment.
The last step is to check that your source folder is clean and nice. The files
that should exist in mytest are: catch.hpp, cluon-complete.hpp, CMakeLists.txt,
helloworld.cpp, messages.odvd, prime-checker.cpp, prime-checker.hpp, and
test-prime-checker.cpp. Any other file, including the build directory can be
removed, using rm -r build, as they are not important for the final program.
Clean up your mytest folder and go through each file and fix any indentation
problems that you might find. It is also highly recommended that you spend a
while thinking about each file, and its purpose, before you continue to the next
tutorial.
If you later want to rebuild the program, you can simply do it with the
following commands (from mytest):
mkdir build
cd build
cmake ..
make