Click here to Skip to main content
15,881,852 members
Please Sign up or sign in to vote.
2.67/5 (3 votes)
See more:
Hello,

I am thinking about a hierachy for my application, and I need some advice regarding it. I want to build a library using c++ which will serve as an abstraction layer between applications and low-level process. The library will provide some APIs for the applications for some purposes.

For example the low-level process, may send an indication to the library i.e. raise an event, and the library in turns send it to all the applications, which have their registered callbacks to this library.

If anyone can you provide me with an example for the previous scenario in c++ using boost library or the standard library will be perfect in Linux environment? The example that I want is: Generate event from a process and pass it to the library, then let an application register callback to the library.

Initially I know in Linux, I may use signal to send events, but my plan is to have a something more general not tighten to a specific OS.

Thanks a lot.
Posted

1 solution

In my opinion you should write a server and both the event consumers and event producers should be clients. Basically your library would be just a client that sends/receives messages from the server. This has many-many benefits:
- Your processes can be on different machines.
- Socket communication can be programmed in almost all langauges.
- One of the most crossplatform languages is C/C++ and in this language you have a more or less standardized socket api.

Don't forget to use a data exchange format that is easy to handle in most languages/scripts. If you don't want to save with bits then I recommend xml or json.

There are many-many frameworks and libraries that you can use to get the job done easier in different languages. I myself usually prototype/test such simple client/server architectures in python where putting together a simple socket server/client takes only 1-2 pages of code and then I can write conforming clients/servers in different languages easier by being able to test it with the python client or server that can be written easily/quickly if you know python.

Note: There are (persistent/non-persistent) message queue solutions and subscription systems that basically implement what you need. Passing a message and receiving it somewhere else... But if you are writing this to learn something then let's reinvent the wheel just for fun! :-) You can learn a lot from this!

EDIT: Persistent and non-persistent message queues: These are quite high level concepts just like a "message bus". You can search for actual implementations with google, I don't link one as creating an implementation in a language is not easy (creating a roboust ones can be more challenging) - I would rather just explain it. A message queue is just a message queue somewhere. It maybe be stored in memory, in pure files, or in a database, or anywhere else. It must be accessible by the "consumers" and "producers" of the queue by using some library. In some scenarios these consuer/producer programs don't even have to know about the actual implementation (whether the queue is in memory, in db, or somewhere else in networking engine buffers between apps) because this is just an implementation detail. If you implement a message queue over tcp protocol then you have an implementation but if you defined a nice general interface then you can easily replace the implementation without modifying the client that uses the interface. I don't have much to tell you about simple pure message queues. Persistent message queues: These are usually implemented in the form of a message queue server backed with a db. You connect with a producer client and you just pass in messages to the queue. You can also connect with consumer clients and you can pull out the messages. The queue is persistent because messages are always persisted and never get lost. If you send a message by a producer client when there is no consumer client connect then the persistent message queue server just stores the message and when the consumer connects it can pull the messages even if they were put to the queue one week before and currently the producer client is disconnected. If you stop the message queue server then its messages are persisted to a db.

"Message bus": This is also a high level concept and I wouldn't compare it with a an actual building block for an implementation like "sockets". TCP socket is just a building block, a possible way to build up a communication channel between two machines or between different(same) processes on a single machines (yes, sockets can also be used as pipes, I love it for this flexibility!). You can actually implement a message bus with TCP connections but its probably not the most effective solution.

Here we can separate your problem into many layers:
1. The first layer is the high level description of your actual problem and the list of requirements that your solution must meet. For example being able to effective do this and that while the other 2-3 operations can be less effective as a tradeoff. The list of requirements will help you to decide which of the possible implementations to choose. Usually there is no "best" "general" solution. If you don't know what you want to do, if you are just playing around then don't try to find a "best"/"optimal" general solution because usually there is no such solution.

2. If you know the problem and you have the requirements then try to find out what kind of operations do you have to perform to do the job effectively. In case of operations I'm thinking about something like a sequence-diagram in case of uml. For example: In case of creating a new user I have to send a create-user message and as a response I get OK/Failure. But these "operations"/"function calls" can be more complicated than this simple example. In this layer we can think about the problem as either "packet/message" exchange or function calls but I like thinking in "function calls" more, this is why: In case of network communication you are usually performing function calls. You serialize the function call parameters, you send them over the wire, and later you get back the serialized return value of the function call. In case of an optimal solution this function call is asynchronous (like the Begin*** calls in C#), you are just starting the function call and as an extra parameter you specify a callback to be called when the result of the operation came back (from somewhere, from a subsystem, from a service, from the network, ...).

1. Choosing actual implementations to realize the building blocks of your solution. For exmaple as a communication channel of your "function calls" you can provide a TCP or UDP based channel or a PIPE, an SSH connection maybe something else... If you separate the channel implementation with an interface then you can play around with this later. In some cases however you need some really optimized communication (like in case of Quake3) but in my opinion such optimization is needed only in 0.00001% of networking apps... Q3 needs very low latency while most apps don't - elegant solutions have almost always priority over optimized ugly ones.

As a first step try to think about the 1st and 2nd layer I was talking about. As a first implementation I would recommend you a simple server/client approach with TCP. A centralized server (as any other solutions) have many benefits and disadvantages.

Some benefits:
- If you put a server to public on the internet you can easily connect with a service from anywhere, even from internal networks from behind NAT.
- It is very easy to program a centralized logic that is inside one server.
Disadvantages:
- You actually need a server somewhere in contras to peer to peer.
- Single point of failure if you server architecture is overly simple.

Any solution has its benefits and disadvantages. There is no perfect. If you know what you want to do and what you need then you can decide which advantages brings you the most benefits with acceptable drawbacks.

If you are just playing around then write a simple TCP server first where clients connect and ask the server to perform something. For example the server may provide the following operations (function calls):
- Add event
- Get all events

If you want several applications then I don't see why couldn't you do that with a TCP server: You send an app id in all function calls, this way you can direct the request/function call to an app specific part of the server. If you want the server to be extendable you can also do that, a client could register as an app and all events that are sent to the server with the app's id could be directed to the app, then the result from the app back tot the server and finally from the server back to the client. I know, this isn't the most optimal solution but believe me, if you implement one like this you will find so many problems in the way, you will learn a lot if you implement this once, then again more elegantly, and then again (for example by using a serialization framework, by abstracting away communication channels, by using async (network) IO...

Don't hesitate especially if you are doing this "just because". Start the TCP server implementation, it will be a challenge for sure.
 
Share this answer
 
v3
Comments
Mr.HA-AS 17-Mar-14 8:12am    
Thanks a lot for your clear explanation.
But can you give me some examples about the (persistent/non-persistent) message queue?
Mr.HA-AS 17-Mar-14 9:34am    
Again, I was thinking about your first suggestion regarding the socket. But the problem for me the abstraction layer or the API layer should deal with a variable number of applications.
So it's hard to establish a connection between each of them every-time.
I think maybe message bus would serve better. What do you think?
Mr.HA-AS 17-Mar-14 19:55pm    
Thanks again for your great explanation, I think I wasn't clear enough and precise in the description of my problem.

Basically what I need is to establish a communication between my low-level process (TCP-IP process) and the library which contains the APIs, and between the library and different kind of applications like FTP server, Video server, etc. This library should be standardized in away that all applications will know about the APIs and their primitives.

The low-level process will receive some indications and upon the reception of these indications I will make a call to the library, in the same time some applications will have already registered callbacks to this library. So the call to this library actually will make call to these application, which in turn returns back the result to the low-level process.

As a hierarchy, consider I have 3 levels. The top level is application level which might be video server, FTP server. etc. Note: These applications are running independently and are not aware about the low-level process, they only communicate with the library. In the bottom, I have the TCP/IP process which will be running in a standalone mode . And in the middle I have the API layer "The Library", to which all application will register their callbacks.

I want an example for this case, Also I need to know if this design is good or not?

Because there are other methods: like using signals in Linux to tell the application what to do. Or using Message bus as what you suggested, where all the applications will register to it and do sending and receiving , also there's Signal2 library in boost but I found it complicated and didn't know how to use it, and finally there's the UDP/TCP, which I don't prefer in my design.

Thanks a lot :)
pasztorpisti 17-Mar-14 20:31pm    
This is quite vague. You need a server with a client library that helps you to write client programs (event consumers and producers). Whether the server runs on a different machine or on the same machine as a service or as a kernel module or whatever is just an implementation detail as well as the actual implementation of the communication channel however I wouldn't use signals for this (as signals were invented for other purposes in my opinion). If you put well designed interface between the server and the client then you can implement that in several ways. For example in my client/server apps I often provide a "null" communication channel that just links the server code into the client to be able to debug the client with an "in-proc server"...

In my opinion you still don't know what you want to do and if you don't know what you wanna do then just write it as a general solution as a client lib + server that uses sockets for communication. Find out what you want. Steps #1 and #2. It has no point of thinking about how to implement "something". If you want to put the "library" too into a separate process as an intermediate layer then just do it but have a good reason for splitting up a solution into so many parts as it has performance overhead. If you don't know how to place these building blocks then you don't know it because you have no requirements. No requirements -> no good solutions as I described previously. Your vague goals are not requirements, they are just too little goals that can be achieved in 100000... ways.
Mr.HA-AS 17-Mar-14 20:48pm    
Sorry I forgot to mention, why I am splitting my solution too three parts, is because I want to hide the implementation of the low level process the TCP/IP process from the applications, the Abstraction layer will server as a wrapper to use the functions provided by the TCP/IP layer
So when a new application wants to integrate with my solution it will only have to include the library and doesn't need to worry about the TCP/IP process. I hope I made it more clear.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900