Too many threads per chance?

Question

3.75/5 (3 votes)

See more:

I am having a problem with a small project. The idea is for a message broker type service; yes I know there are libraries out there but I want this to be pure J2SE with minimal third party stuff.

So what I have is a server set up like this:
- A Connection will send requests to the Broker and receive replies.

- The Broker provides a number of Services to handle different requests. On receipt of a request it forwards the message to the correct Service or replies directly to the Connection that no Service is available. When it receives a reply from the Service it forwards it directly to the Connection.

- Each Service supports a set Agents. On receipt of a request it forwards the message to the correct Agent or replies directly to the Broker that no Agent is available. When it receives a reply from the Agent it forwards it directly to the Broker.

- Each Agent supports a single message type via a set of Workers. On receipt of a request it forwards the message to the next available Worker. When it receives a reply from the Worker it forwards it directly to the Service.

Now each part is run in a separate thread. The current model I have calls for around 20 agents across four services. If you do the maths that's nearly 50 threads for a single worker per agent.

The server cannot be put on a single thread as some of the messages can be processed asynchronous and supply multiple responses [a subscribe/publish model] while others will be single shots such as transactions.

One other piece of fun is that I plan to have a type of worker that connects to another server allowing these to be chained and to provide scalability. This is another nail in any chance of having a single threads.

So, it's over to the hive mind for any suggestions hints or abuse.

Posted 4-Jun-13 23:45pm

Nagy Vilmos

Add a Solution

Comments

Pablo Aliskevicius 5-Jun-13 10:40am

I did something similar once:
* The Broker converts the request into a Job, and puts the Job in a queue. The Broker notifies the client that the Job is in the Queue.
* The Workers (number depends on number of CPUs) pull jobs from the queue, and execute them.
* The Client has a way to query the state of 'his/her' requests.
'Job' is an interface, with one method (Execute()).
In this way, the number of threads was minimized (all Agents disappear; since Job is polimorphic, any worker can do any Job).
In your case, you would also need a Log, that holds the Job state (Queued/Executing/[Succeeded|Failed]). Querying the job state may be done through the Broker, or separately. In my case, the client code may send a request, the client machine may disconnect, and much later, the client reconnects and asks for status.
Pushing notifications to the client is a different matter, but may be done in the context of the execution of the Job.

Just my two bits,
Pablo.

Nagy Vilmos 6-Jun-13 6:25am

I think I'm moving away from the 8 million thread model to a smaller thread per agent model. I have a test bench that I can, err, test with. If that goes FUBAR then I'll need at least a thread per worker.

[no name] 7-Jul-13 7:33am

don't you close the connection just after client sends the request? so it's not a broker service at all, it's a simple sync message invocation and more complex.
there no promise and warranty to execute the task as same as client request, just reply a id(request number) to the client and close the connection, create a queue/topic and store the requests.
Use thread pool for your agents. after request get proceed save the result in either db or a data source. clients has to check the response again with its request id, or if you could reach the client just notify it about the response state. and in a interval removes the requests and responses have expired.

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)