|
Hello everyone,
I’ve got a serious problem that is killing our product and requires an immediate solution. Our situation is CRITICAL!
We have implemented support for a real-time serial protocol in our Windows XP Embedded based device. It uses a regular 16550 based UART/COM port on the motherboard and the following settings: 19.2Kbps, space parity, one stop bit, and eight data bits. The COM port is configured with COMMTIMEOUTS = { 5, 0, 0, 0, 0 }. The machine contains a 1.6GHz Celeron M processor and 1GB or RAM, which is not completely used.
The protocol requires our embedded device to respond to a packet within 20ms so it is imperative that our process receive the packet from the kernel (serial.sys) as fast as possible. The end of a packet is identified by 5ms or more passing between bytes. We’ve implemented several different solutions using both synchronous and overlapped I/O and have determined that the most reliable way to get the data as fast as possible is to start up a read thread with real-time priority and invoke ClearCommError() at 5ms intervals while tracking the amount of data in the receive buffer with COMSTAT. cbInQue. When the amount of data does not change for 5ms we call ReadFile() to obtain it. This solution seems to work fairly well for a few minutes and then suddenly ReadFile() begins returning the data after 100ms or more instead of the 10ms-15ms it was previously. It’s also returning the data in non-contiguous fragments even though our serial line monitor shows that the data is being sent to our device in one solid block (that never exceeds 260 bytes and is usually 32-48 bytes). The only “solution” we’ve found so far is to shutdown and restart the process. We’ve even tried having the process close and reopen the port with no success.
I’ve considered modifying serial.sys to automatically handle packet reception at the device level, but I’m concerned that it wouldn’t solve the problem because it appears to be a latency issue coming from somewhere other than the driver. The protocol code is very large and complex so moving it into a driver is so undesirable that we will probably move to embedded Linux instead (if it’s necessary).
Please help!
Thanks,
Peni
|
|
|
|
|
Hi,
welcome to CodeProject. And, good first post!
first of all it is very ambitious to mention Windows, real-time and 5 or 20 msec all in one sentence.
I do have quite some experience with Windows serial ports, it is getting a bit rusty though. Amongst others, I did a serial protocol where overall bandwidth was most important, but so was latency time for the first message. So I did experiment with all the timeout parameters a lot.
The scheme you described seems rather complex, and makes me wonder how the hell you can ever test that to satisfaction as data can and will come in asynchronously with respect to your 5msec operations, which (1) may interfere with the driver's normal operation, and (2) may have some timing jitter.
BTW: how do you create that 5msec period?
Although you did describe a lot, I would want to know more, such as: Is it binary data, or text (8-bit or 16-bit)? How many bytes or characters would make up a single message? is throughput relevant, or just latency? why did you choose 19200 Baud? would it be possible to change the protocol? and the port's settings?
Also I wouldn't mind some background info, what it really is about; it tends to help me focus on the technical solution when I know what kind of application is involved.
Give me some more info, and I'll probably come up with a direction I would be inclined to take, to get something both simple and hopefully as reliable as Windows can offer.
Luc Pattyn [Forum Guidelines] [My Articles]
- before you ask a question here, search CodeProject, then Google
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get
- use the code block button (PRE tags) to preserve formatting when showing multi-line code snippets
|
|
|
|
|
Hi Luc,
I don’t have control over the protocol specification so I can’t change things like the baud rate, word size, parity, or timing constraints. It is an 8-bit binary protocol so any value in the range [0, 255] may be transmitted or received. It’s an old industry standard protocol used for communication between industrial devices. Unfortunately I can’t go into much detail because I’ve signed an NDA.
The only way I can determine where one packet ends and the next one starts is by measuring 5ms or more between two characters. If there is no next packet then I just need to detect when 5ms passes after the last character is received. This is usually easy to manage because packets are typically transmitted in bursts with less than 1ms between characters.
If there were a way to configure Windows (or serial.sys) to return the data it has received in a more timely fashion to the process waiting for it then I’d have a solution. Ideally, I’d like to be able to rely on the driver to detect the 5ms inter-character period set with COMTIMEOUTS, but trying that resulted in my process receiving the data at least 80ms after it arrived at the PC. It was more like 100ms on average.
|
|
|
|
|
Hi,
the protocol being fixed is very unfortunate; protocols should serve both parties involved, not used one, and receiving is much more difficult than transmitting, you figured that out yourself by now.
AFAIK the Windows serial driver's timing is based on the system timing, which in Windows leaves much to be desired. And all pre-empting, sleeping, and timeouts are based on a single clock which ticks at some 16 msec, the number may vary but AFAIK it never is down to the msec region. I wrote an article on timing from the .NET perspective, it may provide some insight; and if you happen to have .NET installed, you may run the app for some experiments.
Here are two ways for you to go and probably get better results than what you have now:
1. forget all timeouts, just start polling, i.e. have one thread dedicated to reading one byte from the serial port, zero timeout, create your own timeout (1, 2 or 3 msec) with a multimedia timer. That way you should be able, if system load by other processes and threads allows it, to
detect the absence of new characters for an interval of 5 or 6 msec. I would not recommend doing anything other to the serial port than just reading the data byte amd/or the status, changing the status might introduce extra phenomena.
2. the "hard" way: insert another device, micro-controller based, no OS required, that deals with the incoming bytes, and turns them into text-oriented characters with a proper and unique end-of-message marker (could be as simple as hex text, so each incoming 8-bit becomes two hex digits at a higher baud rate, detect end-of-message by timing and send an extra ASCII CR. Then ask the Windows serial driver for a line of text, it knows how to handle a single end-of-line marker, and should work swiftly (i.e. as swiftly as MS stuff can).
Both of the above still make you depend on the limited predictability of Windows stuff.
I have no idea how critical it is to you not to miss a single message...
final remark: do you know anything about the control lines, are they alive, i.e. is the other side signaling start and end of message using one of them. If so, you might poll the line and only launch a read when you already know the data has been received.
Luc Pattyn [Forum Guidelines] [My Articles]
- before you ask a question here, search CodeProject, then Google
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get
- use the code block button (PRE tags) to preserve formatting when showing multi-line code snippets
|
|
|
|
|
Hi Luc,
The solution I’m using now functions in pretty much the way you suggest. I have a read thread with real-time priority that polls for the presence of received serial data by calling ClearCommError(). After this polling loop detects that no new data has arrived for 5ms it calls ReadFile() to obtain and process the data. Prior to this implementation I was invoking ReadFile() and appending new data to a buffer until ReadFile() returned no data for 5ms (or more), but this solution didn’t perform quite as well as using ClearCommError() to poll for the received data size. The protocol also does not use any modem control signals so I can’t use them to detect when data has arrived.
Unfortunately I don’t have the option to add new hardware to the device or I would definitely offload some of the protocol support to a microcontroller. I could also add it to a dedicated Windows driver since the serial device will always be an integrated 16550 compatible USART, but I don’t have enough time right now divide the protocol implementation between a driver and the user mode process.
I found that reducing the time quanta given to each thread in the system provides just barely enough response time to make the protocol implementation function under typical conditions, so the implementation I have now does not satisfy the protocol specification but it does work in typical real-world conditions. Hopefully it will satisfy our needs until a better solution can be implemented.
Thanks for the fast feedback and all your help. The information I have received from everyone in response to my post has been invaluable.
|
|
|
|
|
Luc Pattyn wrote: first of all it is very ambitious to mention Windows, real-time and 5 or 20 msec all in one sentence.
Welcome again in the CP's memorable quote list [^].
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
This is going on my arrogant assumptions. You may have a superb reason why I'm completely wrong.
-- Iain Clarke
[My articles]
|
|
|
|
|
PeniWize wrote: This solution seems to work fairly well for a few minutes and then suddenly ReadFile() begins returning the data after 100ms or more instead of the 10ms-15ms it was previously
That's probably because some other thread is getting a time-slice instead of your responder thread. You could try marking your thread[^] with a time-critical priority in an attempt that it isn't interrupted.
However, a WIndows XP embedded time-slice (aka thread quantum) is around 90-100 milliseconds[^] which means that if you let any thread have processing time instead of your, you will not be able to respond to events in that 90-100ms window. That is why Windows XP (even the embedded version) is not a real-time operating system - you cannot deterministically respond to events within a small time-frame.
You could try reducing the thread quantum[^], but you can't get it below about 30ms.
PeniWize wrote: The protocol code is very large and complex so moving it into a driver is so undesirable that we will probably move to embedded Linux instead (if it’s necessary).
A driver was what I was going to suggest - but embedded Linux is a good option so long as it has a real-time kernel[^].
Good luck!
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
|
|
|
|
|
Hi Stuart,
Thanks for the feedback. It has given me some good leads. I forgot to mention in my post that my process is high priority and its serial I/O thread is real-time priority. Do you know of anything else that I could do to encourage the serial device driver or Windows kernel to return the packets to my process more quickly?
|
|
|
|
|
Remove as many other Windows processes as you possibly can - services you don't need etc.
Tune the thread quantum to as small a value as you can.
The trouble is that the Windows scheduler gives processes a full time slice once they've started, even if your process suddenly gets data. That means that there is always the chance that some other process might be running when your data arrives. You have real time timing constraints, which means you need to design a real time solution - and Windows isn't real time capable.
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
|
|
|
|
|
Setting "thread-priority" to real time would help? 
|
|
|
|
|
Not really - the thread setting is actually 'time critical'.
Windows is not and cannot be a hard real-time OS with the scheduling policies it uses - the thread quanta are too large and are reltively uninterruptible. But as it's not marketed as a real-time OS, that's fine - until people's apps stray into the real-time area, in which case they're in for a whole heap of pain. I know, I've been there, trying to understand why (every so often) I would see 100ms delays for 'no reason'.
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
|
|
|
|
|
Thanks for all the invaluable feedback, everyone. Many of my suspicions have been verified and I’ve learned some things about Windows scheduling that has given me a bit more gray hair.  Fortunately, setting the thread quanta to the minimum (about 33ms) provides just barely enough response time to make the protocol implementation work under typical conditions. The implementation doesn’t meet the specification, but it does work so it’s sufficient for now.
Thanks again for all the help!
|
|
|
|
|
"G d Answer"
|
|
|
|
|
I suggest you look for a long term solution, Windows is not meant to be this close to real time. Not jsut because of the serial port issue but if your system is this time senistive you are likely to hit further problems and trying to balance the scheduling will continue to be a headache.
|
|
|
|
|
I'm a little puzzled by the behavior you are seeing from your serial device. We run XPe and a dozen serial devices in real time, but they are purely interrupt driven and I think that's the difference. The requirement for the 5ms timeout is problematic, though it could be done with driver level code.
(If hard real time is needed and you want free, don't bother with Linux; check out eCos instead. If cost isn't an issue, there's plenty of choices like Windows CE or Integrity from Green Hill Software. [And VxWorks, but I detest it and found that dealing with Wind River Systems was a nightmare.])
Anyone who thinks he has a better idea of what's good for people than people do is a swine.
- P.J. O'Rourke
|
|
|
|
|
The OS is Windows XP, writing in C using win32 API. I cannot find a (just spent an hour googling sigh!) way to have mixed colors in a text string. SetColorText() makes the text all the same color, and so does SetBkColor(). For example I would like to print the string "This is a test." and have the word "test" in red and the other words in black. If anyone could show me code to do this I'd really appreciate it.
Zach
|
|
|
|
|
You'll have to draw 'test' separately from the rest of the text, so you can change the DC's attributes between calls to the text output routine.
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
|
|
|
|
|
If you don't mind the "clutter" you could try using a rich edit control and the EM_DISPLAYBAND message to display formatted text, using the rich edit control you could color your text the way you want or even have multiple font sizes and styles in on line and then use the forementioned message to render it onto a DC... i know it sounds ugly, but hey, sometimes ugly solutions are the only working solutions. (Not that i am saying this is the only way to do what you want)
> The problem with computers is that they do what you tell them to do and not what you want them to do. <
> Life: great graphics, but the gameplay sux. <
|
|
|
|
|
Hello gentlemen,
Is there any option to disable/prevent window minimizing when I press window key(between Ctrl and Alt) + D?
I have one dialog without tile with transparent backgroud, I don't use MFC.
I am capturing these windows messages
case WM_WINDOWPOSCHANGING:
return true;
break;
case WM_WINDOWPOSCHANGED:
return true;
break;
case WM_SIZE:
if(wParam == SIZE_MINIMIZED)
{
return true;
}
break;
Thank you.
|
|
|
|
|
If you want your dialog to stay on top all the time, check out SetWindowPos()[^], and pass &this->wndTopMost to the function.
|
|
|
|
|
Thanks for reply,
my dialog must be at the bottom of the Z order.
I have dialog with text and the bg of the dialog is transparent.So it looks like the text is written directly on the desktop(but it isn't).
The problem is when I hit win. key+D so the dialog is minimized.
I know, it can be done using of windows hooks. I want to avoid of that(if it's possible).
|
|
|
|
|
Win+D does not mean minimize. It means show desktop.
Win+M is minimize.
«_Superman_»
I love work. It gives me something to do between weekends.
|
|
|
|
|
daavena wrote: I am capturing these windows messages
Have you verified with Spy++ that you are handling the correct message(s)?
"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
|
|
|
|
|
I use spy++, but when I hit win+D there is no message for my dialog(nothing about minimizing).
_Superman_ wrote
Win+D does not mean minimize. It means show desktop.
maybe that is the reason why I can not capture the proper message.
|
|
|
|
|
Hello all,
Firstly, does anyone have any info regarding the NtSetSystemPowerState function? I can't seem to find any documentation.
Secondly, is there any way to shutdown which is faster than calling ExitWindowsEx, but slightly better than calling NtShutdownSystem from the kernel?
Oh, and finally, can anyone provide me with some information on the entire windows shutdown process, beginning with the call to ExitWindowsEx?
Thanks in advance.
modified on Saturday, March 14, 2009 3:10 PM
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
