Re: Detecting Analog (A/D) Pulses using Software - Algorithms Discussion Boards

Dale Keller wrote:
I realize this is a crazy question to ask

Actually, it is indecipherable as Roger has said.

It might be helpful to know of any ideas you have thought up.

"The clue train passed his station without stopping." - John Simmons / outlaw programmer

"Real programmers just throw a bunch of 1s and 0s at the computer to see what sticks" - Pete O'Hanlon

"Not only do you continue to babble nonsense, you can't even correctly remember the nonsense you babbled just minutes ago." - Rob Graham

I'm having trouble understanding what you need too. Physics equations are usually on one line, while a programming-language implementation often uses several assignment statements.

The rules for transforming equations are pretty straightforward, and with some practice can be mechanically applied.

Sorry for the crossposting[^] but I'm not quite sure where to post this.

Rather than repeat and retype all of this again, could you please just refer to the referenced post. If you believe the post belongs here, please let me know and I'll move it.

TIA

I'd say this is the place more appropriate for your question but considering how specialized it is, you'll probably get few takers.

I'm not entirely sure what your ultimate question is from the quick read I gave it, I'm still waking up. Smile | :)

However, it sounds like peak detection and separation. A place to look for more software algorithms to handle this is in some of the process industry instrumentation areas. The one I know for sure that has to do peak analysis is chromatography for chemical analysis. The only contact I had with that particular part of the software was trying to port the peak analysis from 16 to 32 bit and C to C++. It was a godawful mess. Sniff | :^)

I also seem to remember seeing a software library advertised that was for general peak fitting and analysis so it has to be somewhat common in instrumentation.

If you don't have the data, you're just another a**hole with an opinion.

Thanks for your suggestion, Tim. I'm looking into it and it looks like it could work but so far all I've found is plot programs that do this and no source code or discussion on the algorithms except for mentioning which ones can be used.

I'll keep looking.

I've read your posts and I don't have a clear idea of what you are trying to do. It looks like you have samples of an analgue radar reflection and, with knowledge of the transmit pulse shape, you want to determine the number and locations of the reflected pulses. If that is the case then you are really looking at standard radar signal processing and there is a substantial literature on the topic.

The two main traditional performance goals would be the ability to detect weak echoes in the presence of noise and the ability to resolve echoes close together. If you need good performance in either of these areas then I would strongly advise you to read up on estimation theory (or get advice in this area) before you settle on your algorithms. Simply trying to mimic what you would do manually will be far from optimal.

The sort of ways that you pose the question mathematically is "If I assume that this waveform consists of N pulses plus random noise, what are the locations and amplitudes of the pulses?".

Another way you can approach the problem is as a system identification problem. Here you say that you transmit waveform s(t) which passes through a filter with impulse response h(t) and you receive waveform r(t) which includes noise as well. Given r(t) and s(t), determine h(t). This can be solved by a least squares approach, but you need care to reduce the sensitivity to noise. In the absence of noise most heuristic algorithms work reasonably well where you simply write r(t) as the weighted sum of delayed versions of s(t) and have some way of optimising the weights and delays.

You really need to settle on your algorithms and know that they will work before you start coding. Typically people would simulate their algorithms in something like Matlab, where they can create samples of signals with various combinations of pulses and noise and test their algorithm's performance.

Peter
"Until the invention of the computer, the machine gun was the device that enabled humans to make the most mistakes in the smallest amount of time."

I'm sorry if I wasn't clear, I've written so much each time I just tried to abbreviate it. No, I'm receiving beacon radar reply pulses from aircraft transponders. Additionally, though beacon radar transmits (interrogates) then turns off the transmitter and turns on the receiver, I am not an integral part of the radar system, primary (search) or secondary (beacon). So, they are not reflections that I am processing as in a primary radar, but responses or replies to interrogations to a secondary radar system.

I am not connected to the radar system and am simply receiving beacon radar replies, considered the downlink, from aircraft. The possibility exists that the system could be adapted to listen to the uplink (the radar transmitting interrogations) and if we did, that would simplify my processing significantly especially for Mode S data since the plane being interrogated would have its unique ID as part of the message and I would know what plane would be responding even if the receipt envelope was slightly corrupted. However, the added cost and development time makes it highly unlikely that this will be done.

Besides MODE-S data there can be Air Traffic Control Radar Beacon System (ATCRBS) replies (the older system prior to Mode-S which mostly general aviation uses except that Mode-S equipped aircraft will respond to ATCRBS interrogations) and finally there are ADS-B replies. All of these replies can be interleaved or overlapped.

Though, I am not totally familiar with all of the noise filtering methods, I have played around with them using D-Plot on the data and have used Least Means Squared, cubic spline, and others and have found while they do have some positive effect, the effect is not better than simply averaging or even decimating the samples. The other methods seem to smooth the data but don't reduce the noise level at the floor or suppress noise spikes. And the tried methods have very little effect on the noise levels on our legacy receiver. A new receiver has been designed and from what I'm told, requires more amplification, shoots up the leading edge much faster but has much more noise. Thus each receiver is different and the design will change.

Keep in mind that I am only over there for a short period of time on loan and that ideally, most of this would have been done in hardware, but there are other considerations as well such as cost. What they have now works for Mode S data using a very simple method, but they don't process anything else and they have problems processing overlapped pulse trains especially at low signal levels because of noise.

Right now, I am looking into the peak fitting suggestion above and its going slow but I will post back my final progress later.

OK I think I now understand that you are trying to decode things like the Pulse Position Modulation on the Mode-S reply.

JohnnyG wrote:
A new receiver has been designed and from what I'm told, requires more amplification, shoots up the leading edge much faster but has much more noise

Your new receiver simply has more bandwidth, this doesn't matter, the algorithm will reduce the noise bandwidth. More noise coming in does make it more important to get the algorithm right.

JohnnyG wrote:
ideally, most of this would have been done in hardware

It doesn't matter where it is done, the algorithm is important.

Simple data smoothing and peak detection is not going to give anywhere near optimal performance in noise.

For what it is worth, my suggestion would be to run some sort of maximum likelihood detector to determine the timing synchronisation (i.e. where the pulse edges are), in it's simplest form this would use the energy in all four preamble pulses (to reduce the effect of noise) and the processing would be quite simple. Determining the timing is the key, once you have this you can process the message, in each bit period integrate the total power in each half and compare the two, outputting a '1' if there is more energy in the first half. For improved noise performance you could look at running a decision directed timing loop to improve timing detection from subsequent bits, or even do a joint timing and data estimation. I don't know what sort of performance you need or what sort of processing you have. Determining the timing using a detector based on all four preamble pulses will vastly out perform any data smoothing / peak detector approach, and would have very simple processing.

If you decimate the samples to reduce the noise you introduce timing errors as you can't guarantee that your decimation coincides with the signal timing, and if you run averaging you are doing much more processing than you need and you still have to work out which average to use - i.e. determine the timing.

Good Luck!

Peter
"Until the invention of the computer, the machine gun was the device that enabled humans to make the most mistakes in the smallest amount of time."

cp9876 wrote:
you could look at running a decision directed timing loop to improve timing detection from subsequent bits, or even do a joint timing and data estimation.

Wow! You said a whole lot of what I don't know. I may be in over my head. A quick search on the Internets via google shows a whole lot of formulas for s(t), etc. I don't think I have the math background to do all of that.

And even though I am being self deprecating, its encouraging to me that many programmers (and hardware engineers) I've approached about what I'm doing, either have no idea of what I'm talking about or if they do, have no idea on how to do it. Your knowledge of these techniques is admirable.

I sure could use a helper.

As far as decimating and averaging, I meant that I just tried that in the plot program to see their outputs. In reality, the legacy program simply skips 25 samples at a time through the data and I do the same except I refine the sample size as I get closer to the downward slope or unexpected changes in direction.

I don't think I have a problem detecting leading edges or even plateaus. Based on what you've said and what one friend suggested, since I do know the timing constraints for the preamble and for ATCRBS messages (framing pulses spaced 20.3 us apart), I can find a leading edge and then jump, for example, ~ 20.3 us ahead in the data and look for another leading edge. Of course, then refining for pulse amplitude and width, I can pass the entire pulse train to another processor to process and/or look for overlapped pulses before doing that.

I need to process more then just Mode-S data and the existing legacy software simply looks for a preamble and then processes the entire 56 usec envelope as Mode S. Your suggestions are very helpful though but I'm not sure I'm capable.

Sigh.

This is a shot in the dark since I'm not sure exactly what you're doing (it's probably over my head).

You want to detect pulses that have a range in amplitude of [1, 255]? There can be pulses with various amplitudes that overlap at the same time?

Could a fourier analysis help you tease apart the pulse waveforms? Are the pulses periodic in any way?

Would an envelope follower help you track the pulses?

Anyway, just throwing stuff out hoping it helps. Good luck.

Leslie Sanford wrote:
This is a shot in the dark since I'm not sure exactly what you're doing (it's probably over my head).

You want to detect pulses that have a range in amplitude of [1, 255]? There can be pulses with various amplitudes that overlap at the same time?

Could a fourier analysis help you tease apart the pulse waveforms? Are the pulses periodic in any way?

Would an envelope follower help you track the pulses?

Anyway, just throwing stuff out hoping it helps. Good luck.

I was thinking Fourier, or perhaps even a discrete wavelet transform.

...that mortally intolerable truth; that all deep, earnest thinking is but the intrepid effort of the soul to keep the open independence of her sea; while the wildest winds of heaven and earth conspire to cast her on the treacherous, slavish shore.

The signal he is trying to demodulate is the Mode-S reply (or others like it)
see here[^] and check out the pulse-position modulated signal at the bottom of the page.

Peter
"Until the invention of the computer, the machine gun was the device that enabled humans to make the most mistakes in the smallest amount of time."

can anyone help me. i have a project that requires me to look for an algorithm,study it, simulate it, the look for the problems of the algorithm when applied to an application..

i chose the "flood fill algorithm" but too bad, i can't seem to find an problems with it

i chose the linear flood fill,,

thanks in advance

here's the general algorithm, in pseudo-code:

startpoint = start point of fill
startcolor = color of pixel at startpoint

fill(startpoint, fillcolor, startcolor);

fill(point p, color fillcolor, color startcolor)
{
  if (p.color != startcolor)
  {
     p.color = fillcolor;

     // fill the surrounding four pixels
     fill(point(p.x+1, p.y), fillcolor, startcolor);
     fill(point(p.x+1, p.y+1), fillcolor, startcolor);
     fill(point(p.x-1, p.y), fillcolor, startcolor);
     fill(point(p.x-1, p.y-1), fillcolor, startcolor);
  }
}

pro-tip:
this is recursive, which can be a problem for large fills. but you can eliminate the recursion using a stack.

image processing toolkits | batch image processing

thank you for this

hi,

where can i find algorithms or artificial intel sites. i have downloaded some ebooks.

Big Grin | :-D

nelsonpaixao@yahoo.com.br

trying to help & get help

What specific information are you looking for?

"The clue train passed his station without stopping." - John Simmons / outlaw programmer

"Real programmers just throw a bunch of 1s and 0s at the computer to see what sticks" - Pete O'Hanlon

"Not only do you continue to babble nonsense, you can't even correctly remember the nonsense you babbled just minutes ago." - Rob Graham

any kinds of algorhitms.
i know a*, i just want to know others, maybe find something i like.

For games, AI, whatever Rose | [Rose]

nelsonpaixao@yahoo.com.br

trying to help & get help

I want an array of colors which can be distinguished eaily by human.
The more colors the better.
At least, manually constructed array is also ok.

Thanks very much.

system

You want to choose red, green, and blue levels as points in a three-dimensional space that are as far apart as possible. This space is a 3D array where each index goes from 0 to 255.

There is also a perceptual component to this problem; a person's perception of color levels is not linear with respect to the numeric indexes. For example, changing from red 20 to red 30 may be perceived as a much different increment than changing from red 120 to red 130.

So one approach is to use the geometric approach to suggest colors that are different, present them to the user, and allow the user to select the colors that are used. An enhancement would be to allow the user to "edit" the color presented to increase the difference to the colors already selected.

In image processing, high contrast colors are often used to display labeled images. That is, image that has gone thru the labeling/connected component process. In that case, to display the resultant image, we use something called "Binary Color Table", which consists of 15 different colors, that are completely different with each other.

These are the colors;

byte bSize = 15; // Binary palette has 15 colors, cycled 15 times.
byte r[] = {255,0,0,255,255,0,255,255,127,127,0,0,255,127,127};
byte g[] = {0,255,0,255,0,255,127,0,255,0,127,255,127,255,127};
byte b[] = {0,0,255,0,255,255,0,127,0,255,255,127,127,127,255};

Take a look also here[^], there are also the text tables for each array.

Russell

I am very worried about a man in the middle attack on my public key cryptography and want to set up a shared key logon. I want the client to have individual shared keys to reduce the likelyhood of a known plaintext/ciphertext attack. I would also really like to set up key distribution so each user recieves a new key for each session and when the session is closed the client recieves a new key to log on. I want this to be like an ssl logon seesion except that I want the entire website encrypted. Could someone please post any good references for a shared key logon or some code,please?

A general suggestion: Depending on a single cipher, no matter how good, is brittle. Using multiple cipher layers makes your code exponentially harder to decrypt.

BTW, how are you distributing keys? If the man in the middle is intercepting communications, he'd also get the new keys.

I've just finished writing a utility to migrate data from optical libraries to a NAS box and now I'm trying to come up with a formula to estimate the time of completion given the following data:

1) Total amount of data
2) No: of drives in library
3) Average read speed of the drives
4) Total no: of files
5) Fixed overhead for each file
6) Average write speed of the NAS box (this also takes into account the network write speed)

The formula that I am using now looks like this:

(Total data / (No: of drives * Average read speed)) + (Total files * Fixed overhead) + (Total data/ Average write speed)

I don't think this right in all cases. The utility launches 1 thread for each drive in the library. So there is some parallelisation of the copy process. But I think the above formula would only work if the copying is done in a sequential manner.

Does anyone have a better idea on how to do this by taking into account that the reads and writes happen in parallel?

Please note that in the program itself, I just use the no: of files processed so far and the time taken to process them to guesstimate the time remaining. This formula is to create an excel file where the user can enter the data given above and get an approximate time of completion before actually starting the migration.

Any help is greatly appreciated.

The user formerly known as pkam.