C Serial Port read and calculate average

Based on your comment to Richard, you've already got the data as strings. What you need to do is
(1) Convert a string to a number
(2) Save this number
(3) Increment your Index to point at the next string
(4) Repeat steps 1 - 3 until you have enough samples
(5) Add N samples together then divide by N.

From your example data, consider the following:

// max number of values to average and the index of current one
int N=5, i;

// space for 5 vales
double values[5];

double total, avg;

for (i=0; i<N; i++)
{
    values[i] = atof( &buffer[i*5] );
}

total = 0;
for (i=0; i<N; i++)
    total += values[i];

avg = total / N;

Note, you could avoid storing the converted numbers into the values array, instead simply clearing the total before the loop that uses atof and adding the result of atof to the total instead of storing it. This would have the advantage of not requiring the values array. It would have the disadvantage of making error checking (which I've not done) a litle trickier. You could assign the return of atof to a temporary value, ensuring that its a valid number before adding it to the total and incrementing another counter that stored the number of successfully converted numbers.