Click here to Skip to main content
Email Password   helpLost your password?

Introduction

The Intel Streaming SIMD Extensions technology enhance the performance of floating-point operations. Visual Studio .NET 2003 supports a set of SSE Intrinsics which allow the use of SSE instructions directly from C++ code, without writing the Assembly instructions. MSDN SSE topics [2] may be confusing for the programmers who are not familiar with the SSE Assembly progamming. However, reading the Intel Software manuals [1] together with MSDN gives the opportunity to understand the basics of SSE programming.

SIMD is a single-instruction, multiple-data (SIMD) execution model. Consider the following programming task: computing of the square root of each element in a long floating-point array. The algorithm for this task may be written by such way:

for each  f in array
    f = sqrt(f)
Let's be more specific:
for each  f in array
{
    load f to the floating-point register
    calculate the square root
    write the result from the register to memory
}
Processor with the Intel SSE support have eight 128-bit registers, each of which may contain 4 single-precision floating-point numbers. SSE is a set of instructions which allow to load the floating-point numbers to 128-bit registers, perform the arithmetic and logical operations with them and write the result back to memory. Using SSE technology, algorithms may be written as:
for each  4 members in array
{
    load 4 members to the SSE register
    calculate 4 square roots in one operation
    write the result from the register to memory
}
The C++ programmer writing a program using SSE Intrinsics doesn't care about registers. He has a 128-byte __m128 type and a set of functions to perform the arithmetic and logical operations. It's up to the C++ compiler to decide which SSE register to use and to make code optimizations. SSE technology may be used when some operation is done with each element of a long floating-point arrays.

SSE Programming Details

Include Files

All SSE instructions and __m128 data type are defined in xmmintrin.h file:
#include <xmmintrin.h>

Since SSE instructions are compiler intrinsics and not functions, there are no lib-files.

Data Alignment

Each float array processed by SSE instructions should have 16 byte alignment. A static array is declared using the __declspec(align(16)) keyword:
__declspec(align(16)) float m_fArray[ARRAY_SIZE];
Dynamic array should be allocated using new _aligned_malloc function:
m_fArray = (float*) _aligned_malloc(ARRAY_SIZE * sizeof(float), 16);
Array allocated by the _aligned_malloc function is released using the _aligned_free function:
_aligned_free(m_fArray);

__m128 Data Type

Variables of this type are used as SSE instructions operands. They should not be accessed directly. Variables of type _m128 are automatically aligned on 16-byte boundaries.

Detection of SSE Support

SSE instructions may be used if they are supported by the processor. The Visual C++ CPUID sample [4] shows how to detect support of the SSE, MMX and other processor features. It is done using the cpuid Assembly command. See details in this sample and in the Intel Software manuals [1].

SSETest Demo Project

SSETest project is a dialog-based application which makes the following calculation with three float arrays:
fResult[i] = sqrt( fSource1[i]*fSource1[i] + fSource2[i]*fSource2[i] ) + 0.5

i = 0, 1, 2 ... ARRAY_SIZE-1
ARRAY_SIZE is defined as 30000. Source arrays are filled using sin and cos functions. The Waterfall chart control written by Kris Jearakul [3] is used to show the source arrays and the result of calculations. Calculation time (ms) is shown in the dialog. Calculation may be done using one of three possible ways: C++ function:
void CSSETestDlg::ComputeArrayCPlusPlus(
          float* pArray1,                   // [in] first source array

          float* pArray2,                   // [in] second source array

          float* pResult,                   // [out] result array

          int nSize)                        // [in] size of all arrays

{

    int i;

    float* pSource1 = pArray1;
    float* pSource2 = pArray2;
    float* pDest = pResult;

    for ( i = 0; i < nSize; i++ )
    {
        *pDest = (float)sqrt((*pSource1) * (*pSource1) + (*pSource2)
                 * (*pSource2)) + 0.5f;

        pSource1++;
        pSource2++;
        pDest++;
    }
}
Now let's rewrite this function using the SSE Instrinsics. To find the required SSE Instrinsics I use the following way: Some SSE Intrinsics are composite and cannot be found by this way. They should be found directly in the MSDN Library (descriptions are very short but readable). The results of such search may be shown in the following table:

Required Function Assembly Instruction SSE Intrinsic
Assign float value to 4 components of 128-bit value movss + shufps _mm_set_ps1 (composite)
Multiply 4 float components of 2 128-bit values mulps _mm_mul_ps
Add 4 float components of 2 128-bit values addps _mm_add_ps
Compute the square root of 4 float components in 128-bit values sqrtps _mm_sqrt_ps

C++ function with SSE Intrinsics:

void CSSETestDlg::ComputeArrayCPlusPlusSSE(
          float* pArray1,                   // [in] first source array

          float* pArray2,                   // [in] second source array

          float* pResult,                   // [out] result array

          int nSize)                        // [in] size of all arrays

{
    int nLoop = nSize/ 4;

    __m128 m1, m2, m3, m4;

    __m128* pSrc1 = (__m128*) pArray1;
    __m128* pSrc2 = (__m128*) pArray2;
    __m128* pDest = (__m128*) pResult;


    __m128 m0_5 = _mm_set_ps1(0.5f);        // m0_5[0, 1, 2, 3] = 0.5


    for ( int i = 0; i < nLoop; i++ )
    {
        m1 = _mm_mul_ps(*pSrc1, *pSrc1);        // m1 = *pSrc1 * *pSrc1

        m2 = _mm_mul_ps(*pSrc2, *pSrc2);        // m2 = *pSrc2 * *pSrc2

        m3 = _mm_add_ps(m1, m2);                // m3 = m1 + m2

        m4 = _mm_sqrt_ps(m3);                   // m4 = sqrt(m3)

        *pDest = _mm_add_ps(m4, m0_5);          // *pDest = m4 + 0.5

        
        pSrc1++;
        pSrc2++;
        pDest++;
    }
}
This doesn't show the function using inline Assembly. Anyone who is interested may read it in the demo project. Calculation times on my computer: Execution time should be estimated in the Release configuration, with compiler optimizations.

SSESample Demo Project

SSESample project is a dialog-based application which makes the following calculation with float array:
fResult[i] = sqrt(fSource[i]*2.8)

i = 0, 1, 2 ... ARRAY_SIZE-1
The program also calculates the minimum and maximum values in the result array. ARRAY_SIZE is defined as 100000. Result array is shown in the listbox. Calculation time (ms) for each way is shown in the dialog:

Assembly code performs better because of intensive using of the SSX registers. However, usually C++ code with SSE Intrinsics performs like Assembly code or better, because it is difficult to write an Assembly code which runs faster than optimized code generated by C++ compiler.

C++ function:

// Input: m_fInitialArray

// Output: m_fResultArray, m_fMin, m_fMax

void CSSESampleDlg::OnBnClickedButtonCplusplus()
{
    m_fMin = FLT_MAX;
    m_fMax = FLT_MIN;

    int i;

    for ( i = 0; i < ARRAY_SIZE; i++ )
    {
        m_fResultArray[i] = sqrt(m_fInitialArray[i]  * 2.8f);

        if ( m_fResultArray[i] < m_fMin )
            m_fMin = m_fResultArray[i];

        if ( m_fResultArray[i] > m_fMax )
            m_fMax = m_fResultArray[i];
    }
}
C++ function with SSE Intrinsics:
// Input: m_fInitialArray

// Output: m_fResultArray, m_fMin, m_fMax

void CSSESampleDlg::OnBnClickedButtonSseC()
{
    __m128 coeff = _mm_set_ps1(2.8f);      // coeff[0, 1, 2, 3] = 2.8

    __m128 tmp;

    __m128 min128 = _mm_set_ps1(FLT_MAX);  // min128[0, 1, 2, 3] = FLT_MAX

    __m128 max128 = _mm_set_ps1(FLT_MIN);  // max128[0, 1, 2, 3] = FLT_MIN


    __m128* pSource = (__m128*) m_fInitialArray;
    __m128* pDest = (__m128*) m_fResultArray;

    for ( int i = 0; i < ARRAY_SIZE/4; i++ )
    {
        tmp = _mm_mul_ps(*pSource, coeff);      // tmp = *pSource * coeff

        *pDest = _mm_sqrt_ps(tmp);              // *pDest = sqrt(tmp)


        min128 =  _mm_min_ps(*pDest, min128);
        max128 =  _mm_max_ps(*pDest, max128);

        pSource++;
        pDest++;
    }

    // extract minimum and maximum values from min128 and max128

    union u
    {
        __m128 m;
        float f[4];
    } x;

    x.m = min128;
    m_fMin = min(x.f[0], min(x.f[1], min(x.f[2], x.f[3])));

    x.m = max128;
    m_fMax = max(x.f[0], max(x.f[1], max(x.f[2], x.f[3])));
}

Sources

  1. Intel Software manuals.
  2. MSDN, Streaming SIMD Extensions (SSE). http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html/vcrefstreamingsimdextensions.asp
  3. Waterfall chart control written by Kris Jearakul. http://www.codeguru.com/controls/Waterfall.shtml
  4. Microsoft Visual C++ CPUID sample. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcsample/html/vcsamcpuiddeterminecpucapabilities.asp
  5. Matt Pietrek. Under The Hood. February 1998 issue of Microsoft Systems Journal. http://www.microsoft.com/msj/0298/hood0298.aspx
You must Sign In to use this message board.
 
 
Per page   
 FirstPrevNext
QuestionHow to make it more efficiently
Shang Chieh, Chou
17:46 15 Mar '09  
Hi:
After reading yor article, I knew how to write a simple sse code.
But if my source data is unsigned short.
How to make it more effiently?
Because after aligning the unsigned short data, it still need the same number count of for loop
such as:
to divide Source1 by Source2
it needs 10 count of for loop when using sse
unsigned short Source1[40]
unsigned short Source1[40]

but if I can pack it into float, maybe it can only need 5 count of loop
does it?
and how to make it?
thanks
AnswerRe: How to make it more efficiently
Alex Fr
10:15 16 Mar '09  
Working with integers, you need MMX or SSE2, and not SSE. This is MMX introduction:
http://www.codeproject.com/KB/recipes/mmxintro.aspx[^]

SSE2 has the same capabilities of integer operations, but has larger registers.
Anyway, these technologies are used for huge arrays, there is no sence to use them for small amount of data. Also, on modern computers SSE and MMX do not give such significant performance boost, like on old Pentium III.
GeneralFaulty performance comparison
JAF1234567890
9:38 20 Jul '07  
The SSE C++ and inline assembly timings include two optimizations; that due to SSE calculation s and that due to a factor of 4 loop unrolling. It will be enlightening to compare a loop unrolled native C++ timing with the other two methods.

Jeff

Jeff

GeneralRe: Faulty performance comparison
Andyb1979
3:31 27 Jan '10  
Good point, I actually did this just now (a whole 3 years after your post!)

and the result on my PC is:
(Intel Quad-core Xeon, 1.8GHz)

C++ (Loop unrolled in blocks of 4): 14ms
Asm: 8ms
C++ Intrinsics: 8ms

Also I increased the array size to 1,000,000 to get a better time.

So yes, unrolling that loop does optimise the process (as C++ compiler is probably using SSE itself).

Now I also converted the application to process doubles not floats (as I require doubles) using SSE2 and the results are that the C++ loop unrolled now executes in 11ms vs 8ms for SSE2.

Hmmm. Well I'm doing something wrong.
GeneralRe: Faulty performance comparison
Andyb1979
4:45 27 Jan '10  
.... In fact, the same operation is only marginally slower (15%) in C# when using loop unrolling Blush

I realise this SSE example is not intended for performance, just for demo purposes but it seems there is more than meets the eye when optimizing SSE code for fast execution.
GeneralRe: Faulty performance comparison
Andyb1979
7:07 27 Jan '10  
.. Wait Im smokin' something obviously. I didn't execute the full loop in C#

Used

for(int i = 0; i < ARRAY_SIZE/4; i++)

whereas I should've used

for(int i = 0; i < ARRAY_SIZE; i += 4)

Just getting it to the point where all implementations actually work (and give same result, no memory errors) and I'll post the results.

C# vs
C# Unrolled vs
C++ vs
C++ Unrolled
vs
C++/SIMD Intrinsics vs
SIMD Asm
GeneralSSE instructions!!
minabeh
2:37 14 Jun '07  
Hi....
is there any instruction to add 4 units of 32 bits in packed data type(__m128)???

Thanks..
GeneralA question
lei_ma2003
23:59 18 Apr '07  
Could SSE be used in managed C++ application?
GeneralQuestion
shaihnc
11:41 22 Jul '05  
I have decided to use MSDN insturction for SSE2 programing.

I load up 8 , 16 bit short number into _m128i variable as follow:

_declspec(align(16)) short t1[100000];
_declspec(align(16)) short t2[100000];
__m128i temp1, temp2;
__m128i mul1,mul2;

temp1 = _mm_load_si128((__m128i*) ((short *) &t1[i]));
temp2 = _mm_load_si128((__m128i*) ((short *) &t2[i]));

then I use the _mm_mullo_epi16 function to get the multipication of my variables.

mul1 = _mm_mullo_epi16(temp1,temp2);

So now, I have the lower 16 bit of 32 bit result in mul1. now, I want to be able to add this 8 - 16 bit short values together or be able to seperate them.

I can not find any instruction whcih lets me do thatFrown Frown Frown

Can some one plzzzzzzzzzzzzzzzzz help me.
GeneralRe: Question
punkbuster
19:55 17 Jan '06  
I know this is probably way late and you already figured it out, but did you try a union?
GeneralA question
Sachini M
17:39 7 Jun '05  
Excellent article!

I'm very new to this topic and have a question. When using SSE, does the number of iterations of each loop always have to be a multiple of 4?
Lets say you need to do a check (if statement inside the loop) at every iteration, is there a way to use SSE? or is there any use using it?

Thanks in advance!

Regards,
Sachini
GeneralRe: A question
Alex Fr
3:43 8 Jun '05  
There is no any restriction on number of iterations, but every iteration works with 4 float numbers. This means, array size should be multiple of 4, and number of iterations is array size/4.
GeneralRe: A question
punkbuster
19:46 16 Jan '06  
Well, actually, arrays do not need to be a multiple of 4. What you can do is for the portion that is a multiple of 4, do the SSE instructions, and with what's left over, do the regular way without SSE (which will be at max 3 iterations). This lets your optimization be dynamic across multiple array sizes.

So say you want to mess with an array of size 37. The first 36 you do with the SSE implementation, the last 1 you do with the normal implementation (without SSE).

It was a great question that wasn't addressed in the article. It's best practice to assume when creating such a function using SSE that it allows for arrays of any size.

---
punkbuster
Generalarray memory alignment ?
not_happy0
21:08 11 May '05  

hi,
i am new to SSE, and is wondering, if _m128 data type is "auto-aligned" why
doing a new _m128[xx] is not aligned ? I seems to have to use _aligned_malloc instead ?
thanks in advance
GeneralRe: array memory alignment ?
Alex Fr
4:07 12 May '05  
I have never try to make _m128[] array, I don't know exactly whether it is aligned or not. What is a purpose to make such array? We need _m128 variable to work with SSE registers, input and output vectors should be kept in float array.
Generalperformance loss using SSE
David St. Hilaire
9:18 3 Dec '04  
Thanks for the article.

I executed your sample apps, and there is a significant performance boost when using SSE instead of just C++.   However, the functions I've written in with SSE intrinsics have been taking 2-3 times as long to execute as their C++ counterparts.   Do you know what might cause this?

Below is a function I wrote to get the minimum and maximum values of an array.   This executes in roughly 80-90 microseconds on an array of 640 numbers.   The C++ function that does the same thing takes 28-31 microseconds.   What gives?   The SSE version has to do the memcpy to get the input array aligned correctly, but this only accounts for about 26 microseconds of the difference.   I realize that I'm using shorts instead of floats, but it should still work.   I converted your SSESample program to use shorts and only calculate the min and max of the input array.   The SSE code executed less than twice as fast as the C++ code after that, but it was still faster.

Here's the code:
<code>
void FindArrayMinMax(short *pnArray, long nCount, short &nMin, short &nMax)
{
     short *pnIn = (short*) _aligned_malloc(nCount*sizeof(short), 16);     //     16-byte aligned for SSE
     memcpy(pnIn, pnArray, nCount*sizeof(short));
     long nOutputSize = 4 + nCount%4;
     short *pnMaxOut = (short*) _aligned_malloc(nOutputSize*sizeof(short), 16);
     short *pnMinOut = (short*) _aligned_malloc(nOutputSize*sizeof(short), 16);

     __m64 *pmIter = (__m64*) pnIn;
     __m64 *pmMax = (__m64*) pnMaxOut;
     __m64 *pmMin = (__m64*) pnMinOut;

     *pmMax = *pmMin = *pmIter;     //     save first 4 values as minima and maxima
     long nLoop = nCount/4;
     for (int i=1; i<nLoop; i++)
     {
          //     get next 4 values and compare them to the saved minima and maxima
          pmIter++;
          *pmMax = _mm_max_pi16(*pmIter, *pmMax);
          *pmMin = _mm_min_pi16(*pmIter, *pmMin);
     }

     //     get strays, in case nCount is not a multiple of 4
     short nVal(0);
     nLoop = nCount % 4;
     for (i=1; i<=nLoop; i++)
     {
          nVal = pnIn[nCount-i];
          pnMaxOut[i+3] = nVal;
          pnMinOut[i+3] = nVal;
     }

     //     get max and min indices
     nMax = pnMaxOut[0];
     nMin = pnMinOut[0];
     for (i=1; i<nOutputSize; i++)
     {
          if (nMax < pnMaxOut[i])
               nMax = pnMaxOut[i];
          if (nMin > pnMinOut[i])
               nMin = pnMinOut[i];
     }

     //     cleanup
     _aligned_free(pnIn);
     _aligned_free(pnMinOut);
     _aligned_free(pnMaxOut);
     _mm_empty();
}
</code>
GeneralRe: performance loss using SSE
Alex Farber
9:33 3 Dec '04  
640 is not significant number to use SSE. You need to do this for very long arrays, whuch are used in image processing, graphics, 3D etc.
My second sample shows how to find minimum and maximum, I don't see something similar in your code. Does it give right result? Instead of copying of the whole array to aligned array, you need to start from the first aligned input array member.
Anyway, you need to use MMX for this short numbers, take a look at my MMX article. On Pentium 4 you can use SSE2.
Sorry that I don't try to understand your code, SSE programming takes a lot of time. I can try to do this, but code must be clear, without float-short tricks.
GeneralRe: performance loss using SSE
David St. Hilaire
10:38 3 Dec '04  
Thanks for your response.   I realize that 640 is not a lot of elements, but this function is called many, many, times and it is slowing down my app.
   I do use code similar to yours to find the min and max, except that I'm using _mm_min/max_pi16 instead of _mm_min/max_ps.   It does return the correct result; I've checked it against the C++ version of the function.
   There aren't min and max functions in MMX, but I was able to get it working by using the greater than function.   Unfortunately, it takes more instructions and is a little slower than SSE.   I don't know what you mean by "float-short" tricks in my code.   There were no floats at all in the code that I posted.
   You don't have to read my code if you don't want to.   The example I posted isn't the only time I've had SSE code run slower than C++.   I just thought you or someone else might have some ideas why SSE in general would run slower than normal C++ code.

How do you determine which element of an array is the first aligned input array member?

Thanks again,
Dave

GeneralRe: performance loss using SSE
Alex Farber
22:30 3 Dec '04  
Well, this is my code:

void FindMinMaxC(short* pnArray, int size, short& min, short& max)
{
max = SHRT_MIN;
min = SHRT_MAX;

for ( int i = 0; i < size; i++ )
{
if ( *pnArray < min )
min = *pnArray;

if ( *pnArray > max )
max = *pnArray;

pnArray++;
}
}

void FindMinMaxSSE(short* pnArray, int size, short& min, short& max)
{
int i;

union u
{
__m64 m;
short n[4];
} x;


for ( i = 0; i < 4; i++ )
x.n[i] = SHRT_MIN;

__m64 max64 = x.m;

for ( i = 0; i < 4; i++ )
x.n[i] = SHRT_MAX;

__m64 min64 = x.m;


__m64* pSource = (__m64*) pnArray;

for ( i = 0; i < size/4; i++ )
{
min64 = _mm_min_pi16(*pSource, min64);
max64 = _mm_max_pi16(*pSource, max64);

pSource++;
}

x.m = min64;
min = min(x.n[0],
min(x.n[1],
min(x.n[2],
x.n[3])));

x.m = max64;
max = max(x.n[0],
max(x.n[1],
max(x.n[2],
x.n[3])));
}

I don't care about alignment and array size in the FindMinMaxSSE function, assuming that client does this.
Test results for 1000000 members:
C++ 20 ms
SSE 7 ms

Testing for 10000 members I get 0 in both cases.

Tests must be done in Release configuration. Again, there is no need to use SSE for small arrays. It doesn't matter that you call function many times. Array must be very long to get performance boost from SSE. In your case, use C++ code.
GeneralAMD support
Jens froslev-nielsen
2:20 1 Dec '04  
Thanks for 2 wellwritten articles (sseintro & mmxintro).
Now I wonder do U - or perhaps anybody in here know how to implement/using the 3DNow technology in a same matter as shown in here?.

Generalq: movaps vs. movups
yoaz
9:31 4 Nov '04  
sorry to bother u again with beginner's questions, but i'm quite stuck.
I have a class using SSE. I'm declaring a member private variable:
__declspec(align(16))unsigned char m_nodes[ARRAY_SIZE];
later on i try to use it in an asm block,
movaps	xmm0, [esi]
with esi pointing to the array base address. This however throws an exception, which is because the array is not aligned (the base address should be a multiple of 16, am i right?).
I can't figure it out. why isn't my array aligned?
another, final, question: do you know, or can u point me to the actual performance difference between movaps and movups
thanks

there are no facts, only interpretations
GeneralRe: q: movaps vs. movups
Alex Farber
3:35 5 Nov '04  
1) What is ARRAY_SIZE value? Why variable type is unsigned char and not float? What exception exactly do you have?
2) Take a look at Assembly code generated by C++ compiler from movaps and movups.
GeneralRe: q: movaps vs. movups
yoaz
3:54 5 Nov '04  
Alex Farber wrote: What is ARRAY_SIZE value? it's an int, value=16

Alex Farber wrote: Why variable type is unsigned char and not float? I want to use SSE2 for SIMD operations on 16 bytes

Alex Farber wrote: What exception exactly do you have? SEHException. But it occurs with movaps and not with movups.

Alex Farber wrote: Take a look at Assembly code generated by C++ compiler from movaps and movups. i was hoping to generate the Assembly code myself (working with inline Assembly), but i'll debug again.

Thanks a lot for the suggestions. I've managed to work around this, by using _aligned_malloc, though I have no idea why this aligns member variables, and __declspec(align(16)) doesn't. Any ideas?

thanks again,

there are no facts, only interpretations
GeneralExcelent! + a question
yoaz
0:52 20 Sep '04  
A realy interesting and enlightening article. I have a small question: as I understand, MMX uses mm0-mm7 registers, which are actually CPU floating point registers, whereas SSE/2 uses xmm0-xmm7 registers, which where especially defined for SIMD purposes. And finally, the question(s) -
  1. Does this mean that I can use both types of registers simultaneously?
  2. Does this mean that I can do without the EMMS instruction when writing pure SSE/2 code?
thanks, I realy enjoyed this article

there are no facts, only interpretations
GeneralRe: Excelent! + a question
Alex Farber
1:39 20 Sep '04  
AFAIK, EMMS instruction must be used only with MMX:

The EMMS instruction must be used to clear the MMX™ technology state at the end of all MMX™ technology routines and before calling other procedures or subroutines that may execute floating-point instructions. If a floating-point instruction loads one of the registers in the FPU register stack before the FPU tag word has been reset by the EMMS instruction, a floating-point stack overflow can occur that will result in a floating-point exception or incorrect result.

SSE doesn't require this instruction.
I don't have experience in using SSE2.


Last Updated 11 Jul 2003 | Advertise | Privacy | Terms of Use | Copyright © CodeProject, 1999-2010