12,829,374 members (39,339 online)
alternative version

#### Stats

304.7K views
91 bookmarked
Posted 18 Aug 2003

# Basic concepts on Endianness

, 18 Aug 2003
 Rate this:
A beginner introduction to Endianness.

## Introduction

A long time ago, in a very remote island known as Lilliput, society was split into two factions: Big-Endians who opened their soft-boiled eggs at the larger end ("the primitive way") and Little-Endians who broke their eggs at the smaller end. As the Emperor commanded all his subjects to break the smaller end, this resulted in a civil war with dramatic consequences: 11.000 people have, at several times, suffered death rather than submitting to breaking their eggs at the smaller end. Eventually, the 'Little-Endian' vs. 'Big-Endian' feud carried over into the world of computing as well, where it refers to the order in which bytes in multi-byte numbers should be stored, most-significant first (Big-Endian) or least-significant first (Little-Endian) to be more precise [1]

• Big-Endian means that the most significant byte of any multibyte data field is stored at the lowest memory address, which is also the address of the larger field.
• Little-Endian means that the least significant byte of any multibyte data field is stored at the lowest memory address, which is also the address of the larger field.

For example, consider the 32-bit number, 0xDEADBEEF. Following the Big-Endian convention, a computer will store it as follows:

Figure 1. Big-Endian: The most significant byte is stored at the lowest byte address.

Whereas architectures that follow the Little-Endian rules will store it as depicted in Figure 2:

Figure 2. Little-endian: Least significant byte is stored at the lowest byte address.

The Intel x86 family and Digital Equipment Corporation architectures (PDP-11, VAX, Alpha) are representatives of Little-Endian, while the Sun SPARC, IBM 360/370, and Motorola 68000 and 88000 architectures are Big-Endians. Still, other architectures such as PowerPC, MIPS, and Intel’s 64 IA-64 are Bi-Endian, i.e. they are capable of operating in either Big-Endian or Little-Endian mode. [1].

Endianess is also referred to as the NUXI problem. Imagine the word UNIX stored in two 2-byte words. In a Big-Endian system, it would be stored as UNIX. In a little-endian system, it would be stored as NUXI.

## Which format is better?

Like the egg debate described in the Gulliver's Travels, the Big- .vs. Little-Endian computer dispute has much more to do with political issues than with technological merits. In practice, both systems perform equally well in most applications. There is however a significant difference in performance when using Little-Endian processors instead of Big-Endian ones in network devices (more details below).

## How to switch from one format to the other?

It is very easy to reverse a multi-byte number if you need the other format, it is simply a matter of swapping bytes and the conversion is the same in both directions. The following example shows how an Endian conversion function could be implemented using simple C `union`s:

```unsigned long ByteSwap1 (unsigned long nLongNumber)
{
union u {unsigned long vi; unsigned char c[sizeof(unsigned long)];};
union v {unsigned long ni; unsigned char d[sizeof(unsigned long)];};
union u un;
union v vn;
un.vi = nLongNumber;
vn.d[0]=un.c[3];
vn.d[1]=un.c[2];
vn.d[2]=un.c[1];
vn.d[3]=un.c[0];
return (vn.ni);
}```

Note that this function is intented to work with 32-bit integers.

A more efficient function can be implemented using bitwise operations as shown below:

```unsigned long ByteSwap2 (unsigned long nLongNumber)
{
return (((nLongNumber&0x000000FF)<<24)+((nLongNumber&0x0000FF00)<<8)+
((nLongNumber&0x00FF0000)>>8)+((nLongNumber&0xFF000000)>>24));
}```

And this is a version in assembly language:

```unsigned long ByteSwap3 (unsigned long nLongNumber)
{
unsigned long nRetNumber ;

__asm
{
mov eax, nLongNumber
xchg ah, al
ror eax, 16
xchg ah, al
mov nRetNumber, eax
}

return nRetNumber;
}```

A 16-bit version of a byte swap function is really straightforward:

```unsigned short ByteSwap4 (unsigned short nValue)
{
return (((nValue>> 8)) | (nValue << 8));

}```

Finally, we can write a more general function that can deal with any atomic data type (e.g. `int`, `float`, `double`, etc) with automatic size detection:

```#include <algorithm> //required for std::swap

#define ByteSwap5(x) ByteSwap((unsigned char *) &x,sizeof(x))

void ByteSwap(unsigned char * b, int n)
{
register int i = 0;
register int j = n-1;
while (i<j)
{
std::swap(b[i], b[j]);
i++, j--;
}
}```

For example, the next code snippet shows how to convert a data array of `double`s from one format (e.g. Big-Endian) to the other (e.g. Little-Endian):

```double* dArray; //array in big-endian format
int n; //Number of elements

for (register int i = 0; i <n; i++)
ByteSwap5(dArray[i]);```

Actually, in most cases, you won't need to implement any of the above functions since there are a set of socket functions (see Table I), declared in winsock2.h, which are defined for TCP/IP, so all machines that support TCP/IP networking have them available. They store the data in 'network byte order' which is standard and endianness independent.

FunctionPurpose
`ntohs`Convert a 16-bit quantity from network byte order to host byte order (Big-Endian to Little-Endian).
`ntohl`Convert a 32-bit quantity from network byte order to host byte order (Big-Endian to Little-Endian).
`htons`Convert a 16-bit quantity from host byte order to network byte order (Little-Endian to Big-Endian).
`htonl`Convert a 32-bit quantity from host byte order to network byte order (Little-Endian to Big-Endian).

Table I: Windows Sockets Byte-Order Conversion Functions [2]

The socket interface specifies a standard byte ordering called network-byte order, which happens to be Big-Endian. Consequently, all network communication should be Big-Endian, irrespective of the client or server architecture.

Suppose your machine uses Little Endian order. To transmit the 32-bit value `0x0a0b0c0d` over a TCP/IP connection, you have to call `htonl()` and transmit the result:

`TransmitNum(htonl(0x0a0b0c0d)); `

Likewise, to convert an incoming 32-bit value, use `ntohl()`:

`int n = ntohl(GetNumberFromNetwork()); `

If the processor on which the TCP/IP stack is to be run is itself also Big-Endian, each of the four macros (i.e. `ntohs`, `ntohl`, `htons`, `htonl`) will be defined to do nothing and there will be no run-time performance impact. If, however, the processor is Little-Endian, the macros will reorder the bytes appropriately. These macros are routinely called when building and parsing network packets and when socket connections are created. Serious run-time performance penalties occur when using TCP/IP on a Little-Endian processor. For that reason, it may be unwise to select a Little-Endian processor for use in a device, such as a router or gateway, with an abundance of network functionality. (Excerpt from reference [1]).

One additional problem with the host-to-network APIs is that they are unable to manipulate 64-bit data elements. However, you can write your own `ntohll()` and `htonll()` corresponding functions:

• `ntohll`: converts a 64-bit integer to host byte order.
• `ntonll`: converts a 64-bit integer to network byte order.

The implementation is simple enough:

```#define ntohll(x) (((_int64)(ntohl((int)((x << 32) >> 32))) << 32) |
(unsigned int)ntohl(((int)(x >> 32)))) //By Runner
#define htonll(x) ntohll(x)```

## How to dynamically test for the Endian type at run time?

As explained in Computer Animation FAQ, you can use the following function to see if your code is running on a Little- or Big-Endian system:

```#define BIG_ENDIAN      0
#define LITTLE_ENDIAN   1

int TestByteOrder()
{
short int word = 0x0001;
char *byte = (char *) &word;
return(byte[0] ? LITTLE_ENDIAN : BIG_ENDIAN);
}```

This code assigns the value `0001h` to a 16-bit integer. A char pointer is then assigned to point at the first (least-significant) byte of the integer value. If the first byte of the integer is `0x01h`, then the system is Little-Endian (the `0x01h` is in the lowest, or least-significant, address). If it is `0x00h` then the system is Big-Endian.

Similarly,

```bool IsBigEndian()
{
short word = 0x4321;
if((*(char *)& word) != 0x21 )
return true;
else
return false;
}```

which is just the reverse of the same coin.

You can also use the standard byte order API’s to determine the byte-order of a system at run-time. For example:

`bool IsBigEndian() { return( htonl(1)==1 ); }`

## Auto detecting the correct Endian format of a data file

Suppose you are developing a Windows application that imports Nuclear Magnetic Resonance (NMR) spectra. High resolution NMR files are generally recorded in Silicon or Sun Workstations but recently Windows or Linux based spectrometers are emerging as practical substitutes. It turns out that you will need to know in advance the Endian format of the file to parse correctly all the information. Here are some practical guidelines you can follow to decipher the correct Endianness of a data file:

1. Typically, the binary file includes a header with the information about the Endian format.
2. If the header is not present , you can guess the Endian format if you know the native format of the computer from which the file comes from. For instance, if the file was created in a Sun Workstation, the Endian format will most likely be Big-Endian.
3. If none of the above points apply, the Endian format can be determined by trial and error. For example, if after reading the file assuming one format, the spectrum does not make sense, you will know that you have to use the other format.

If the data points in the file are in floating point format (`double`), then the `_isnan()` function can be of some help to determine the Endian format. For example:

```double dValue;
FILE* fp;
(...)
bool bByteSwap = _isnan(dValue) ? true : false```

Note that this method does only guarantee that the byte swap operation is required if `_isnan()` returns a nonzero value (`TRUE`); if it returns 0 (`FALSE`), then it is not possible to know the correct Endian format without further information.

## Acknowledgments

Thanks to Santiago Domínguez, Ehsan Akhgari, Santiago Fraga and Ignacio Sordo for their helpful suggestions.

## References

1. <A name=[1]>Introducction to Endianness, by Michael Barr, Embedded Systems Programming.
2. <A name=[2]>Visual C++ Concepts: Adding Functionality. Windows Sockets: Byte Ordering

A list of licenses authors might use can be found here

## Share

 Spain
No Biography provided

## You may also be interested in...

 View All Threads First Prev Next
 SIMD Optimized SwapEndian function for large Unicode Strings immo26-Aug-03 10:28 immo 26-Aug-03 10:28
 This function is designed to swap endian of large memory buffer (reprezented by "memory" class in this listing) extremaly fast, using various code for different CPUs (you must use CPUID function or sth like it, look at "cpu_info" class).```static void cvSwapEndian() { // copyright by Piotr Sawicki // please send me info if You are using this code: immo@medialab.pl if ( !memory.GetSize() ) return; #ifdef CV_ASSEMBLY_OFF if ( cpu_info.dwFeatures & CV_CPU_MMX ) { int len = (memory.GetSize() >> 3) + 1; __m64 *src = (__m64*)(BYTE*)memory; src += len; len = -len; do src[len] = _m_por(_m_psllwi(src[len], 8), _m_psrlwi(src[len], 8)); while ( ++len != 0 ); _m_empty(); } else { int len = (memory.GetSize() >> 2) + 1; DWORD *src = (DWORD*)(BYTE*)memory; src += len; len = -len; do src[len] = ((src[len] >> 8) & 0x00ff00ff) | ((src[len] << 8) & 0xff00ff00); while ( ++len != 0 ); } #else // CV_ASSEMBLY_OFF if ( cpu_info.dwFeatures & CV_CPU_SSE2 ) { DWORD src = (DWORD)(PBYTE)memory; DWORD len = (memory.GetSize() + 64) & -64; __asm { mov esi, DWORD PTR [src] mov ecx, DWORD PTR [len] lea esi, DWORD PTR [esi+ecx] neg ecx align 16 \$sse2_simd: mov eax, DWORD PTR [esi+ecx+64] ; prefetching movdqa xmm0, XMMWORD PTR [esi+ecx+0*16] ; mov double quad movdqa xmm2, XMMWORD PTR [esi+ecx+1*16]; movdqa xmm4, XMMWORD PTR [esi+ecx+2*16]; movdqa xmm6, XMMWORD PTR [esi+ecx+3*16]; movdqa xmm1, xmm0 movdqa xmm3, xmm2 movdqa xmm5, xmm4 movdqa xmm7, xmm6 psllw xmm1, 8 ; packed shift left logical word psrlw xmm0, 8 ; packed shift right logical word por xmm1, xmm0 ; packed or psllw xmm3, 8 psrlw xmm2, 8 por xmm3, xmm2 psllw xmm5, 8 psrlw xmm4, 8 por xmm5, xmm4 psllw xmm7, 8 psrlw xmm6, 8 por xmm7, xmm6 movdqa XMMWORD PTR [esi+ecx+0*16], xmm1 movdqa XMMWORD PTR [esi+ecx+1*16], xmm3 movdqa XMMWORD PTR [esi+ecx+2*16], xmm5 movdqa XMMWORD PTR [esi+ecx+3*16], xmm7 add ecx, 64 jnz \$sse2_simd } } else if ( cpu_info.dwFeatures & CV_CPU_MMX ) { DWORD src = (DWORD)(PBYTE)memory; DWORD len = (memory.GetSize() + 32) & -32; __asm { mov esi, DWORD PTR[src] mov ecx, DWORD PTR[len] shr ecx, 3 lea esi, DWORD PTR[esi+ecx*8] neg ecx align 16 \$mmx_simd: mov eax, DWORD PTR[esi+ecx*8+32] ; prefetching movq mm0, MMWORD PTR[esi+ecx*8] ; mov quad movq mm2, MMWORD PTR[esi+ecx*8+8]; movq mm4, MMWORD PTR[esi+ecx*8+16]; movq mm6, MMWORD PTR[esi+ecx*8+24]; movq mm1, mm0 movq mm3, mm2 movq mm5, mm4 movq mm7, mm6 psllw mm1, 8 ; packed shift left logical word psrlw mm0, 8 ; packed shift right logical word por mm1, mm0 ; packed or psllw mm3, 8 psrlw mm2, 8 por mm3, mm2 psllw mm5, 8 psrlw mm4, 8 por mm5, mm4 psllw mm7, 8 psrlw mm6, 8 por mm7, mm6 movq MMWORD PTR[esi+ecx*8], mm1 movq MMWORD PTR[esi+ecx*8+8], mm3 movq MMWORD PTR[esi+ecx*8+16], mm5 movq MMWORD PTR[esi+ecx*8+24], mm7 add ecx, 4 jnz \$mmx_simd emms; ; empty mmx flag } } else { DWORD src = (DWORD)(PBYTE)memory; DWORD len = (memory.GetSize() + 4) & -4; __asm { mov esi, DWORD PTR[src] mov ecx, DWORD PTR[len] shr ecx, 2 lea esi, DWORD PTR[esi+ecx*4] neg ecx align 16 \$386_simd: mov eax, DWORD PTR[esi+ecx*4] mov edx, eax shr eax, 8 shl edx, 8 xor eax, edx and eax, 00ff00ffH xor eax, edx mov DWORD PTR[esi+ecx*4], eax inc ecx jnz \$386_simd } } #endif // CV_ASSEMBLY_OFF } ``` Piotr Sawickihome page: http://www.medialab.pl/cvservice/GCS/IT d-(+) s+: a-- C++ UL++ !P L++@ !E W+++\$ N+ o+ !K w+++ !O M- !VPS+ PE+ Y+ !PGP t 5? X+ R tv- b++ DI+ D++ G e++>+++ h--- r+++ y+++*
 Last Visit: 31-Dec-99 19:00     Last Update: 29-Mar-17 19:08 Refresh 1