Click here to Skip to main content
Click here to Skip to main content

A nice 64-bit error in C

By , 30 Nov 2009
Rate this:
Please Sign up or sign in to vote.

In C language, you may use functions without defining them. Pay attention that I speak about C language, not C++. Of course, this ability is very dangerous. Let us have a look at an interesting example of a 64-bit error related to it. Below is the correct code that allocates and uses three arrays, 1 GB each:

#include <stdlib.h>

void test()
  const size_t Gbyte = 1024 * 1024 * 1024;
  size_t i;
  char *Pointers[3];

  // Allocate
  for (i = 0; i != 3; ++i)
    Pointers[i] = (char *)malloc(Gbyte);

  // Use
  for (i = 0; i != 3; ++i)
    Pointers[i][0] = 1;

  // Free
  for (i = 0; i != 3; ++i)

This code correctly allocates memory, writes one into the first item of each array and frees the allocated memory. The code is absolutely correct on a 64-bit system.

Now delete or comment the line “#include <stdlib.h>”. The code still compiles but the program crashes after the launch. As the header file “stdlib.h” is disabled, the C compiler considers that malloc function will return int type. The first two allocations are most likely to be successful. After the third call, malloc function will return the array’s address outside the range of the first two Gbyte. As the compiler considers the function’s result to have int type, it interprets the result incorrectly and saves the incorrect value of the pointer in Pointers array.

To make it clearer, let us consider an assembler code generated by Visual C++ compiler for the 64-bit Debug version. At first look at the correct code generated when malloc function is defined (i.e. the file “stdlib.h” is included):

Pointers[i] = (char *)malloc(Gbyte);
mov   rcx,qword ptr [Gbyte]
call  qword ptr [__imp_malloc (14000A518h)]
mov    rcx,qword ptr [i]
mov    qword ptr Pointers[rcx*8],rax

Now consider the variant of the incorrect code when malloc function is not defined:

Pointers[i] = (char *)malloc(Gbyte);
mov    rcx,qword ptr [Gbyte]
call   malloc (1400011A6h)
mov    rcx,qword ptr [i]
mov    qword ptr Pointers[rcx*8],rax

Consider the CDQE instruction (convert doubleword to quadword). The compiler supposed the result to be kept in eax registers and extended it to a 64-bit value to write into Pointers array. Respectively, the high-order bits of rax register are lost. Even if the address of the allocated memory is inside the range of the first 4 GB, we still get the incorrect result when the high-order bit of eax register equals 1. For example, the address 0x81000000 turns into 0xFFFFFFFF81000000.

Fortunately, this type of error is easy to define. For example, Visual C++ compiler generates two warnings informing about a potential problem:

warning C4013: ‘malloc’ undefined; assuming extern returning int
warning C4312: ‘type cast’ : conversion from ‘int’ to ‘char *’ of greater size


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Russian Federation Russian Federation

We develop the PVS-Studio static code analyzer for C/C++/C++11.

PVS-Studio is a static code analyzer for C/C++ (Visual Studio 2005/2008/2010) with a simple licensing and pricing policies which is easy to install and use without need to deploy a complex maintenance environment.

Follow on   Twitter

Comments and Discussions

QuestionDoes anyone ignore 'undefined function' warnings? Pinmembersupercat91-Dec-09 6:06 
I suppose that because the return of malloc() is being typecast, there won't be a warning about casting an integer to a pointer, but there would still certainly be a warning about calling a function with no definition. Although C may technically allow such usage, I wouldn't expect any remotely-decent program to go into production with even one such warning outstanding.
BTW, I've sometimes wondered whether it would be useful to have a compiler option to warn if a non-static function is used without a prototype-only declaration, even if the function is defined earlier. In the style of project I much prefer, every function will have exactly one prototype in the project, in a header file that is #include'd by the module that defines the function. This will ensure that the prototype will be validated against the function definition. If during refactoring, however, a function gets moved from one module to another without its prototype being moved as well, it's possible to lose such validation. If the function definition is later changed, that may break things. Note that on some compilers, a 'const' qualifier on a pointer may substantially change how it's handled e.g. if a pointer large enough to handle data space is not large enough to handle code space as well), so even a change from const to non-const or vice versa can break things.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140421.2 | Last Updated 1 Dec 2009
Article Copyright 2009 by
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid