Home/ Questions/Q 168825
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-11T12:31:17+00:00

In case of integer overflows what is the result of (unsigned int) * (int)

  • 0

In case of integer overflows what is the result of (unsigned int) * (int) ? unsigned or int? What type does the array index operator (operator[]) take for char*: int, unsigned int or something else?

I was auditing the following function, and suddenly this question arose. The function has a vulnerability at line 17.

// Create a character array and initialize it with init[]  // repeatedly. The size of this character array is specified by  // w*h. char *function4(unsigned int w, unsigned int h, char *init) {     char *buf;     int i;      if (w*h > 4096)         return (NULL);      buf = (char *)malloc(4096+1);     if (!buf)         return (NULL);      for (i=0; i<h; i++)         memcpy(&buf[i*w], init, w);  // line 17      buf[4096] = '\0';      return buf; }

Consider both w and h are very large unsigned integers. The multiplication at line 9 have a chance to pass the validation.

Now the problem is at line 17. Multiply int i with unsigned int w: if the result is int, it is possible that the product is negative, resulting in accessing a position that is before buf. If the result is unsigned int, the product will always be positive, resulting in accessing a position that is after buf.

It’s hard to write code to justify this: int is too large. Does anyone has ideas on this?

Is there any documentation that specifies the type of the product? I have searched for it, but so far haven’t found anything.

I suppose that as far as the vulnerability is concerned, whether (unsigned int) * (int) produces unsigned int or int doesn’t matter, because in the compiled object file, they are just bytes. The following code works the same no matter the type of the product:

unsigned int x = 10; int y = -10;  printf('%d\n', x * y);  // print x * y in signed integer printf('%u\n', x * y);  // print x * y in unsigned integer

Therefore, it does not matter what type the multiplication returns. It matters that whether the consumer function takes int or unsigned.

The question here is not how bad the function is, or how to improve the function to make it better. The function undoubtedly has a vulnerability. The question is about the exact behavior of the function, based on the prescribed behavior from the standards.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. To answer your question: the type of an expression multiplying an int and an unsigned int will be an unsigned int in C/C++.

    To answer your implied question, one decent way to deal with possible overflow in integer arithmetic is to use the ‘IntSafe‘ set of routines from Microsoft:

    http://blogs.msdn.com/michael_howard/archive/2006/02/02/523392.aspx

    It’s available in the SDK and contains inline implementations so you can study what they’re doing if you’re on another platform.

    • 0