Question

The following code

char buffer[BSIZE];
...
if(buffer[0]==0xef)
...

Gives the compiler warning “comparison is always false due to limited range of data type”.
The warning goes away when I change the check to

if(buffer[0]==0xffffffef)

This feels very counterintuitive. What is the correct way to check a char against a specific byte value in hex? (other than making it unsigned)

Answer 1

To understand why buffer[0] == 0xef triggers a warning, and buffer[0] == 0xffffffef does not, you need to understand exactly what’s happening in that expression.

Firstly, the == operator compares the value of two expressions, not the underlying representation – 0xef is the number 239, and will only compare equal to that number; likewise 0xffffffef is the number 4294967279 and will only compare equal to that.

There is no difference between the constants 0xef and 239 in C: both have type int and the same value. If your char has the range -128 to 127, then when you evaluate buffer[0] == 0xef the buffer[0] is promoted to int, which leaves its value unchanged. It can therefore never compare equal to 0xef, so the warning is correct.

However, there is potentially a difference between the constants 0xffffffef and 4294967279; decimal constants are always signed, but hexadecimal constant may be unsigned. On your system, it appears to have an unsigned type – probably unsigned int (because the value is too large to store in an int, but small enough to store in an unsigned int). When you evaluate buffer[0] == 0xffffffef, the buffer[0] is promoted to unsigned int. This leaves any positive value unchanged, but negative values are converted by adding UINT_MAX + 1 to them; with a char that has range -128 to 127, the promoted values are in either of the ranges 0 to 127 or 4294967168 to 4294967295. 0xffffffef lies within this range, so it is possible for the comparison to return true.

---

If you are storing bit patterns rather than numbers, then you should be using unsigned char in the first place. Alternatively, you may inspect the bit pattern of an object by casting a pointer to it to unsigned char *:

if (((unsigned char *)buffer)[0] == 0xef)

(This is obviously more conveniently done by using a separate variable of type unsigned char *).

As PaulR says, you can also use buffer[0] == '\xef' – this works because '\xef' is defined to be an int constant with the value that a char object with the bit pattern 0xef would have when converted to an int; eg. on a 2s complement system with signed chars, '\xef' is a constant with the value -17.