I is not wrong optimization. The guy is explicitly breaking the C specification by doing that.
The correct way of accessing a variable like that is trough an union. Here is an example http://pastebin.com/8V3whcL3
Alternatively one can make both of them as volatile. That will basically force the compiler do all the loads and stores.
I really hate it when people write code that breaks the spec and then complains to the compiler maker. And now they actually changed the compiler to work with the code that is simply wrong.
Just as what happened with glibc memcpy (works like memmove due to people not knowing what they are doing and then crying for the libraries to change their behaviour to match their own incompetence).
> The correct way of accessing a variable like that is trough an union.
It's funny you should say that, because people have been taking the exact opposite viewpoint for years: “union do not allow type-punning because it was only clarified in TC3 of C99 that they do. memcpy has always been the correct way to type-pun.”
The bug reporter is referring to the rules described in 6.5:6 and 6.5:7 of C11 (the same rules are included in C99, but interestingly not in C90, meaning that “gcc -std=c90” should go theoretically easy with the type-based optimizations. It doesn't). Here is a link: http://port70.net/~nsz/c/c11/n1570.html#6.5p6
And an excerpt for your convenience:
If a value is copied into an object having no declared type using memcpy or memmove, or […], then the effective type of the modified object for that access and for subsequent accesses that do not modify the value is the effective type of the object from which the value is copied, if it has one
I've been under the impression that unions don't truly allow type punning, but given that the practice was widespread (e.g., it was used in the code Protocol Buffers generated a few years ago), it was a pretty safe bet compilers would support it as an extension.
I was also under the impression that the One True Way to type pun was to cast to char*, which is allowed to alias, and then cast again to the type you actually want.
I was also under the impression that the One True Way to type pun was to cast to char, which is allowed to alias, and then cast again to the type you actually want.*
That clearly doesn't work, but I can see from a naive perspective why some think that. It feels like a nice hack, but a cast to char* doesn't "remove" the original type.
To be clear: char* can alias any type, but only char* can alias char*.
Casting through chard doesn't help: how do you get to a specific pointer value doesn't really matter. The C aliasing rules are only about dereferencing pointers.
What is allowed, as an explicit exception to the aliasing rules, is reading an object representation by dereferencing a char pointer. This is why memcpy is the approved way to do type punning, but you do have to copy.
int a = 5;
char *b = memcpy(malloc(sizeof(a)), &a, sizeof(a));
...should fail. The memcpy(3) src is &a, a pointer-to-int; the memcpy(3) dest is a pointer-to-void (i.e. an untyped pointer) from malloc(3). Thus, memcpy(3)'s return type should be a pointer-to-int.
It is, yes, according to the typespec of memcpy(3). But according to the standard, it shouldn't be; it should be returning a [whatever-type-src-was-passed-in-as]. I have doubt that there's any way in C to specify variable return type like that—but the C standard, AFAIK, doesn't require that memcpy(3)—or anything else from libc—be implementable in C. It can be, as Lisp would put it, a "special form."
To get an implicit pointer-to-char-typed buffer out of memcpy(3), you should have to attach the malloc(3)-returned buffer to a pointer-to-char-typed variable (or give it an explicit cast) to give it a type, and then pass it in as dest.
The guy is explicitly breaking the C specification by doing that.
You assert this, but clearly there are people in the bug thread who believe otherwise. In the thread, the submitter references a portion of the C11 spec that he feels defends his usage:
C11, 6.5p6: If a value is stored into an object having no declared type
through an lvalue having a type that is not a character type, then the
type of the lvalue becomes the effective type of the object for that
access and for subsequent accesses that do not modify the stored value.
I'm not familiar enough with the spec to know the context, nor would I defend this code as good practice, but superficially this seems to be evidence that this is a bug. Is there another specification that can you point to that this "explicitly" breaks?
C11, 6.5p7: An object shall have its stored value accessed only by an lvalue expression that has one of
the following types:
— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the object,
— a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object,
— an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or
— a character type
---------------------
If we interprete 6.5p6 like the submitter did then the whole p7 becomes meaningless for any mallocs. In that case everything always aliases everything ever when it's trough a malloced pointer. Which I hardly imagine is the point of the former paragraph. If so they have fundamentally changed the whole function of malloc. Which makes it odd as in C99 (The p6 wording was introduced there. It was not there in C89) they explicitly allowed unions to alias, just due to this issue. Would have been unnecessary if malloced things always alias.
The wording is not clear however. It would be nice to see the standards committee to publish a comment about this. This is also not the first thing in the spec that is unclear.
6.5:6 is certainly not saying that malloced memory always aliases with “everything”. Quite the contrary, it says that malloced memory cannot be used for type-punning. The following works and is idiomatic:
uint32_t u;
float f = …;
memcpy(&u, &f, sizeof u);
/* use u as a uint32_t in computations */
You cannot do the same thing with malloced memory: if u had been malloced memory instead of a variable, it would have been illegal to access that memory with an lvalue of type uint32_t in subsequent computations. 6.5:6 is certainly no license to use malloc'ed memory any which way.
There is a non-normative note in 6.5:6 "Allocated objects have no declared type."
Sure that is not strictly part of the standard. But helps in interpreting it. I'd say it quite clearly refers to malloced objects. Malloced object is an object with no declared type.
If you look at the bug report it uses malloced object as the example. And accesses it using two pointers of different types.
I think you are misinterpreting Pascal's example. He's saying that because 'u' is a variable with a declared type (uint32_t), it can still be used as that type even after it is written to with another type. He then claims that were it an "allocated object" with no declared type, the effective type would change to the type of the most recent write (float), and thus it could no longer be used as the original type (uint32_t).
I'm not sure he's right, though. The quoted part of the standard says "stored into an object having no declared type through an lvalue having having a type that is not a character type". I would think that "through an lvalue" is referring only to the case given in the bug report, where the store is expressed as "*ptr = val", and not to the case where ptr is used as an argument to memcpy(). Am I wrong?
What you're missing is that storing to an object doesn't "access it's stored value".
Once you've stored to the malloced memory through an int-typed lvalue, it's effective type becomes int, and you can't access it (ie. read it) through a double-typed lvalue. But you can then write to it through a double-typed lvalue, because that doesn't access it's stored value - it overwrites it, and changes its effective type to double.
There is a lot of code written and released that doesn't follow the standard to the letter. New compilers with new optimizations that require strict language adherence otherwise they generate broken programs cause issues for the developers but, most importantly, the users.
The memcpy/memmove debacle could have been avoided entirely if memcpy performed a simple check about what was being copied where and changed the direction of the copy based on that. Simple, cheap, and an entire class of bugs can be avoided.
People should still be aware of the behavior of their development system but the system shouldn't punish when a slightly wrong choice is made.
Optimizations are great but software that continues to function is even better.
Instead of just having a C or C++ program you actually have MSVC++ program (this is actually really common) or GCC program or clang program.
Sure one can argue that the C standard is too lax. But these kind of errors come simply from the fact that people learn C by trial and error, which doesn't work with that language.
Personally as long as the compilers still allow one to enable the optimizations it's not that bad in the end. Portability issues for badly written programs remain however.
-fstrict-aliasing is on by default in gcc at -O2 and above. I'm surprised that the bug discussion doesn't mention that.
The PathScale compiler had a fair number of customers who complained about "bugs" that went away with -fnostrict-aliasing. We told them to use -fnostrict-aliasing.
I think it's quite obvious strict aliasing is implied in this case. The bug is than even with strict aliasing he should get the result he wants, because the last time he wrote to the pointer before reading it as an int he wrote it as an int and hence it should keep the int-value 1.
Having that said, balancing aliasing like this is just asking for trouble, don't try this at home.
EDIT: The more i look at it the more i think he is wrong. Since pi and pd shouldn't be able to alias, the last assignment to both of them is allowed to reorder. This is just UB.
If the possibility of this flaw were covered by a regression test, then the flaw wouldn't exist.
As long as bugs exist and are found in the field, we can continue to ask the question why these bugs were not covered by testing.
The other part is that bugs are found because "extensive" isn't anywhere near "exhaustive".
For this given bug, there exists some requirements in the ISO C standard. We could ask, why don't we have coverage of all ISO C requirements in the test suite. But even if we had that coverage of all requirements, it won't catch all bugs like this. There are ways of validating these requirements such that the bug won't show up.
The requirements end up violated here through an obscure cascade of unintended consequences, not through a lack of implementation. The repro test case is quite contrived. It treats a piece of memory with no declared type as int. It loads the value from that memory into a temporary variable, then temporarily re-purposes them emory as type double, and then re-purposes it again as int---by storing the original, stashed temp value back into it. This happens all in the same scope. Because the re-purposing of memory is valid, the store back of the temporary variable isn't in fact redundant in this case, as the optimizer is programmed to think.
Catching the problem requires the specific combination of testing the redundancy elimination together with code which re-purposes memory twice in the same scope.
Suppose that you have 12 different optimizations which take advantage of strict aliasing, and 11 of them respect the concept of "effective type" for an object of no declared type. To catch the problem, you need to write the code which exercises requirements related to "effective type", and which elicits that one buggy optimization out of the compiler.
Such combinations of interacting features this explode the testing space. It's not enough to test them in isolation. That optimization, tested in isolation, looks fine. "Effective type" is honored just fine---in absence of that optimization.
No, apparently a gcc developer agreed that there was a regression and it wasn't working as intended. They've just fixed a reduced test case [1], although according to the original reporter, there remains an additional problem to be fixed.
It treats "int a=* pi; * pi=a" as a redundant store and removes it, as explained in the 5th comment:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69776#c5