LoDThe's blog

Compilation in GNU G++ 5.1.0
By LoDThe, history, 7 years ago, translation, In English

GNU G++ 5.1.0 doesn't compile next code:

#include <vector>

using namespace std;

vector <pair <int, int>> a(0);

int main()
{
	return 0;
}

But succeыsfully compile this code: 

#include <vector>

using namespace std;

#define pii pair <int, int>

vector <pii> a(0);

int main()
{
	return 0;
}

Why the second code compile without space (" ") after "pii"?

  • Vote: I like it
  • +7
  • Vote: I do not like it

»
7 years ago, # |
  Vote: I like it -16 Vote: I do not like it

I tried this on ideone:

#include <vector>

using namespace std;

#define v vector<pair<int, int>>

v a(0);

int main()
{
	return 0;
}

It doesn't compile with C++ 5.1 but does with C++14. Interesting, please someone explain this, too :D Is it some sort of smart compiler optimization?

→ Reply
»
7 years ago, # |
  Vote: I like it 0 Vote: I do not like it

This is off the top of my head, so the details may not be 100% precise.

Older versions of C++ standard (standard itself, not compilers) don't allow writing >> without a space when nesting templates because it makes parsing harder (">>" looks just like the operator ">>"). The compiler may compile such code (as is the case with any undefined behavior), but it doesn't have to.

Newer versions of C++ standard (IIRC, already C++11?) allow omitting the space.

→ Reply
»
7 years ago, # |
← Rev. 2   Vote: I like it +1 Vote: I do not like it

Here is how programming language parsing often works. Some programs may do it differently or have some exceptions. C++ is extra nasty in this sense and blurs the border between the steps, but general idea remains.

Compiler/interpreter frontends have 2 stages:

  1. lexing — transform sequence of characters into sequence of tokens
  2. parsing — transform sequence of tokens into abstract syntax tree or directly to bytecode

Instead of working with sequences of characters directly parsers often work with sequences of tokens. Each token is sequence of one or more symbols. One token could be math operator + *=, identifier main i myFunction, constant 3 "abc" True, special keyword if else. This has multiple benefits two simpler languages instead of one complicated, more easily ignore whitespaces, alternative character sequences for the same token (and==&&), hacks like handling python indentation in smart lexer.

Lexers are usually simple and don't know much about the whole program instead they try to greedily create one of the tokens from not knowing if it form a correct program or not.

In this case we are interested in these tokens (real C++ is more complicated than that, but it's good enough for explaining):

  • IDENTIFIER: [a-zA-Z]+
  • R_SHIFT >>
  • LT: <
  • GT: >

Templates in <=C++03 expected something like IDENTIFIER LT argument_list GT. R_SHIFT is clearly not the same as GT so it causes parsing error. C++11 changed the rules to allow >> for ending template, but it makes formal language description even more complicated.

As for why second version of code doesn't fail. In c++ preprocessor works with sequences of tokens not sequences of characters and after being recognized as two separate tokens they aren't combined unless special token concatenation operator ## is used.

It means that character sequence vector<pii> first gets split into tokens identifier(vector) < identifier(pii) >. In the next step preprocessor recognizes that identifier pii is macro and replaces it with matching sequence of tokens identifier(vector) < int >. Parser doesn't know if two > tokens came from two space separated '>' character or macro expansion so they are treated the same.

Here are some examples which are not affected by c++11 changes in template parsing.

    cout << (8 >> 1) << "\n";  // 8 right shift 1 -> 4
    cout << (8 > > 1) << "\n";  // Doesn't compile 8 greater than greater than 1 does't make sense.
#define A >
    cout << (8 A> 1) << "\n";  // Doesn't compile. There is no space, but A> gets recognized as 2 tokens before macro expansion.
#define B(a) a>
    cout << (8 B(>) 1) << "\n"; // Doesn't compile, 2 tokens.
#define C(a) a ## >
    cout << (8 C(>) 1) << "\n";  // 4. ## operator concatenates 2 greater than tokens in to right shift.
#define D J(>,
#define J(a, b) a ## b
    cout << (8 D>) 1) << "\n";  // 4
    cout << (8 D >) 1) << "\n";  // 4 Same as above.
→ Reply