The scanner overflow is due to the (rather dated) preprocessor.
You have a large amount of code ifdef'd out.
You have to keep the sections that are 'commented' out to under 4k and you have to put a tokenizable entry between the sections.
One way to see this is to look in the .i file (preprocessor output).

Think of it like this:

The tokenizer replaces every character that is to be removed (comments and #ifdef’d out lines) with a space to easily keep track of what line it is on.  When the tokenizer tries to process these pre-processed, large strings of spaces, it overflows the internal buffer and you get an error.

You must either comment out less stuff or break the removed area with a tokenizable line…ie not a comment.

Note that the version 5 tokenizer has a much smaller buffer than the 6,7, 8 and 10.