The scanner overflow is due to the preprocessor.


This fatal error will occur when you have a large amount of code (more than 4kb) surrounded by #ifdef/#endif preprocessor calls. 


The tokenizer replaces every character that is to be removed (comments, #ifdef’d out lines) with a space to easily keep track of what line it is on.  When the tokenizer tries to process these pre-processed, large strings of spaces, it overflows the internal buffer and you get an error.


Solution:

You must reduce the amount of contiguous #ifdef/#endif and comments.  Break up the #ifdef/#endif sections with a tokenizable line of code. 

Note that the version 5 tokenizer has a much smaller buffer than the 6,7, 8 and 10 versions.