This fatal error will occur when you have a large amount of code (more than 4kb) surrounded by #ifdef/#endif preprocessor calls.
The tokenizer replaces every character that is to be removed (comments, #ifdef’d out lines) with a space to easily keep track of what line it is on. When the tokenizer tries to process these pre-processed, large strings of spaces, it overflows the internal buffer and you get an error.
Solution:You must reduce the amount of contiguous #ifdef/#endif and comments. Break up the #ifdef/#endif sections with a tokenizable line of code.
Note that the version 5 tokenizer has a much smaller buffer than the 6,7, 8 and 10 versions.