You need to increase the memory allocated to the compiler. In order to do this:
- Right click on the project node in the Projects window and select Properties
- In the project properties dialog, click on Build -> Compiling
- In the Additional Compiler Options text box, enter the Xms and Xms switches as needed (for example -J-Xms128m -J-Xmx512m)
- Click OK in the dialog
- Make sure external Javac compiler will be executed by entering the following line to private.properties of the project (Important Files/Per-user Project Properties):
That should resolve the issue.
Source: NetBeans User FAQ