You need to increase the memory allocated to the compiler. In order to do this:

1. Right click on the project node in the Projects window and select Properties
2. In the project properties dialog, click on Build -> Compiling
3. In the Additional Compiler Options text box, enter the Xms and Xms switches as needed (for example -J-Xms128m -J-Xmx512m)
4. Click OK in the dialog
5. Make sure external Javac compiler will be executed by entering the following line to private.properties of the project (Important Files/Per-user Project Properties):

       build.compiler=extJavac
   

That should resolve the issue.

Source: NetBeans User FAQ

 Related Tips

• I have written a module. Can I sell it?
• 2D texture font - NeHe Tutorial JOGL Port
• 3D Human Animation Applet
• 3D lens flare with occlusion testing - NeHe Tutorial JOGL Port
• 3D shapes - NeHe Tutorial JOGL Port
• A model with a light which is moved by a modeling transformation
• A Morph object to animate a shape between two key shapes
• A red sphere using the Sphere utility class