Basically you need to pass an option argument of the form
-Xmx384M
to the java interpreter (where 384M means a maximum of 384 MBytes of RAM).
So:
(1) If you run the thing using "ant run", you can edit build-user.xml to add a jvmarg tag to the java task, something like
should be all you need.
(2) If you are running it inside Eclipse, you would use Run -> Run... and set up appropriate JVM arguments to a run task that do what you want.
(3) If you want to "just do it yourself" at the shell prompt, you can do something like
java -Xmx384M -jar ~/reprap/Reprap/jar/Reprap.jar
or use a similar longer command line that runs from the .class files directly.
Are you actually finding bugs that only show up in complex objects? I'm intrigued -- please tell us what they are! So far, I find I can happily tweak and test things with simple objects and not worry about RAM usage, and then package the result via host-package-release, unpack the resulting .zip file and run using the reprap.sh script to do anything "serious" with complex objects. But I can see why that might get annoying if you like to stay inside Eclipse and are working with bugs that you can't duplicate with simpler objects!
Jonathan
