...
You do not need Eclipse or a Java SDK to follow the setup instructions for running the JUnit tests, but in order to actually run them, you will need the Java SDK, and some sort of Java development environment. You can read about the Eclipse setup that the Pentaho team uses here.
Necessary Tweaks to the Server Configuration
In order to run the server standalone, there are a number of "tweaks" we need to perform to align things just right, and exclude those components (like security) that are not necessary during a JUnit test run.
...
This file gets copied into the classes directory when the server's Java files are compiled, so make sure that the file that is in the classes directory has the proper values. That is the copy of the file that the standalone session will read and run with. The path to the classes directory in Eclipse is [pentaho_project]/build/pentaho-classes.
4. Allow Anonymous Access to the Repository
We have removed security from the standalone execution for JUnit tests. So in order to get the repository to ignore any ACLs that are set, we need to modify the pentaho.xml configuration file.
This file can be found in [pentaho-solutions]/system/pentaho.xml.
Open this file, and replace the string com.pentaho.security.acls.voter.PentahoBasicAclVoter with com.pentaho.security.acls.voter.PentahoAllowAllAclVoter.