Hero Run Best Practices

From OpenSFS
Revision as of 08:46, 20 September 2013 by BenEvans (talk | contribs) (Tools)
Jump to: navigation, search

The Hero Run team is tasked with:

  • Establishing a process to determine the peak streaming performance of a clustered filesystem (both read and write)
  • Describing what the test is doing, and detailing why we chose it over other options
  • Providing an optional form to detail the system that was tested (servers, targets, interconnect, clients, etc.)

The Hero Run team is optionally tasked with:

  • Establishing a process to determine the peak random I/O performance of a clustered filesystem (both read and write)
  • Describing what the test is doing, and detailing why we chose it over other options
  • Establishing an algorithm to combine Streaming read+write, Random read+write into a single number.

The Hero Run team is not tasked with:

  • Creating a Top500 list
  • Determining scaling (though the tests can be used to establish that)
  • Comparing vendors offerings, or maintaining a database of results.


Tools

For testing standard POSIX-compliant filesystems, IOR will be used along with an MPI infrastructure. IOR is available Here . Client allocation in the cluster is up to the team performing the test.

Peak Streaming Performance

Base test: mpirun -np {NP} -wd {Working Dir} -machinefile {Machine File} {IOR Pathname}/IOR -v -a POSIX -i5 -g -e -w -r -b 3600g -T 10 -F -C -t 4m

Where:

 {NP} is Total number of threads in the test
 {Working Dir} is the directory where files get created
 {Machine File} is a list of clients involved in the test, this is either created by hand, or by a scheduler
 {IOR Pathname} The full path to IOR, this must be accessible on all clients