Hero Run Best Practices

From OpenSFS Wiki
Revision as of 08:09, 20 September 2013 by BenEvans (talk | contribs)
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

The Hero Run team is tasked with:

  • Establishing a process to determine the peak streaming performance of a clustered filesystem (both read and write)
  • Describing what the test is doing, and detailing why we chose it over other options
  • Providing an optional form to detail the system that was tested (servers, targets, interconnect, clients, etc.)

The Hero Run team is optionally tasked with:

  • Establishing a process to determine the peak random I/O performance of a clustered filesystem (both read and write)
  • Describing what the test is doing, and detailing why we chose it over other options
  • Establishing an algorithm to combine Streaming read+write, Random read+write into a single number.

The Hero Run team is not tasked with:

  • Creating a Top500 list
  • Determining scaling (though the tests can be used to establish that)
  • Comparing vendors offerings, or maintaining a database of results.


Tools

For testing standard POSIX-compliant filesystems, IOR will be used along with an MPI infrastructure. IOR is available [[here]http://sourceforge.net/projects/ior-sio/]. Client allocation in the cluster is up to the team performing the test.

Peak Streaming Performance

Base test: mpirun -np {NP} -wd {Working Dir} -machinefile {Machine File} {IOR Pathname}/IOR -v -a POSIX -i5 -g -e -w -r -b 3600g -T 10 -F -C -t 4m

Where:

 {NP} is Total number of threads in the test
 {Working Dir} is the directory where files get created
 {Machine File} is a list of clients involved in the test, this is either created by hand, or by a scheduler
 {IOR Pathname} The full path to IOR, this must be accessible on all clients