Difference between revisions of "Hero Run Best Practices"

From OpenSFS
Jump to: navigation, search
(Created page with "The Hero Run team is tasked with: * Establishing a process to determine the peak streaming performance of a clustered filesystem (both read and write) * Describing what the te...")
 
Line 13: Line 13:
 
* Determining scaling (though the tests can be used to establish that)
 
* Determining scaling (though the tests can be used to establish that)
 
* Comparing vendors offerings, or maintaining a database of results.
 
* Comparing vendors offerings, or maintaining a database of results.
 +
 +
 +
==== Tools ====
 +
For testing standard POSIX-compliant filesystems, IOR will be used along with an MPI infrastructure.  IOR is available [[here]http://sourceforge.net/projects/ior-sio/].  Client allocation in the cluster is up to the team performing the test.
 +
 +
==== Peak Streaming Performance ====
 +
 +
Base test:  mpirun -np {NP} -wd {Working Dir} -machinefile {Machine File} {IOR Pathname}/IOR -v -a POSIX -i5 -g -e -w -r -b 3600g -T 10 -F -C -t 4m
 +
 +
Where:
 +
  {NP} is Total number of threads in the test
 +
  {Working Dir} is the directory where files get created
 +
  {Machine File} is a list of clients involved in the test, this is either created by hand, or by a scheduler
 +
  {IOR Pathname} The full path to IOR, this must be accessible on all clients

Revision as of 08:09, 20 September 2013

The Hero Run team is tasked with:

  • Establishing a process to determine the peak streaming performance of a clustered filesystem (both read and write)
  • Describing what the test is doing, and detailing why we chose it over other options
  • Providing an optional form to detail the system that was tested (servers, targets, interconnect, clients, etc.)

The Hero Run team is optionally tasked with:

  • Establishing a process to determine the peak random I/O performance of a clustered filesystem (both read and write)
  • Describing what the test is doing, and detailing why we chose it over other options
  • Establishing an algorithm to combine Streaming read+write, Random read+write into a single number.

The Hero Run team is not tasked with:

  • Creating a Top500 list
  • Determining scaling (though the tests can be used to establish that)
  • Comparing vendors offerings, or maintaining a database of results.


Tools

For testing standard POSIX-compliant filesystems, IOR will be used along with an MPI infrastructure. IOR is available [[here]http://sourceforge.net/projects/ior-sio/]. Client allocation in the cluster is up to the team performing the test.

Peak Streaming Performance

Base test: mpirun -np {NP} -wd {Working Dir} -machinefile {Machine File} {IOR Pathname}/IOR -v -a POSIX -i5 -g -e -w -r -b 3600g -T 10 -F -C -t 4m

Where:

 {NP} is Total number of threads in the test
 {Working Dir} is the directory where files get created
 {Machine File} is a list of clients involved in the test, this is either created by hand, or by a scheduler
 {IOR Pathname} The full path to IOR, this must be accessible on all clients