Benchmarking Working Group
Benchmarking Working Group Wiki Pages
Information about the BWG is found at the at opensfs.org website
Benchmarking Working Group Tasks
File System I/O hero run best practices
Lead: Ben Evans
Members: Mark Nelson, Ilene Carpenter, Rick Roloff, Nathan Rutman, Liam Forbes
I/O Workload Characterization
Lead: Pietro Cicotti
Members: Ilene Carpenter, Rick Mohr, Mike Booth, Ben Evans
Application I/O kernel extraction
Lead: Ilene Carpenter
Members: Jeff Layton, Pietro Ciccotti, Bobbie Lind
Lead: Liam Forbes
Members: Alan Wild, Andrew Uselton, Ben Evans, Cheng Shao, Jeff Garlough, Jeff Layton, Mark Nelson, Nic Henke
Metadata Performance Evaluation
Lead: Sorin Fabish
Members: Branislav Radovanovic, Rick Roloff, Cheng Shao, Wang Yibin, Keith Mannthey, Bobbie Lind, Greg Farnum
The task of the BWG Metadata Performance Evaluation Effort (MPEE) group is to:
- Build/select tools that will allow evaluation of File System Metadata performance and scalability
- The tools will help detect pockets of Metadata low performance in cases when users complain of extreme slowness of MD operations
- Benchmark tools will support: POSIX, MPI, and Transactional operations (for CEPH and DAOS)
- Address the very high end HPC as well as small and medium installations benchmark needs
- Tools applicable to Lustre and: CEPH, GPFS…
Current MPEE proposed list of benchmarks:
- mdtest – widely used in HPC
- fstest - used by pvfs/OrangeFS community
- Postmark and MPI version - old NetApp benchmark
- Netmist and MPI version – used by SPECsfs
- Synthetic tools – used by LANL, ORNL
- MDS-Survey - Intel’s metadata workload simulator.
- Any known open source metadata tools used in HPC
- Add new Lustre statistics specific to MD operations.
- mdtest: test file MD operations on MDS: open, create, lookups, readdir; used in academia and as a comparison tool of FS MD.
- fstest: small I/O’s and small files as well as lookups, targeting both MDS and OSS operations and MD HA for multiple MDS’s.
- Postmark: old NetApp benchmark – I built an MPI version; it is used to measure MD operations and file size scalability and files per directory scalability.
- Netmist: used to model any workload from statistics including all MD operations and file operations. Can model Workload objects for I/O performance mixes and combination of I/O and MD. Suitable for initial evaluation of storage as well as for performance troubleshooting.
MPEE Proposed Roadmap
- Collect benchmark tools candidates from OpenSFS
- Evaluate all the tools and the workloads that can benchmarked
- Recommend a small set of MD benchmark tools to cover the majority of MD workloads
- Collect stats from users of MD benchmarks
- Build scripts to allow ease of use of the recommended tools
- Write documentation for troubleshooting MD performance problems using the toolset
- Create a special website for MD tools
MPEE Asks from OpenSFS
- Share any open source synthetic benchmarks code
- Share a list of MD benchmark tools they currently use to allow select the most suitable and used candidates
- Share MD operations tested to allow build Netmist workload objects
- Share the MD workloads that create pain points to Lustre FS
- Share cases of poor MD performance workloads and applications
Return to http://wiki.opensfs.org/Benchmarking_Working_Group page.