Difference between revisions of "Contract SFS-DEV-001 early draft"

From OpenSFS Wiki
Jump to navigation Jump to search
m
m
Line 1: Line 1:
 
SFS-DEV-001 includes three areas for development. These areas are listed below with highlighted results.
 
SFS-DEV-001 includes three areas for development. These areas are listed below with highlighted results.
 
  
 
== Lustre FSCK: OI Scrub ==
 
== Lustre FSCK: OI Scrub ==
Line 12: Line 11:
 
OI Scrub project provides ground work for a complete Lustre FSCK tool. Crucial to success of this tool is performance. OI Scrub has been demonstrated checking over 100K files/second on a typical MDS hardware node. In addition, OI Scrub provides a on-line mode that has been shown to add negligible system load with limited scan rates:
 
OI Scrub project provides ground work for a complete Lustre FSCK tool. Crucial to success of this tool is performance. OI Scrub has been demonstrated checking over 100K files/second on a typical MDS hardware node. In addition, OI Scrub provides a on-line mode that has been shown to add negligible system load with limited scan rates:
  
[[File:on-line-impact-oiscrub.png|frame|100px|OI Scrub provides on-line scanning for corruption. A successful online tool should not impact system performance. This figure show the fall-off in performance from a system without OI Scrub running in the background (0 on the x-axis) through to a system with OI Scrub running as fast as possible (-1 on the x-axis.) The results from this graph indicate that negligible performance impact will be observed while running OI Scrub with Kfiles per second limited between zero and five.]]
+
[[File:on-line-impact-oiscrub.png|frameless|300px|OI Scrub provides on-line scanning for corruption. A successful online tool should not impact system performance. This figure show the fall-off in performance from a system without OI Scrub running in the background (0 on the x-axis) through to a system with OI Scrub running as fast as possible (-1 on the x-axis.) The results from this graph indicate that negligible performance impact will be observed while running OI Scrub with Kfiles per second limited between zero and five.]]
 +
 
 +
== Single Server Metadata Performance: Parallel Directory Operations ==
 +
=== Key Message ===
 +
* Only a single lock is available for a directory on Lustre.
 +
* Clients wanting access to a file in the same directory must do so in serial.
 +
* This is fixed in
  
 +
=== Recent Results
  
 
[http://wiki.whamcloud.com/display/opensfs/OpenSFS+Lustre+Development Whamcloud site] for more information.
 
[http://wiki.whamcloud.com/display/opensfs/OpenSFS+Lustre+Development Whamcloud site] for more information.

Revision as of 12:31, 5 December 2012

SFS-DEV-001 includes three areas for development. These areas are listed below with highlighted results.

Lustre FSCK: OI Scrub

Key Message

  • Enable file-level backup and restoration of an MDT.
  • Core infrastructure for online consistency checking.

Details of the internal of this work is available at on the Whamcloud community wiki.

Recent results

OI Scrub project provides ground work for a complete Lustre FSCK tool. Crucial to success of this tool is performance. OI Scrub has been demonstrated checking over 100K files/second on a typical MDS hardware node. In addition, OI Scrub provides a on-line mode that has been shown to add negligible system load with limited scan rates:

OI Scrub provides on-line scanning for corruption. A successful online tool should not impact system performance. This figure show the fall-off in performance from a system without OI Scrub running in the background (0 on the x-axis) through to a system with OI Scrub running as fast as possible (-1 on the x-axis.) The results from this graph indicate that negligible performance impact will be observed while running OI Scrub with Kfiles per second limited between zero and five.

Single Server Metadata Performance: Parallel Directory Operations

Key Message

  • Only a single lock is available for a directory on Lustre.
  • Clients wanting access to a file in the same directory must do so in serial.
  • This is fixed in

=== Recent Results

Whamcloud site for more information.