Contract SFS-DEV-001 early draft

From OpenSFS Wiki
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

SFS-DEV-001 includes three areas for development. These areas are listed below with highlighted results.

Project Documents

This contract defines development in three broad areas described in the Statement of Work.

  • Project 1: Single Server Metadata Performance Improvements
    • Subproject 1.1: SMP Node Affinity
    • Subproject 1.2: Parallel Directory Operations
  • Project 2. Distributed Namespace: Remote Directories and Striped Directories
    • Subproject 2.1: Remote Directories
    • Subproject 2.2: Striped Directories
  • Project 3. Lustre File System Checker
    • Subproject 3.1: Inode Iterator & OI Scrub
    • Subproject 3.2: MDT-OST Consistency
    • Subproject 3.3: MDT-MDT Consistency
    • Subproject 3.4: Performance

Each subproject is developed using a 6 point milestone plan. Completed documents are available from the project page.

Lustre FSCK: OI Scrub

Key Message

  • Enable file-level backup and restoration of an MDT.
  • Core infrastructure for online consistency checking.

Details of the internal of this work is available at on the Whamcloud community wiki.

Recent results

OI Scrub project provides ground work for a complete Lustre FSCK tool. Crucial to success of this tool is performance. OI Scrub has been demonstrated checking over 100K files/second on a typical MDS hardware node. In addition, OI Scrub provides a on-line mode that has been shown to add negligible system load with limited scan rates:

OI Scrub provides on-line scanning for corruption. A successful online tool should not impact system performance. This figure show the fall-off in performance from a system without OI Scrub running in the background (0 on the x-axis) through to a system with OI Scrub running as fast as possible (-1 on the x-axis.) The results from this graph indicate that negligible performance impact will be observed while running OI Scrub with Kfiles per second limited between zero and five.

Single Server Metadata Performance: Parallel Directory Operations

Key Message

  • Only a single lock is available for a directory on Lustre.
  • Clients wanting access to a file in the same directory must do so in serial.
  • This is fixed in

=== Recent Results

Whamcloud site for more information.

Retrieved from "http://wiki.opensfs.org/index.php?title=Contract_SFS-DEV-001_early_draft&oldid=385"