Contract SFS-DEV-001

From OpenSFS Wiki
Revision as of 09:36, 30 March 2016 by RichardHenwood (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

NOTE: This contract was completed with the delivery of Lustre Release 2.8 during March 2016

Project Definition

This contract, awarded to Whamcloud (now Intel HPDD) in 2011, defines Lustre development for three projects and eight subprojects. The full program description is available in the Statement of Work.

  • Project 1: Single Server Metadata Performance Improvements
    • Subproject 1.1: SMP Node Affinity
    • Subproject 1.2: Parallel Directory Operations
  • Project 2. Distributed Namespace: Remote Directories and Striped Directories
    • Subproject 2.1: Remote Directories
    • Subproject 2.2: Striped Directories
  • Project 3. Lustre File System Checker
    • Subproject 3.1: Inode Iterator & OI Scrub
    • Subproject 3.2: MDT-OST Consistency
    • Subproject 3.3: MDT-MDT Consistency
    • Subproject 3.4: Performance

OpenSFS assigned Dave Dillow as the Technical Representative for this project. In January 2014, Jason Hill was assigned as the Technical Representative for this project. Jason leads a Project Approval Committee (PAC) that meets regularly with Intel and assists reviewing Intel's completion of project milestones.

Key People


  • Jason Hill (ORNL) - Technical Representative, PAC Chair

Project Approval Committee

  • Cory Spitz, Cray Inc.
  • Patrick Farell, Cray Inc.
  • Nathan Rutman, Seagate
  •  ??


  • Andreas Dilger, Doug Oucharek, Richard Henwood

Project Milestones

Each subproject follows a detailed milestone plan composed of the following phases.

Scope Statement is a brief (2-3 pages) summary of the Contractor's understanding of the problem statement and resulting project scope.
Solution Architecture is a document that outlines requirements, use cases, a solution framework, and appropriate test cases.
High-Level Design (HLD) is a description of the recommended solution that addresses the subproject requirements and describes how the solution will work, including basic protocol structures (as applicable).
Implementation occurs after completion of the HLD and includes development and unit testing of new code. OpenSFS tracks progress during this phase with weekly progress meetings with the Contractor.
Demonstration of the subproject is defined during the architecture phase and occurs after the Contractor completes the appropriate functionality of the subproject with execution of test cases designed to prove the acceptance criteria defined during the Solution Architecture.
Delivery occurs when the Contractor integrates the project development branch for the subproject into the canonical Lustre source tree for inclusion in the next available community Lustre feature release.

Completed milestone documents for each subproject are available in the project status section below.

Project Status

The table below lists the milestone acceptance dates for the subprojects defined in OpenSFS' development contract with Intel HPDD (SFS-DEV-001).
Documentation of each milestone is hyperlinked to its completion date. We will update the table as Intel completes new milestones.

Project Subproject Scope Statement Solution Architecture High-level Design Implementation Demonstration Delivery Landing
MDS Performance PDIR Ops 2011-09-28

MDS PDirOps ScopeStatement wiki version


MDS PDirOps SolutionArchitecture wiki version


MDS PDirOps HighLevelDesign wiki version


MDS PDirOps Implementation wiki version


MDS PDirOps Demonstration wiki version

2012-04-30 2.2
SMP Node Affinity 2012-03-09

MDS SMP Node Affinity ScopeStatement wiki version


MDS SMP Node Affinity SolutionArchitecture wiki version


MDS SMP Node Affinity HighLevelDesign wiki version


MDS SMP Node Affinity Implementation wiki version


MDS SMP Node Affinity Demonstration wiki version


MDS SMP Node Affinity FinalReport wiki version

Distributed Name
Space (DNE)
Remote Directories 2011-10-19

DNE RemoteDirectories ScopeStatement wiki version


DNE RemoteDirectories SolutionArchitecture wiki version


DNE RemoteDirectories HighLevelDesign wiki version

DNE RemoteDirectories Implementation wiki version


DNE RemoteDirectories Demonstration wiki version


DNE RemoteDirectories Final wiki version

Striped Directories 2013-05-13

DNE StripedDirectories ScopeStatement wiki version


DNE StripedDirectories SolutionArchitecture wiki version


DNE StripedDirectories HighLevelDesign wiki version

2014-12-23 2015-08-21 2016-03-25 2.8
Lustre File System
Check (LFSCK)
OI Scrub 2011-10-05 2011-11-03

LFSKC OIScrub Solution Architecture wiki version

n/a 2012-04-06 2012-06-06 2012-12-07 2.3
Fid-in-dirent & linkEA consistency 2012-10-31 2012-11-20

LFSKC FID-in-dirent Solution Architecture wiki version


LFSKC FID-in-dirent High Level Design wiki version

2013-01-16 2013-05-01 2013-05-31 2.4
MDT-OST Consistency 2013-05-01 2013-05-29

LFSKC MDT-OST Consistency Solution Architecture wiki version


LFSKC MDT-OST Consistency High Level Design wiki version

2013-12-20 2014-02-26 2014-08-07 2.6
MDT-MDT Consistency 2014-04-22 2014-05-29

LFSKC MDT-MDT Consistency Solution Architecture wiki version


LFSKC MDT-MDT High Level Design wiki version

2014-09-18 2015-03-24 2015-03-24 2.7
Performance 2014-10-12 2014-12-04

LFSKC Performance Solution Architecture wiki version

n/a 2014-12-23 2015-05-18 2016-03-25 2.8