SSF: Difference between revisions

From OpenSFS Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
Single-shared file performance in Lustre is highly dependent on the interaction between the task data placement within the file, and the file's layout on Lustre OSTs (striping).  
Single-shared file performance in Lustre is highly dependent on the interaction between the task data placement within the file, and the file's layout on Lustre OSTs (striping).  
Original PDF: [[File:SSFstriping.pdf|Original PDF]]


[[File:terms.jpg|none|500px|IOR and Lustre layout terms]]
[[File:terms.jpg|none|500px|IOR and Lustre layout terms]]
Line 7: Line 6:


None of this has any impact on how Lustre physically organizes the data among it's OSTs (but it may very well have a large performance impact!) The Lustre layout is entirely controlled by the striping parameters StripeSize and StripeCount. StripeCount is the number of OSTs that Lustre will use for RAID0 stripes of a given file. StripeSize is the size of each of those RAID0 stripes - the amount of data Lustre will store on one OST before moving on to the next OST.
None of this has any impact on how Lustre physically organizes the data among it's OSTs (but it may very well have a large performance impact!) The Lustre layout is entirely controlled by the striping parameters StripeSize and StripeCount. StripeCount is the number of OSTs that Lustre will use for RAID0 stripes of a given file. StripeSize is the size of each of those RAID0 stripes - the amount of data Lustre will store on one OST before moving on to the next OST.
[[File:IOR_ideal.jpg|none|500px|IOR ideal layout]]
[[File:IOR_largeblock.jpg|none|500px|IOR large blocks]]
[[File:IOR_smallblock.jpg|none|500px|IOR small blocks]]
[[File:IOR_misaligned.jpg|none|500px|IOR misalignment]]
Original PDF: [[File:SSFstriping.pdf|Original PDF]]

Revision as of 13:24, 6 February 2015

Single-shared file performance in Lustre is highly dependent on the interaction between the task data placement within the file, and the file's layout on Lustre OSTs (striping).

IOR and Lustre layout terms

In IOR single-shared file mode, a group of processes all write to a single file. A number of parameters control how those processes layout and transfer the data to the storage system. Each individual process makes a series of TransferSize sequential write calls to a individual IOR block, totaling BlockSize bytes of continuous data. One block from each process are laid out sequentially into a Segment, multiple segments form the total file size. The stride is the distance between two blocks from the same process, and is the same as the SegmentSize.

None of this has any impact on how Lustre physically organizes the data among it's OSTs (but it may very well have a large performance impact!) The Lustre layout is entirely controlled by the striping parameters StripeSize and StripeCount. StripeCount is the number of OSTs that Lustre will use for RAID0 stripes of a given file. StripeSize is the size of each of those RAID0 stripes - the amount of data Lustre will store on one OST before moving on to the next OST.

IOR ideal layout
IOR large blocks
IOR small blocks
IOR misalignment

Original PDF: File:SSFstriping.pdf