Difference between revisions of "SSF"

From OpenSFS
Jump to: navigation, search
Line 7: Line 7:
 
None of this has any impact on how Lustre physically organizes the data among it's OSTs (but it may very well have a large performance impact!) The Lustre layout is entirely controlled by the striping parameters StripeSize and StripeCount. StripeCount is the number of OSTs that Lustre will use for RAID0 stripes of a given file. StripeSize is the size of each of those RAID0 stripes - the amount of data Lustre will store on one OST before moving on to the next OST.
 
None of this has any impact on how Lustre physically organizes the data among it's OSTs (but it may very well have a large performance impact!) The Lustre layout is entirely controlled by the striping parameters StripeSize and StripeCount. StripeCount is the number of OSTs that Lustre will use for RAID0 stripes of a given file. StripeSize is the size of each of those RAID0 stripes - the amount of data Lustre will store on one OST before moving on to the next OST.
  
[[File:IOR_ideal.jpg|none|500px|IOR ideal layout]]
+
[[File:IOR_misaligned.jpg|none|500px|IOR misalignment]]
 
[[File:IOR_largeblock.jpg|none|500px|IOR large blocks]]
 
[[File:IOR_largeblock.jpg|none|500px|IOR large blocks]]
 
[[File:IOR_smallblock.jpg|none|500px|IOR small blocks]]
 
[[File:IOR_smallblock.jpg|none|500px|IOR small blocks]]
[[File:IOR_misaligned.jpg|none|500px|IOR misalignment]]
+
[[File:IOR_ideal.jpg|none|500px|IOR ideal layout]]
  
 
Original PDF: [[File:SSFstriping.pdf|Original PDF]]
 
Original PDF: [[File:SSFstriping.pdf|Original PDF]]

Revision as of 13:28, 6 February 2015

Single-shared file performance in Lustre is highly dependent on the interaction between the task data placement within the file, and the file's layout on Lustre OSTs (striping).

IOR and Lustre layout terms

In IOR single-shared file mode, a group of processes all write to a single file. A number of parameters control how those processes layout and transfer the data to the storage system. Each individual process makes a series of TransferSize sequential write calls to a individual IOR block, totaling BlockSize bytes of continuous data. One block from each process are laid out sequentially into a Segment, multiple segments form the total file size. The stride is the distance between two blocks from the same process, and is the same as the SegmentSize.

None of this has any impact on how Lustre physically organizes the data among it's OSTs (but it may very well have a large performance impact!) The Lustre layout is entirely controlled by the striping parameters StripeSize and StripeCount. StripeCount is the number of OSTs that Lustre will use for RAID0 stripes of a given file. StripeSize is the size of each of those RAID0 stripes - the amount of data Lustre will store on one OST before moving on to the next OST.

IOR misalignment
IOR large blocks
IOR small blocks
IOR ideal layout

Original PDF: File:SSFstriping.pdf