Community Development Working Group: Difference between revisions

From OpenSFS Wiki
Jump to navigation Jump to search
No edit summary
Line 26: Line 26:
* [[Media:OpenSFS-Intel_Tree_Report_-_Q4_2012_FINAL.pdf]]‎
* [[Media:OpenSFS-Intel_Tree_Report_-_Q4_2012_FINAL.pdf]]‎
* [[Media:OpenSFS-Intel_Tree_Report_-_Q1_2013_FINAL.pdf]]‎
* [[Media:OpenSFS-Intel_Tree_Report_-_Q1_2013_FINAL.pdf]]‎
==Lustre Community Development in Progress ==
Features that are currently being developed for future Lustre releases both at Intel and by other organizations in the Lustre community. These will be eligible for inclusion in future Lustre releases as per our processes. Features are listed in alphabetical order, not in priority or landing order.  Projects not currently under development, or ideas for features, should be listed on the Project Ideas page.
Please note that the targeted release information is provided by the developing organization for informational purposes only and is in no way a roadmap commitment until it appears on the roadmap .
{| class="wikitable sortable"
|-
! Feature !! Feature Summary !! Developer !! Tracker !! Targeted Release !! Landed Release
|-
| Client module build process change to adapt to vanilla kernel || Example || EMC || . || 2.5 || .
|-
| 3.6/FC18 Server Patches || . || ORNL/NRL/Suse/EMC || LU-1812 || 2.5 || .
|-
| Patchless Server || Remove Lustre kernel patches to allow Lustre servers to be more easily ported to new kernels, and to be built against vendor kernels without changing the vendor kernel RPMs || Intel || LU-20 || 2.5 || .
|-
| End-to-End Data Integrity with T10 || Add a path to pass T10 Protection Information from Lustre clients through to server disks || Xyratex|| LU-2488/LU-2584 || 2.5 || .
|-
| Extended Attribute Batching || Allow clients to fetch multiple xattrs in a single RPC for efficiency || Intel || LU-549 || 2.5 || .
|-
| HSM - Release || Release action discard all data from a Lustre file, mainly by removing all associated OST objects, and flags it as released (in LMA). This is used when a file has been successfully copied in an external storage, a HSM || CEA || LU-1333 || 2.5 || .
|-
| LFSCK Phase 2 - MDT-OST consistency || Traverse all inodes in an MDT and verify their corresponding objects on the OSTs exist. This work is part of the OpenSFS contract awarded to Whamcloud. LFSCK Architecture and Design Docs available soon || Intel || LU-957 || 2.5 || .
|-
| LFSCK Phase 3 - DNE consistency || Enhance LFSCK to work with DNE filesystems, including remote directory entries, and OST orphan handling for multiple MDTs || Intel || LU-2307 || 2.5 || .
|-
| Portability changes || To facilitate Mac clients || NRL || . || . || .
|-
| client-server shared modules cleanup || Clean up Lustre client to facilitate pushing it into upstream Linux kernel. Need to split some server code dependencies from client to avoidmaking extensive server code changes || EMC || LU-1214/LU-1330 || . || .
|-
| libcfs cleanup || clean up libcfs to use Linux kernel APIs || EMC || LU-1346 || 2.5 || .
|-
| HSM -Nanosecond Time Stamps || Modify the Lustre network protocol to allow for passing and storing nanosecond time stamp from clients to servers and back again || Intel || LU-1158 || 2.5 || .
|-
| HSM - Coordinator || Today Coordinator runs in the MDT layer, code need to be reorganized to achieve a better layering MDT/MDD so some changes can be made in a single transaction || CEA || LU-2061 || 2.5 || .
|-
| HSM - Copy tool || The copy tool is the HSM component which moves data between Lustre and an external storage system. This feature will be for POSIX copy tool which can use any POSIX-compliant filesystem as a backend target system. The copy tool is used for all the sanity tests made for HSM features (sanity-hem.sh) || CEA || LU-2062 || 2.5 || .
|-
| HSM - Open Lock Issue || Example || Intel || LU-2063 || 2.5 || .
|-
| HSM - Changes to Test Framework || Modify test framework to allow for adequate testing of HSM || CEA/Intel || LU-2064 || 2.5 || .
|-
| HSM - Layout Lock Protocol Cleanup || Example || Intel || LU-2065 || 2.5 || .
|-
| Example || Example || Example || Example || Example || Example
|}

Revision as of 15:32, 9 April 2013

Basic information about the CDWG including details about the email list and conference calls are located at opensfs.org.

The CDWG has crafted the Community_Tree_Development_Contract and is working on updates. The CDWG has also crafted the maintenance and feature release strategy for the Community Lustre Roadmap.

Also, there is an effort started to improve testing and quality assurance, see Improving_the_test_framework.

Resources

Meeting Minutes

Tree Contract Reports

Lustre Community Development in Progress

Features that are currently being developed for future Lustre releases both at Intel and by other organizations in the Lustre community. These will be eligible for inclusion in future Lustre releases as per our processes. Features are listed in alphabetical order, not in priority or landing order. Projects not currently under development, or ideas for features, should be listed on the Project Ideas page. Please note that the targeted release information is provided by the developing organization for informational purposes only and is in no way a roadmap commitment until it appears on the roadmap .

Feature Feature Summary Developer Tracker Targeted Release Landed Release
Client module build process change to adapt to vanilla kernel Example EMC . 2.5 .
3.6/FC18 Server Patches . ORNL/NRL/Suse/EMC LU-1812 2.5 .
Patchless Server Remove Lustre kernel patches to allow Lustre servers to be more easily ported to new kernels, and to be built against vendor kernels without changing the vendor kernel RPMs Intel LU-20 2.5 .
End-to-End Data Integrity with T10 Add a path to pass T10 Protection Information from Lustre clients through to server disks Xyratex LU-2488/LU-2584 2.5 .
Extended Attribute Batching Allow clients to fetch multiple xattrs in a single RPC for efficiency Intel LU-549 2.5 .
HSM - Release Release action discard all data from a Lustre file, mainly by removing all associated OST objects, and flags it as released (in LMA). This is used when a file has been successfully copied in an external storage, a HSM CEA LU-1333 2.5 .
LFSCK Phase 2 - MDT-OST consistency Traverse all inodes in an MDT and verify their corresponding objects on the OSTs exist. This work is part of the OpenSFS contract awarded to Whamcloud. LFSCK Architecture and Design Docs available soon Intel LU-957 2.5 .
LFSCK Phase 3 - DNE consistency Enhance LFSCK to work with DNE filesystems, including remote directory entries, and OST orphan handling for multiple MDTs Intel LU-2307 2.5 .
Portability changes To facilitate Mac clients NRL . . .
client-server shared modules cleanup Clean up Lustre client to facilitate pushing it into upstream Linux kernel. Need to split some server code dependencies from client to avoidmaking extensive server code changes EMC LU-1214/LU-1330 . .
libcfs cleanup clean up libcfs to use Linux kernel APIs EMC LU-1346 2.5 .
HSM -Nanosecond Time Stamps Modify the Lustre network protocol to allow for passing and storing nanosecond time stamp from clients to servers and back again Intel LU-1158 2.5 .
HSM - Coordinator Today Coordinator runs in the MDT layer, code need to be reorganized to achieve a better layering MDT/MDD so some changes can be made in a single transaction CEA LU-2061 2.5 .
HSM - Copy tool The copy tool is the HSM component which moves data between Lustre and an external storage system. This feature will be for POSIX copy tool which can use any POSIX-compliant filesystem as a backend target system. The copy tool is used for all the sanity tests made for HSM features (sanity-hem.sh) CEA LU-2062 2.5 .
HSM - Open Lock Issue Example Intel LU-2063 2.5 .
HSM - Changes to Test Framework Modify test framework to allow for adequate testing of HSM CEA/Intel LU-2064 2.5 .
HSM - Layout Lock Protocol Cleanup Example Intel LU-2065 2.5 .
Example Example Example Example Example Example
Retrieved from "http://wiki.opensfs.org/index.php?title=Community_Development_Working_Group&oldid=341"