Contract SFS-DEV-004
Overview
The goal of the CLIO Simplification Implementation contract is the implementation in the Lustre source code of the CLIO Simplification Design that resulted from Project 2 of Contract SFS-DEV-003.
For the contract statement of work, see SFS-DEV-004_SOW.pdf
Key People
OpenSFS
- Sarp Oral - OpenSFS Contract Administrator
- Christopher Morrone - OpenSFS Technical Representative
Project Approval Committee (PAC)
- Christopher Morrone - PAC Chair
- Colin Faber
- Patrick Farrell
- Jason Hill
- James Simmons
- Cory Spitz
Intel
- Richard Henwood - Project Manager
- Andreas Dilger - Consulting Architect
- Jinshan Xiong - Lead Engineer
Important Dates
The official start date of work is agreed to be October 13, 2014.
The contract lists milestone target dates in weeks relative to the start date. With the start date agreed, here we can just list actual dates to keep things easy to understand.
| Milestone task | Target Completion | Actual Completion |
|---|---|---|
| Implementation | Jan 26th 2015 | |
| Test and fix | Apr 6th 2015 | |
| Demonstration | May 4th 2015 | |
| Landing | Jun 1st 2015 |
Meeting Minutes
Working in Progress or Completed
cl_lock re-factoring (simplified and cache-less) DONE
LU-3259 cl_lock re-factoring The cl_lock is necessary because it communicates the DLM lock for a specific IO. The current implementation is highly complex. This work will write a simplified cl_lock. The new lock will be cache-less and replace the current implementation.
| Planning | Review | Landed |
|---|---|---|
| 10858 LU-3259 clio: cl_lock simplification | ||
Removal of liblustre DONE
| Planning | Review | Landed |
|---|---|---|
| 10657 LU-2675 build: remove liblustre and libsysio | ||
| 11772 LU-2675 mgc: remove libmgc.c | ||
| 10172 LU-2675 llite: remove liblustre includes | ||
| 10195 LU-2675 lmv: remove liblustre includes | ||
| 10196 LU-2675 lov: remove liblustre includes | ||
| 11423 LU-2675 build: remove Darwin "support" | ||
| 11385 LU-2675 build: remove WinNT "support" |
function calls implementation STARTED
LU-5823 Replace some obsolete obd operations with CLIO ioctl interfaceOBD API operations for read, write, setattr, getattr, etc. became obsolete after MDT, OFD and client reconstructing were completed. This work removes these redundant operations.
| Planning | Review | Landed |
|---|---|---|
| 13514 LU-5823 llite: Remove access of stripe in ll_setattr_raw | 12452 LU-5823 clio: add cl_object_ioctl() interface | |
| 12494 LU-5823 clio: add CL_IOC_FIND_CBDATA cl_object_ioctl ops | ||
| 12535 LU-5823 clio: add CL_IOC_FIEMAP cl_object_ioctl ops | ||
| 12638 LU-5823 clio: add CL_IOC_DATA_VERSION and CL_IOC_GETATTR ops | ||
| 12748 LU-5823 clio: get rid of ll_glimpse_ioctl() | ||
| 12639 LU-5823 clio: get rid of lov_stripe_md reference in setting file stripe info | ||
| 13422 LU-5823 clio: use CIT_SETATTR for FSFILT_IOC_SETFLAGS | ||
Function, Statuso_precreate GONEo_create Used by Echocliento_create_async GONEo_destroy Used by Echocliento_setattr Used by Echocliento_setattr_async GONEo_getattr Used by Echocliento_getattr_async GONEo_brw GONEo_merge_lvb GONEo_adjust_kms GONEo_punch GONEo_sync GONEo_migrate GONEo_copy GONEo_preprw Used by Echoclient and OST server-side codeo_commitrw Used by Echoclient and OST server-side codeo_enqueue GONEo_cancel GONEo_change_cbdata GONEo_find_cbdata GONEo_change_cbdata GONEo_extent_calc GONE
Remove lov_stripe_md (LSM) direct access beyond LOV layer STARTED
LU-5814 encapsulate lov_stripe_md (LSM) to LOV layer The current CLIO implementation has a good interface to file layout operations. Legacy code still exists that does not use this interface. The code that does not use the file layout interface will be reviewed and targeted for removal or re-design to use the file layout interface.
| Planning | Review | Landed |
|---|---|---|
| Attempt to remove lli_has_smd. | LU-5814 obd: remove unused LSM parameters | 12442 LU-5814 lov: remove LL_IOC_RECREATE_{FID,OBJ} |
| Analyze use of lli_maxbytes | 12446 LU-5814 echo: remove userspace LSM handling | |
| Remove lsm member from struct lustre_md | 12445 LU-5814 lov: remove unused {get,set}_info handlers | |
| lsm tail cleanup header patch | 12447 LU-5418 echo: replace lov_stripe_md with lov_oinfo | |
| Remove lov_packmd and lov_unpackmd. blocked on series | 12618 LU-5814 llite: remove ll_objects_destroy() | |
| 12581 LU-5814 lov: flatten struct lov_stripe_md] |
NOTE: struct obd_info:oi_md cannot be removed now because of inter-dependencies between clean-up patches.
Remove non-linux interfaces STARTED
Two parts
Remove some cfs_ prefixed functions. DONE
| Planning | Review | Landed |
|---|---|---|
| NON-INTEL: 6956 LU-1346 libcfs: cleanup libcfs primitive (linux-prim.h) | ||
| NON-INTEL: 11797 LU-3963 libcfs: remove last of cfs list wrappers | ||
| NON-INTEL: 11797 LU-3963 libcfs: remove last of cfs list wrappers | ||
| NON-INTEL: 13070 LU-3963 libcfs: Use kernel's strncasecmp and remove cfs_get_blocked_sigs | ||
NOTE:
- cfs_snprintf() does have uses, not equivilent to snprintf(), will remain.
- cfs_hlist* are needed for Linux kernel compatibility and will remain.
Remove ccc_ layer STARTED
LU-5971 removal of ccc_ layerWith the removal of liblustre, the ccc_ layer is redundant and complex. The remaining useful functions will be merged into vfs vm posix layer and the ccc_ layer will be removed.
| Planning | Review | Landed |
|---|---|---|
| 13318 LU-5971 llite: reorganize {llite,vvp}_internal.h | 12592 LU-5971 llite: merge lclient.h into llite/vvp_internal.h | |
| 13376 LU-5971 llite: move vvp_io functions to vvp_io.c | 13075 LU-5971 llite: rename ccc_device to vvp_device | |
| 13377 LU-5971 llite: rename ccc_req to vvp_req | 13077 LU-5971 llite: rename ccc_object to vvp_object | |
| 13715 LU-5971 llite: rename vvp_thread_info to ll_thread_info | 13086 LU-5971 llite: rename ccc_page to vvp_page | |
| 13716 LU-5971 rename ccc_thread info to vvp_threadinfo | 13088 LU-5971 llite: rename ccc_lock to vvp_lock | |
| 13714 LU-5971 llite: rename struct ccc_grouplock to ll_grouplock | 13351 LU-5971 llite: merge ccc_io and vvp_io | |
| 13717 LU-5971 llite: remove ccc_global_{init,fini} | 13347 LU-5971 llite: remove struct ll_ra_read | |
| 13363 LU-5971 llite: use vui prefix for struct vvp_io members | ||
Cleanup obsolete OBD methods
OBD API operations for read, write, setattr, getattr, etc because obsolete after MDT, OFD and client restructuring were completed. Redundant code remains in CLIO and interfaces that are not referenced by any module will be targeted for removal.
| Planning | Review | Landed |
|---|---|---|
| Remove lov_setattr_async. | 13422 | |
| Remove dead or unused lsm parameters from several functions (o_get_info, mgc_enqueue, osc_brw_prep_request,...). | 13426 | |
| Remove lsm use from ll_getxattr. Blocked on series |
NOTE: o_create, o_destroy are still in use by the echo code. Their LSM parameters have been removed.
Regressions
| Planning | Review | Landed |
|---|---|---|
| Move definition of LDLM_GID_ANY to lustre_dlm.h | ||
| LU-6046 audit comments in cl_object.h |
Change requests
004-001 CLIO ioctl's should be functions
CHANGE REQUEST: 004-001 CLIO ioctl's should be functions.
BACKGROUND: Ioctl calls were included in the CLIO Simplification design to replace some obsolete ODB operations. Alternatively, individual functions can replace the ODB operations instead.
CHANGE: Do not implement ioctl calls. Implement functions.
ACTIONS REQUIRED:
- Ensure none of the current patches are land.
- Update the design document with the new design.
- Update the ticket LU-5823 with new activity.
- Execute work to complete LU-5823.