Exnode specification: Difference between revisions

From ReddNet
Jump to navigation Jump to search
No edit summary
No edit summary
Line 8: Line 8:
*L-Store writes extra erasure allocations.
*L-Store writes extra erasure allocations.
**These allocations are also stored in the exnode file
**These allocations are also stored in the exnode file
**These allocations are only useful to a data-rebuilder not the data user
**These allocations are only useful to a data-management area and not the data-client
*L-Store can write a LoRS compatible exnode file that excludes the erasure allocations.
*L-Store can write a LoRS compatible exnode file that excludes the erasure allocations.
**LoRS compatible
**LoRS compatible
**LoRS does not have access to L-Store's capability to regain a file.
**LoRS does not have access to L-Store's capability to regain a file.
*The data user does not need (and probably should not be bothered with) to handle error recovery.
*The data-client does not know how to handle error recovery.
*data recovery, data warming, etc all are done by services/agents apart from the data consumer.
*data recovery, data warming, etc all are done in the data-management area, not the data-client.
*So what user interface and development API are needed by the data users?
*user interface and development API are needed by the data client
**The base exnode (LoRS)
*command line tools
*command line tools
**file movement
**file movement

Revision as of 20:29, 16 January 2008

The exnode specification is implied by current usage in L-Store and LoRS.

Example LoRS exnode xml file

Example L-Store xml file

exnodes and erasure codings.

  • L-Store writes extra erasure allocations.
    • These allocations are also stored in the exnode file
    • These allocations are only useful to a data-management area and not the data-client
  • L-Store can write a LoRS compatible exnode file that excludes the erasure allocations.
    • LoRS compatible
    • LoRS does not have access to L-Store's capability to regain a file.
  • The data-client does not know how to handle error recovery.
  • data recovery, data warming, etc all are done in the data-management area, not the data-client.
  • user interface and development API are needed by the data client
    • The base exnode (LoRS)
  • command line tools
    • file movement
    • file system (some use the term namespace management) tools
      • list files, create directories, etc
  • C/C++ API
    • just need my simple offsets and lengths
    • would like to read a vector of data blocks (non-blocked reads)
    • don't want to deal with erasure stuff at this point.
      • is there a nice way to recover from errors?


Schema

The exnode xml is fairly unstructured and the functionality of a schema is minimal.

Simplified xml file

  • (removed xmlns to view structure more clearly)

Simple schema

  • The schema shows that only the element mapping has much structure.
  • The element metadata's use of attributes is completely open.
  • The only re-used type is the IBP_capability (read,write,manage)