IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changeset 4943


Ignore:
Timestamp:
Sep 1, 2005, 6:20:29 PM (21 years ago)
Author:
jhoblitt
Message:

update

Location:
trunk
Files:
4 edited

Legend:

Unmodified
Added
Removed

  • trunk/Nebulous-Server/docs/design.txt

    r2785 r4943  
    1 # Copryight (C) 2004  Joshua Hoblitt
     1# Copyright (C) 2004-2005  Joshua Hoblitt
    22#
    3 # $Id: design.txt,v 1.1.1.1 2004-12-22 02:16:23 jhoblitt Exp $
    4 
    5 Terminology
    6 --
    7     - Storage Object
    8 There is one Storage Object per resource (a file) that is being managed by the
    9 IPP Pixel Data Server.  A Storage Object maps an External Identifier (eg.
    10 file name) to a Storage ID that is internal to the IPP Pixel Data Server.  In
    11 addition, creation time and comments may be stored in a Storage Object.
    12 Storage Objects /do not/ contain the storage location of a resource as we need
    13 to allow for N copies of a resource to be stored.  The actual storage location
    14 of a resource is held in a Storage Object Instance, which maps to a Storage ID
    15 in a many-to-one relationship.
    16 
    17     - [Storage Object] Instance
     3# $Id: design.txt,v 1.2 2005-09-02 04:20:29 jhoblitt Exp $
     4
     5Overview
     6--
     7
     8Nebulous is a user-space distributed object (file) storage system.  The system
     9is distributed in the sense of it's data storage model where objects are stored
     10across a number of storage volumes and may have redundant copies.  While IPC
     11is done in the traditional client/server model with a single[1] centralized
     12server containing all storage object meta-data.  This system was designed,
     13unlike most distributed filesystems, specifically so that Nebulous clients
     14could also host one or more storage volumes. Although, there is no requirement
     15that clients also provide storage space.  If you so choose, clients and the
     16storage volumes can reside on independent hardware.
     17
     18There are 4 principle components required for a working Nebulous system;
     19server, client, data transport, and data storage.  The Nebulous system only
     20provides the server and the client software.  Data transport[2] and Data
     21storage are left to 3rd party software packages.  As this function is typically
     22included with most operation system it is unlikely that those specific
     23technologies can be improved upon.
     24
     25[1] Multiple Nebulous servers should be possible via database replication.
     26[2] A future version of Nebulous may include it's own data transport layer
     27based on the WEB DAV protocol.
     28
     29Client <-> Server IPC
     30--
     31
     32The Nebulous Server and clients communicate via the SOAP protocol.  Since SOAP
     33is platform independent this allows native Nebulous client to be written in a
     34variety of languages.  With in the Nebulous Server itself the IPC mechanism is
     35abstracted to allow ether multi-protocol support or complete replacement of
     36SOAP by another protocol.
     37
     38XXX SOAP namespace
     39
     40Data model
     41--
     42
     43In Nebulous terminology a file is refereed to as a a resource.  Each resource
     44is represented inside the Nebulous Server as a "Storage Object".  There is one
     45Storage Object per resource (a file) that is being managed by the Nebulous
     46Server.  A Storage Object maps an External Identifier (eg. filename) to a
     47Storage ID.  In addition, creation time and comments may be stored in a
     48Storage Object.  Storage Objects /do not/ contain the storage location of a
     49resource as we need to allow for N copies of a resource to be stored.  The
     50actual storage location of a resource is held in a Storage Object Instance,
     51which maps to a Storage ID in a many-to-one relationship.
     52
    1853A Storage Object Instance, or just Instance, represents a single copy of a
    19 resource being managed by the IPP Pixel Data Server. An Instance contains an
    20 Instance ID, the location of the resource as a URI, a checksum of the
    21 resource, a Storage ID, a read lock semaphore (unsigned integer), a write lock
    22 mutex (enum( 'write', 'wait', 'free' ) - is this better then using an int and
    23 constants?), and a UTC timestamp of the last modification date & time.
    24 
    25     - Lock Record
    26 A Lock Record is used to double check the indicated lock state of an Instance
    27 and can be used to identify system failures.  It must contain the Instance ID
    28 of of the locked instance and the type of lock, eg, enum( 'read', 'write').
     54resource being managed by the Nebulous Server. An Instance contains an
     55Instance ID, the parent Storage ID, the location of the resource as a URI, and
     56other assorted meta-data.
     57
     58Storage IDs are a signed 64-bit integer.  Negative values are considered
     59reserved for future use.  Allocation should be incremental and, as we are
     60unlikely to exceed 2^63 Storage Objects in the life time of the project, never
     61reused. Instance IDs follow the same scheme as Storage IDs but exists in a
     62unique namespace.
     63
     64Database record relationships
     65--
     66* All Storage Objects must have at least once associated Instance.
     67
     68* All Instances must be associated with a Storage Object.
     69
     70* All Instances must be associated with a valid URI (file exists).
     71
     72* All Storage Objects with locks set must have the correct number of
     73corresponding Lock Records.
     74
     75* All Lock Records must be associated with a Storage Object.
     76
     77Data transport
     78--
     79XXX
     80
     81Data Storage
     82--
     83XXX
     84
     85Storage volume state tracking.
     86
     87Nebulous itself does not attempt to determine the state of storage volumes.
     88Instead, it relies on some 3rd party to determine both the state
     89(available/not) and capacity (total size + % utilized).  This information is
     90then feed into Nebulous via either the administrative API or CLI management
     91tools.
     92
     93XXX Storage Location determination
     94
    2995
    3096Locking semantics
    3197--
    32 A lock is required to either read from or write to a Storage Object Instance
    33 being managed by the IPP Pixel Data Server.  Stated another way, locking
    34 granularity is managed on the level of Storage Object Instances instead of on
    35 the more abstract Storage Objects.  Although, from the client perspective,
    36 operations are carried out on a Storage Object.   The locking state of a
    37 Storage Object Instance is maintained by in the Storage Object Instance and a
    38 Lock Record.
    39 
    40 Two types of locks are used on an Instance, a semaphore is used to track read
    41 locks and an exclusive mutex is for write locking.
     98Nebulous uses 'advisory locking' where clients are excepted to check for the
     99existence of locks but this policy is not enforced.  Locks are set on a
     100Storage Object and apply to all Instances of that object.
     101
     102Two types of locks can be set; 'read' locks and 'write' locks.  Read locks are
     103implemented as a semaphore while write locks use an exclusive mutex.
    42104
    43105    - Read lock
    44 In order to read from a Storage Object Instance the Read Lock semaphore must
    45 first be incremented by one and a Lock Record created.  After the read is
    46 completed the Lock Record must be deleted and the Read Lock semaphore
    47 decremented by one.  The incrementation of the Read Lock and Lock Record
    48 creation or the decrementation of the Read Lock and the Lock Record deletion
    49 must happen atomically.  A Read lock can not be aquired unless the Write mutex
    50 is in the state of 'free'.  If a Read lock can not be aquired, a client must
    51 not "spin lock" on the Instanace ID as it may be in the process of being
    52 removed.  Instead, the "spin lock" should be re-resolving the external
    53 identifier to a Storage ID, and the Storage ID to Storage Instances.
     106In order to obtain a Read Lock on a Storage Object the Write Lock mutex must
     107be in the state of 'free'.  When a Read Lock is acquire the Read semaphore is
     108incremented by one.  When a Read Lock is released the Read semaphore is
     109decremented by one.  This must happen atomically with the
     110creation/destruction of a Lock Record.
    54111
    55112    - Write lock
    56 In order to obtain a Write lock on a Storage Object Instance the Write Lock
    57 mutex must be in the state of 'free' and the Read lock semaphore must have a
    58 value of zero.  A client should either "spin lock" or fail if all the Lock
    59 mutex is set or the Read Semaphore has a non-zero value.  A Lock Record must
    60 be created atomically with the setting of the Write mutex.  The setting of the
    61 Write mutex and Lock Record insertion or the clearing of the Write mutex and
    62 the Lock Record deletion must happen atomically.  If the Write lock has a
    63 state of 'free' and the Read lock has a non-zero value, the Write lock may be
    64 set to the 'wait' state to prevent new Read locks from being aquired.
    65 
    66 Storage Object Operations
    67 --
    68     - Add
    69 The resource must already be on disk (having been named through some sort of
    70 mechanism to generate new filenames).  A new Storage Object must be created with a new/unqiue  Storage ID and has
    71 an associated with an External Identifier.  At least one Instance must be
    72 created atomicaly at the same time.  Each new Instance must be initially created with a Write lock.
    73 The resource may then be written to it's storage location.  When the writing
    74 of the resource is completed the Write lock must be cleared as early as is
    75 feasible.  The association of an External Identifier must wait until Instance
    76 creation is completed
    77 
    78     - Delete
    79 In order to remove a Storage Object, a Write lock must be obtained on
    80 /all/ associated Instances.  Then the resources are removed.  As each resource
    81 is deleted the corresponding Instance must be deleted.  After all Instances are
    82 removed the Storage Object must then be deleted.
    83 
    84     - Update
    85 Storage Object Instances are not truly written to.  Instead, a new set of
    86 Storage Object Instances and a new Storage Object are created in parallel.
    87 
    88 Write locks must be aquired on all the orignal Intances of a Storage Object.
    89 Then an Add operation is performed to create a new Storage Object and
    90 associated Instances.  Then the External Identifier must be atomically moved
    91 from the original Storage Object to the new Storage Object.  After the
    92 External Identifier migration, the original Storage Object and it's Instances
    93 should be removed with a special case of the Delete operation (one that will
    94 operate on already held locks).  If the old Storage Object is merely
    95 abandoned, ie. it still exists but doesn't have an External Identifier, it and
    96 the associated Instances will be removed during the next Consistency Sweep.
    97 If a Delete is not peformed, the Write locks on the orignal Instances must be
    98 released. [use a dead Storage Object table or flag, similar to mark and sweep
    99 garbage collection?]
    100 
    101     - Move
    102 This is the same operation as an Update except the new Instances are identical
    103 to the old.
    104 
    105     - Open
    106 Sets a Read lock on an Instances of a Storage Object and determines it's
    107 storage location.  A Read lock must only be placed on one Instance of a
    108 Storage Object.  All Instances of a Storage Object should be iterated through
    109 in a "spin lock" until a Read lock is aquired.
    110 
    111     - Close
    112 Releases a Read lock aquired with open.
    113 
    114 The lock must be released as soon as is feasible.
    115 
    116 Lock sweeping
    117 --
     113In order to obtain a Write lock on a Storage Object the Write Lock mutex must
     114be in the state of 'free' and the Read lock semaphore must have a value of
     115zero.  When a Write Lock is acquired the Write mutex is set to 'write'.  When a
     116Write Lock is released the Write mutex is set to to 'free'.  This must happen
     117atomically with the creation/destruction of a Lock Record.  As an extension, a
     118sate of 'wait' may be implemented such that clients waiting on Read Locks to
     119be released may prevent any additional Read Locks from being acquired.
     120
     121XXX Lock Records are currently unimplemented
     122    - Lock Records A Lock Record is used to double check the indicated lock
     123      state of a Storage Object and can be used to identify logical system
     124errors.  A Lock Record contains the Storage ID of the object and the type of
     125lock set.  eg, enum( 'read', 'write').
     126
     127    - Client best practices
     128When attempting to acquire a lock a client should either "spin lock" with a
     129reasonable pause between lock attempts or fail completely.  Aggressive "spin
     130locking" is considered antisocial and may eventually require the Nebulous
     131Server to identify and ignore such clients.
     132
     133
     134House keeping
     135--
     136    - Lock sweeping
    118137In the event that a Storage Object operation fails to complete successfully
    119138stale locks will have to be identified and removed from the IPP Pixel Data
    … …  
    129148Record entries themselves must be removed from the lock table.
    130149
    131 Consistency sweeping
    132 --
     150    - Consistency sweeping
    133151Periodically the IPP Pixel Data Server meta-data and Storage Object will need
    134152to be checked for sanity.  This would be similar to running fsck on a modern
    135153filesystem.  Consistency sweeping should include Lock sweeping and should be
    136 considered a superset.
    137 
    138 All Storage Objects must have at least once associated Instance.
    139 
    140 All Storage Object Instances must be associated with a Storage Object.
    141 
    142 All Storage Object Instances with a lock set must have the correct number of
    143 corresponding Lock Records.
    144 
    145 All Lock Records must be plausible, ie. the Instance pointed at must be in a
    146 corresponding state.
    147 
    148 Storage ID Allocation
    149 --
    150 Storage IDs are a signed 64-bit integer.  Negative values are considered
    151 reserved for future use.  Allocation should be incremental and, as we are
    152 unlikely to exceed 2^63 Storage Objects in the life time of the project, never
    153 reused.
    154 
    155 Allocated via [SQL auto-increment/stored procedure/app server?]
    156 
    157 Instance ID Allocation
    158 --
    159 Indentical to a Storage ID but exists in a unique namespace.
    160 
    161 Storage Location determination
    162 --
    163 ???
     154considered a super-set.
     155
     156
     157Server Operations
     158--
     159    - setup
     160    - create_object
     161    - rename_object
     162        XXX unimplemented
     163    - replicate object
     164    - lock_object
     165    - unlock_object
     166    - find_instances
     167    - delete_instance
     168    - split_instance
     169        XXX unimplemented
     170    - stat_object
     171    - stat_instance
     172        XXX unimplemented
     173
     174Client Operations
     175--
     176    - create
     177Creates and opens new Storage Object
     178
     179    - replicate
     180Adds an Instance to a Storage Object
     181
     182    - cull
     183Removes an Instance from a Storage Object
     184
     185This operation can not remove the last Instance of a Storage Object.
     186
     187    - lock
     188Trys to acquire a lock on a storage object
     189
     190This operation times out after a default interval of 10s.
     191
     192    - unlock
     193Trys to release a lock on a storage object
     194   
     195    - find_instances
     196Find all instances of a storage object
     197
     198Only Instances on active storage volumes are found.
     199
     200    - find
     201Find any instance of a storage object
     202
     203Only Instances on active storage volumes are found.
     204
     205    - open
     206Open a storage object for read or write
     207
     208Opening a storage object as 'write' will remove all but one Instance to
     209prevent Instances from becoming inconsistent.
     210
     211    - delete
     212Delete a storage object and all of it's instances
     213
     214Removes a storage object and all of it's instances by sequentially deleting each
     215Instance.  When a Storage Object has no more associated instances it is
     216automatically removed by the Nebulous server.
     217
     218    - copy
     219Copy a storage object
     220
     221The new storage object will be created with only one Instance.  Most properties
     222of the source Storage Object will be preserved.
     223
     224    - move
     225Rename a storage object
     226
     227Currently this operation will remove all but one instance of the storage
     228object and may change it's storage location.
     229
     230    - delete_instance
     231   
     232Remove a storage object instance
     233
     234Removing the last Instance of a Storage Object will destroy the Storage
     235Object.
     236
     237    - stat
     238View the properties of a storage object
     239
     240
     241    - import
     242        XXX unimplemented
     243Import a file into Nebulous
     244
     245Creates a new Storage Object with the specified External Identifier.  The
     246source file will then be "copied" into the Nebulous system leaving the original
     247file unmodified.
     248
     249    - fission
     250        XXX unimplemented
     251Create a new Storage Object from an existing one by either splitting off an
     252Instance and re-parenting it or by cloning an existing Instance and associating
     253it with the new Storage Object.

  • trunk/Nebulous-Server/docs/setup.txt

    r4921 r4943  
    2525    mkdir /po03/nebulous
    2626    mkdir /po04/nebulous
    27     chown nobody:nebulous /po0?/nebulous
    28     chmod 0770 /po0?/nebulous
     27    perl -e 'for (1..24) { $i = sprintf "po%02d", $_ ; mkdir "/$i/nebulous" }'
     28    chown nobody:nebulous /po??/nebulous
     29    chmod 0770 /po??/nebulous
     30    ls -lad /po??/nebulous
    2931
    3032=head2 Build and install the Nebulous Perl modules (see: README)
    … …  
    174176    neb-addvol --name po03 --uri file:/po03/nebulous
    175177    neb-addvol --name po04 --uri file:/po04/nebulous
     178
     179    perl -e 'for (1..24) { $i = sprintf "po%02d", $_ ; system "neb-addvol --name $i --uri file:/$i/nebulous" }'
    176180    .
    177181    .

  • trunk/Nebulous/docs/design.txt

    r2785 r4943  
    1 # Copryight (C) 2004  Joshua Hoblitt
     1# Copyright (C) 2004-2005  Joshua Hoblitt
    22#
    3 # $Id: design.txt,v 1.1.1.1 2004-12-22 02:16:23 jhoblitt Exp $
    4 
    5 Terminology
    6 --
    7     - Storage Object
    8 There is one Storage Object per resource (a file) that is being managed by the
    9 IPP Pixel Data Server.  A Storage Object maps an External Identifier (eg.
    10 file name) to a Storage ID that is internal to the IPP Pixel Data Server.  In
    11 addition, creation time and comments may be stored in a Storage Object.
    12 Storage Objects /do not/ contain the storage location of a resource as we need
    13 to allow for N copies of a resource to be stored.  The actual storage location
    14 of a resource is held in a Storage Object Instance, which maps to a Storage ID
    15 in a many-to-one relationship.
    16 
    17     - [Storage Object] Instance
     3# $Id: design.txt,v 1.2 2005-09-02 04:20:29 jhoblitt Exp $
     4
     5Overview
     6--
     7
     8Nebulous is a user-space distributed object (file) storage system.  The system
     9is distributed in the sense of it's data storage model where objects are stored
     10across a number of storage volumes and may have redundant copies.  While IPC
     11is done in the traditional client/server model with a single[1] centralized
     12server containing all storage object meta-data.  This system was designed,
     13unlike most distributed filesystems, specifically so that Nebulous clients
     14could also host one or more storage volumes. Although, there is no requirement
     15that clients also provide storage space.  If you so choose, clients and the
     16storage volumes can reside on independent hardware.
     17
     18There are 4 principle components required for a working Nebulous system;
     19server, client, data transport, and data storage.  The Nebulous system only
     20provides the server and the client software.  Data transport[2] and Data
     21storage are left to 3rd party software packages.  As this function is typically
     22included with most operation system it is unlikely that those specific
     23technologies can be improved upon.
     24
     25[1] Multiple Nebulous servers should be possible via database replication.
     26[2] A future version of Nebulous may include it's own data transport layer
     27based on the WEB DAV protocol.
     28
     29Client <-> Server IPC
     30--
     31
     32The Nebulous Server and clients communicate via the SOAP protocol.  Since SOAP
     33is platform independent this allows native Nebulous client to be written in a
     34variety of languages.  With in the Nebulous Server itself the IPC mechanism is
     35abstracted to allow ether multi-protocol support or complete replacement of
     36SOAP by another protocol.
     37
     38XXX SOAP namespace
     39
     40Data model
     41--
     42
     43In Nebulous terminology a file is refereed to as a a resource.  Each resource
     44is represented inside the Nebulous Server as a "Storage Object".  There is one
     45Storage Object per resource (a file) that is being managed by the Nebulous
     46Server.  A Storage Object maps an External Identifier (eg. filename) to a
     47Storage ID.  In addition, creation time and comments may be stored in a
     48Storage Object.  Storage Objects /do not/ contain the storage location of a
     49resource as we need to allow for N copies of a resource to be stored.  The
     50actual storage location of a resource is held in a Storage Object Instance,
     51which maps to a Storage ID in a many-to-one relationship.
     52
    1853A Storage Object Instance, or just Instance, represents a single copy of a
    19 resource being managed by the IPP Pixel Data Server. An Instance contains an
    20 Instance ID, the location of the resource as a URI, a checksum of the
    21 resource, a Storage ID, a read lock semaphore (unsigned integer), a write lock
    22 mutex (enum( 'write', 'wait', 'free' ) - is this better then using an int and
    23 constants?), and a UTC timestamp of the last modification date & time.
    24 
    25     - Lock Record
    26 A Lock Record is used to double check the indicated lock state of an Instance
    27 and can be used to identify system failures.  It must contain the Instance ID
    28 of of the locked instance and the type of lock, eg, enum( 'read', 'write').
     54resource being managed by the Nebulous Server. An Instance contains an
     55Instance ID, the parent Storage ID, the location of the resource as a URI, and
     56other assorted meta-data.
     57
     58Storage IDs are a signed 64-bit integer.  Negative values are considered
     59reserved for future use.  Allocation should be incremental and, as we are
     60unlikely to exceed 2^63 Storage Objects in the life time of the project, never
     61reused. Instance IDs follow the same scheme as Storage IDs but exists in a
     62unique namespace.
     63
     64Database record relationships
     65--
     66* All Storage Objects must have at least once associated Instance.
     67
     68* All Instances must be associated with a Storage Object.
     69
     70* All Instances must be associated with a valid URI (file exists).
     71
     72* All Storage Objects with locks set must have the correct number of
     73corresponding Lock Records.
     74
     75* All Lock Records must be associated with a Storage Object.
     76
     77Data transport
     78--
     79XXX
     80
     81Data Storage
     82--
     83XXX
     84
     85Storage volume state tracking.
     86
     87Nebulous itself does not attempt to determine the state of storage volumes.
     88Instead, it relies on some 3rd party to determine both the state
     89(available/not) and capacity (total size + % utilized).  This information is
     90then feed into Nebulous via either the administrative API or CLI management
     91tools.
     92
     93XXX Storage Location determination
     94
    2995
    3096Locking semantics
    3197--
    32 A lock is required to either read from or write to a Storage Object Instance
    33 being managed by the IPP Pixel Data Server.  Stated another way, locking
    34 granularity is managed on the level of Storage Object Instances instead of on
    35 the more abstract Storage Objects.  Although, from the client perspective,
    36 operations are carried out on a Storage Object.   The locking state of a
    37 Storage Object Instance is maintained by in the Storage Object Instance and a
    38 Lock Record.
    39 
    40 Two types of locks are used on an Instance, a semaphore is used to track read
    41 locks and an exclusive mutex is for write locking.
     98Nebulous uses 'advisory locking' where clients are excepted to check for the
     99existence of locks but this policy is not enforced.  Locks are set on a
     100Storage Object and apply to all Instances of that object.
     101
     102Two types of locks can be set; 'read' locks and 'write' locks.  Read locks are
     103implemented as a semaphore while write locks use an exclusive mutex.
    42104
    43105    - Read lock
    44 In order to read from a Storage Object Instance the Read Lock semaphore must
    45 first be incremented by one and a Lock Record created.  After the read is
    46 completed the Lock Record must be deleted and the Read Lock semaphore
    47 decremented by one.  The incrementation of the Read Lock and Lock Record
    48 creation or the decrementation of the Read Lock and the Lock Record deletion
    49 must happen atomically.  A Read lock can not be aquired unless the Write mutex
    50 is in the state of 'free'.  If a Read lock can not be aquired, a client must
    51 not "spin lock" on the Instanace ID as it may be in the process of being
    52 removed.  Instead, the "spin lock" should be re-resolving the external
    53 identifier to a Storage ID, and the Storage ID to Storage Instances.
     106In order to obtain a Read Lock on a Storage Object the Write Lock mutex must
     107be in the state of 'free'.  When a Read Lock is acquire the Read semaphore is
     108incremented by one.  When a Read Lock is released the Read semaphore is
     109decremented by one.  This must happen atomically with the
     110creation/destruction of a Lock Record.
    54111
    55112    - Write lock
    56 In order to obtain a Write lock on a Storage Object Instance the Write Lock
    57 mutex must be in the state of 'free' and the Read lock semaphore must have a
    58 value of zero.  A client should either "spin lock" or fail if all the Lock
    59 mutex is set or the Read Semaphore has a non-zero value.  A Lock Record must
    60 be created atomically with the setting of the Write mutex.  The setting of the
    61 Write mutex and Lock Record insertion or the clearing of the Write mutex and
    62 the Lock Record deletion must happen atomically.  If the Write lock has a
    63 state of 'free' and the Read lock has a non-zero value, the Write lock may be
    64 set to the 'wait' state to prevent new Read locks from being aquired.
    65 
    66 Storage Object Operations
    67 --
    68     - Add
    69 The resource must already be on disk (having been named through some sort of
    70 mechanism to generate new filenames).  A new Storage Object must be created with a new/unqiue  Storage ID and has
    71 an associated with an External Identifier.  At least one Instance must be
    72 created atomicaly at the same time.  Each new Instance must be initially created with a Write lock.
    73 The resource may then be written to it's storage location.  When the writing
    74 of the resource is completed the Write lock must be cleared as early as is
    75 feasible.  The association of an External Identifier must wait until Instance
    76 creation is completed
    77 
    78     - Delete
    79 In order to remove a Storage Object, a Write lock must be obtained on
    80 /all/ associated Instances.  Then the resources are removed.  As each resource
    81 is deleted the corresponding Instance must be deleted.  After all Instances are
    82 removed the Storage Object must then be deleted.
    83 
    84     - Update
    85 Storage Object Instances are not truly written to.  Instead, a new set of
    86 Storage Object Instances and a new Storage Object are created in parallel.
    87 
    88 Write locks must be aquired on all the orignal Intances of a Storage Object.
    89 Then an Add operation is performed to create a new Storage Object and
    90 associated Instances.  Then the External Identifier must be atomically moved
    91 from the original Storage Object to the new Storage Object.  After the
    92 External Identifier migration, the original Storage Object and it's Instances
    93 should be removed with a special case of the Delete operation (one that will
    94 operate on already held locks).  If the old Storage Object is merely
    95 abandoned, ie. it still exists but doesn't have an External Identifier, it and
    96 the associated Instances will be removed during the next Consistency Sweep.
    97 If a Delete is not peformed, the Write locks on the orignal Instances must be
    98 released. [use a dead Storage Object table or flag, similar to mark and sweep
    99 garbage collection?]
    100 
    101     - Move
    102 This is the same operation as an Update except the new Instances are identical
    103 to the old.
    104 
    105     - Open
    106 Sets a Read lock on an Instances of a Storage Object and determines it's
    107 storage location.  A Read lock must only be placed on one Instance of a
    108 Storage Object.  All Instances of a Storage Object should be iterated through
    109 in a "spin lock" until a Read lock is aquired.
    110 
    111     - Close
    112 Releases a Read lock aquired with open.
    113 
    114 The lock must be released as soon as is feasible.
    115 
    116 Lock sweeping
    117 --
     113In order to obtain a Write lock on a Storage Object the Write Lock mutex must
     114be in the state of 'free' and the Read lock semaphore must have a value of
     115zero.  When a Write Lock is acquired the Write mutex is set to 'write'.  When a
     116Write Lock is released the Write mutex is set to to 'free'.  This must happen
     117atomically with the creation/destruction of a Lock Record.  As an extension, a
     118sate of 'wait' may be implemented such that clients waiting on Read Locks to
     119be released may prevent any additional Read Locks from being acquired.
     120
     121XXX Lock Records are currently unimplemented
     122    - Lock Records A Lock Record is used to double check the indicated lock
     123      state of a Storage Object and can be used to identify logical system
     124errors.  A Lock Record contains the Storage ID of the object and the type of
     125lock set.  eg, enum( 'read', 'write').
     126
     127    - Client best practices
     128When attempting to acquire a lock a client should either "spin lock" with a
     129reasonable pause between lock attempts or fail completely.  Aggressive "spin
     130locking" is considered antisocial and may eventually require the Nebulous
     131Server to identify and ignore such clients.
     132
     133
     134House keeping
     135--
     136    - Lock sweeping
    118137In the event that a Storage Object operation fails to complete successfully
    119138stale locks will have to be identified and removed from the IPP Pixel Data
    … …  
    129148Record entries themselves must be removed from the lock table.
    130149
    131 Consistency sweeping
    132 --
     150    - Consistency sweeping
    133151Periodically the IPP Pixel Data Server meta-data and Storage Object will need
    134152to be checked for sanity.  This would be similar to running fsck on a modern
    135153filesystem.  Consistency sweeping should include Lock sweeping and should be
    136 considered a superset.
    137 
    138 All Storage Objects must have at least once associated Instance.
    139 
    140 All Storage Object Instances must be associated with a Storage Object.
    141 
    142 All Storage Object Instances with a lock set must have the correct number of
    143 corresponding Lock Records.
    144 
    145 All Lock Records must be plausible, ie. the Instance pointed at must be in a
    146 corresponding state.
    147 
    148 Storage ID Allocation
    149 --
    150 Storage IDs are a signed 64-bit integer.  Negative values are considered
    151 reserved for future use.  Allocation should be incremental and, as we are
    152 unlikely to exceed 2^63 Storage Objects in the life time of the project, never
    153 reused.
    154 
    155 Allocated via [SQL auto-increment/stored procedure/app server?]
    156 
    157 Instance ID Allocation
    158 --
    159 Indentical to a Storage ID but exists in a unique namespace.
    160 
    161 Storage Location determination
    162 --
    163 ???
     154considered a super-set.
     155
     156
     157Server Operations
     158--
     159    - setup
     160    - create_object
     161    - rename_object
     162        XXX unimplemented
     163    - replicate object
     164    - lock_object
     165    - unlock_object
     166    - find_instances
     167    - delete_instance
     168    - split_instance
     169        XXX unimplemented
     170    - stat_object
     171    - stat_instance
     172        XXX unimplemented
     173
     174Client Operations
     175--
     176    - create
     177Creates and opens new Storage Object
     178
     179    - replicate
     180Adds an Instance to a Storage Object
     181
     182    - cull
     183Removes an Instance from a Storage Object
     184
     185This operation can not remove the last Instance of a Storage Object.
     186
     187    - lock
     188Trys to acquire a lock on a storage object
     189
     190This operation times out after a default interval of 10s.
     191
     192    - unlock
     193Trys to release a lock on a storage object
     194   
     195    - find_instances
     196Find all instances of a storage object
     197
     198Only Instances on active storage volumes are found.
     199
     200    - find
     201Find any instance of a storage object
     202
     203Only Instances on active storage volumes are found.
     204
     205    - open
     206Open a storage object for read or write
     207
     208Opening a storage object as 'write' will remove all but one Instance to
     209prevent Instances from becoming inconsistent.
     210
     211    - delete
     212Delete a storage object and all of it's instances
     213
     214Removes a storage object and all of it's instances by sequentially deleting each
     215Instance.  When a Storage Object has no more associated instances it is
     216automatically removed by the Nebulous server.
     217
     218    - copy
     219Copy a storage object
     220
     221The new storage object will be created with only one Instance.  Most properties
     222of the source Storage Object will be preserved.
     223
     224    - move
     225Rename a storage object
     226
     227Currently this operation will remove all but one instance of the storage
     228object and may change it's storage location.
     229
     230    - delete_instance
     231   
     232Remove a storage object instance
     233
     234Removing the last Instance of a Storage Object will destroy the Storage
     235Object.
     236
     237    - stat
     238View the properties of a storage object
     239
     240
     241    - import
     242        XXX unimplemented
     243Import a file into Nebulous
     244
     245Creates a new Storage Object with the specified External Identifier.  The
     246source file will then be "copied" into the Nebulous system leaving the original
     247file unmodified.
     248
     249    - fission
     250        XXX unimplemented
     251Create a new Storage Object from an existing one by either splitting off an
     252Instance and re-parenting it or by cloning an existing Instance and associating
     253it with the new Storage Object.

  • trunk/Nebulous/docs/setup.txt

    r4921 r4943  
    2525    mkdir /po03/nebulous
    2626    mkdir /po04/nebulous
    27     chown nobody:nebulous /po0?/nebulous
    28     chmod 0770 /po0?/nebulous
     27    perl -e 'for (1..24) { $i = sprintf "po%02d", $_ ; mkdir "/$i/nebulous" }'
     28    chown nobody:nebulous /po??/nebulous
     29    chmod 0770 /po??/nebulous
     30    ls -lad /po??/nebulous
    2931
    3032=head2 Build and install the Nebulous Perl modules (see: README)
    … …  
    174176    neb-addvol --name po03 --uri file:/po03/nebulous
    175177    neb-addvol --name po04 --uri file:/po04/nebulous
     178
     179    perl -e 'for (1..24) { $i = sprintf "po%02d", $_ ; system "neb-addvol --name $i --uri file:/$i/nebulous" }'
    176180    .
    177181    .
Note: See TracChangeset for help on using the changeset viewer.