| | 1 | = Nebulous Issues = |
| | 2 | |
| | 3 | == Validation == |
| | 4 | |
| | 5 | We currently do not have a validation system in place. It would be useful to have a neb-fsck program that could do the following things: |
| | 6 | 1) Scan the instance table, and confirm that the files exist on disk. |
| | 7 | 1) Confirm that the md5sum matches for all instances of a given storage object. |
| | 8 | 1) Scan the nebulous directory trees and identify files that do not have a matching entry in the instance table. These files could be reinserted, as the full disk path describes the ins_id and the ext_id for a given file. It could be reassociated with the storage_object (if it exists), or simply deleted. |
| | 9 | |
| | 10 | In addition, all nebulous tasks that create a file or copy a file should validate that the files agree before the function completes successfully. Not doing this can introduce errors if we silently replicate a broken copy of the file. |
| | 11 | |
| | 12 | == Database structure == |
| | 13 | |
| | 14 | The user defined number of copies of a storage_object is recorded in the storage_object_xattr table. This requires joins, and given that most storage objects have an entry here, moving this information from storage_object_xattr into a column of storage_object makes sense. |
| | 15 | |
| | 16 | We also currently do not record an md5sum value for every storage_object. This is somewhat expensive to compute, but would provide a record of what the file contents are, so the neb-fsck program would have a reference to work from. Otherwise, if the md5sums disagree between two files, how do we decide which file is correct? |
| | 17 | |
| | 18 | == Location awareness == |
| | 19 | |
| | 20 | We do not currently have each nebulous request tagged with the host it came from. If we did so, then we could have nebulous attempt to supply a copy from that host if one exists. This would reduce the network interaction and increase throughput. This can also be extended into a preference hierarchy, such that if this host doesn't have a copy, look in this cabinet, and then across all the cabinets at this location, and finally across the entire nebulous set. |
