| 1 | | There is a desire to organize the datastores for PV3 into sky chunks that are of a similar size of data, organized by dxlayer and load merge machine. The intent is to always feed the PSPS a chunk of batches that add up to a similar size in bytes for each PSPS machine. This page describes Ken's suggested datastore tessellation, |
| | 1 | There is a desire to organize the datastores for PV3 into sky chunks that are of a similar size of data, organized by dxlayer and load merge machine. The intent is to always feed the PSPS a chunk of batches that add up to a similar size in bytes for each PSPS machine so there are no surprises. Over the course of the ingest and merge, the only thing that will significantly vary as we ingest across the sky is |
| | 2 | the cumulative size of the merged database. Each subsequent addition prior to merge, will be about the same size, and will be handled by a similar machine. Our greatest uncertainty at this stage is the scaling with the total size, not the scaling per chunk. This page describes Ken's suggested datastore tessellation, as |
