IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 1 and Version 2 of Nebulous_Throughput_Issues


Ignore:
Timestamp:
Feb 17, 2011, 9:09:57 AM (15 years ago)
Author:
Serge CHASTEL
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Nebulous_Throughput_Issues

    v1 v2  
    3131
    3232 * choice of hash key.  The hash key could be defined in a number of ways.  One extreme would be to use the entire filename (neb key) as the hash key.  At the other extreme, we could use just the base directory-like component (eg all chars up to the first '/' char).  Since all entries with the same key will land on the same server, the latter strategy is probably not advantageous.  On the other hand, it probably is advantageous for a set of similar files to land on the same machine.  If we want to break the bottle-neck for the camera stage, we definitely want the files from different chips to be well distributed.  My recommendation would be to strip the extension from the key (all chars after the last '.') and use the rest of the string as the key.  That would keep all of the output results from a single stage (eg, one chip) on one database, but distribute different runs even within the same exposure across multiple machines.
     33
     34=== More on Performance ===
     35* [http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/neb-touch-gory-details neb-touch gory detail]