Changes between Version 15 and Version 16 of Raw_Images_Files_in_Nebulous

Timestamp:

Feb 29, 2012, 3:44:17 PM (14 years ago)

Author:

Serge CHASTEL

Comment:

--

Raw_Images_Files_in_Nebulous

@@ -93 +93 @@
 
 === Are we good or bad? ===
-We have something like 420,000 exposures (from pzDownloadImfile for gpc1, with fault = 0). Ideally, at mhpcc, we should have only one ota on one node, therefore the expected ota count on each node should be precisely 420,000 (the actual mean being 274682, the maximum being 376716, the minimum 201702). On average, we have a difference of (420,000 - 274,682) roughly 150,000 otas / node. If they were uniformly spread on the nodes, we would have somtehing like 2500 exposures/node (2500 is roughly 150,000 otas spread on 59 nodes). The last histogram shows a peak around 2500 which is precisely what we expect.
+We have something like 420,000 exposures (from pzDownloadImfile for gpc1, with fault = 0). Ideally, at mhpcc, we should have only one ota on one node, therefore the expected ota count on each node should be precisely 420,000 (the actual mean being 274682, the maximum 376716, the minimum 201702). On average, we have a difference of (420,000 - 274,682) roughly 150,000 otas / node. If they were uniformly spread on the nodes, we would have something like 2500 exposures/node (2500 is roughly 150,000 otas spread on 59 nodes). The last histogram shows a peak around 2500 which is precisely what we expect.
 
-
-
-
+To see if shuffling works, we have to repeat the same experiment in the future and see if that number of 2500 gets smaller. It would mean that the files are moving towards where they should be that is, all the files related to one ota on the same node.