Should be able to define per-camera reduction classes without parent in global recipes/*.config file

[jester@…]

As promised...:

At present, a per-camera config file, e.g. sdssmosaic/ppImage.config, can only define a symbolic name for a collection of parameters if that same symbolic name is included in the global recipes/ppImage.config. Since the entry in recipes/ppImage.config only has to be present, but can be empty, that entry carries no information. As a result, we need to change two files consistently whenever a reduction class is added or renamed.

So to remove a source of error and confusion (and repeated emails ;-) the config file system should allow per-camera config files to add new symbolic names that do *not* already have a parent in the recipes/*.config equivalent.

[Paul Price]

This is a feature, not a bug! It catches errors, and prevents you from thinking you're using one value when it's actually using another.

[jester@…]

In a later email, I said:

... variable names which I'm free to define myself, i.e. symbolic
names for reduction classes/recipes within a personal config file ... should not need to have a global parent in recipes/*.config

and Paul replied:

Yes, absolutely. Gene made some changes towards this a few months ago, but I don't think he got this feature in.

So I think we agree that this *should* be fixed for the case I described originally. Except that maybe it's up to Gene to fix this, rather than Paul?

[Paul Price]

The problem is that there can be two 'classes' of METADATA-type components of a recipe file:

• A container for things that are related; and
• A symbolic name meant to override definitions higher up.

These two can't be treated in the same way: the first should only survive a merge if it exists in the higher level, which the second should always survive a merge.

Gene has played with this problem, but I don't think he got a solution working.

[eugene]

It is important to keep in mind that the rules we are describing are those enforced by the recipe loading code when reading files in the metadata config (MDC) format. It is not intrinsic to the metadata structure in particular.

The code I put in added optional directives to the METADATA entries in the MDC files: UPDATE and RESET. The UPDATE directive says that when two MDCs are merged, and duplicate metadata containers are found, the new metadata container values replace existing values, keeping the existing ones. The RESET directive says that the contents of the old metadata are completely replaced by the new one. I added this so that the metadata containers which list repeated options (eg, in psphot.config, EXTENDED_SOURCE_MODELS) can define defaults at the system level, but be guaranteed to have only the elements of interest at the camera level.

I don't think this work addresses the problem Sebastian raises. What we need is for the variant recipe names not to be required; this may require an additional METADATA directive to implement that, but I don't think it is really terrible. All we need to do is to tell pmConfigRecipes how to recognize the metadata folders which represent variant abstract names, and not apply the strict loading to those names.