Repository navigation
Multi-channel select_mean via deferred Derived put_db - #82
Open
jjguerrette wants to merge 1 commit into
Open
jjguerrette wants to merge 1 commit into
jjguerrette wants to merge 1 commit into
Conversation
Loop filtervars.nvars() for cell means, share jv==0 keep-mask, and put_db all DerivedObsValue/DerivedObsError after the compute phase so IODA Derived preference cannot poison later channel ObsValue reads. Index DerivedObsError by nlocs. Add qc_gauss_thinning_mean_multichannel ctest. Co-authored-by: Cursor <cursoragent@cursor.com>
Author
|
@mo-RowanMcL @BenjaminRuston @twsearle GitHub suggests you contributed to this segment of A little more info may be useful. Currently I am attempting to combine two Gaussian Thinning steps, the first doing super-obbing by - filter: Gaussian Thinning
filter variables:
- name: brightnessTemperature
channels: 1-12
horizontal_mesh: 50.0
min_num_obs_per_bin: 1
select_mean: true
- filter: Gaussian Thinning
horizontal_mesh: 121
use_reduced_horizontal_grid: true
distance_norm: geodesic
action:
name: reduce obs spaceWithout this PR, the Variational application provides the following log output:The hard-coded single-variable implementation yields missing values for With this PR, the output changes to:When I remove the
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Extends Gaussian Thinning
select_meanso a single channelled filter variable (YAMLVariablewith a channel list,filtervars.size() == 1,filtervars.nvars() >= 1) gets a per-channel arithmetic mean inDerivedObsValue, with one shared keep-mask for MetaData.Previously
select_meanonly usedfiltervars.variable(0). WritingDerivedObsValuechannel-by-channel during the gather loop is unsafe: IODA prefers Derived over ObsValue, andObsAccessor(used by Gaussian Thinning) does not passskipDerived=true, so later channel reads see a partial Derived product instead of the original ObsValue.This PR changes only
Gaussian_Thinning.cc(parameters API unchanged):jv = 0 … nvars()-1, gather ObsValue (and optional error fields), runidentifyThinnedObservationsMean, and build local mean/error vectors withoutput_db.jv == 0for thinning flags (one geometry/priority decision per cell).put_dballDerivedObsValue/ optionalDerivedObsError.DerivedObsErrorto missing for allnlocsand write bylocalObsId(hardens sparse apply layouts).Scalar / single-component
select_mean(nvars() == 1) is unchanged in contract. Adds MPI ctestqc_gauss_thinning_mean_multichannel.Out of scope / known orthogonal issue: During development, I found
qc_gauss_thinning_mean_errorEffectiveError layout mismatches under float geodesic distance ties (GeodesicDistanceCalculator+std::min_elementfirst-of-ties). That failure reproduces on my local build without this PR diff and is not addressed here. There are several possible reasons for that failure and possible solutions, in case it arises during review of this PR.Issue(s) addressed
N/A (no linked issue)
Dependencies
Impact
Expected impact on downstream repositories:
select_meanfor radiance (e.g. brightness temperature with a channel list) in observer YAML.select_meanconfigs should behave as before.Checklist