[Vp-integration-subgroup] [EXT] [Vp-reproduce-subgroup] Credibility “for EACH USE”Re: White paper revision

[Morse, Katherine L.]

The defense simulation community has been struggling with this problem (like so many others) for decades. We haven't solved it, but we've made some progress. Further progress would require political will and funding that isn't forthcoming. The approach includes metadata about the underlying models such as William identified below as well as interfaces and types and formats of required data. That metadata gets stored in a searchable catalog. Using the catalog and the metadata, a potential user can rapidly do a first order culling of models / simulations that are clearly no applicable. This doesn't rise to the level of solution, but it's better than guessing or sending out mass emails.

BTW, one of the things you could put in the metadata is links to verification, validation, and accreditation (VV&A) reports. Despite the provable impossibility of asserting the validity of the composition, it can get a potential user a couple of steps closer to finding potentially interoperable components.

KLM
---
Katherine L. Morse, PhD
IEEE Fellow
Principal Professional Staff, JHU/APL
11100 Johns Hopkins Road
Laurel, MD  20723-6099
(240)917-9602 (w)
(858)775-8651 (m) 
 

On 5/17/21, 7:11 AM, "William Waites" <wwaites at ieee.org> wrote:

    APL external email warning: Verify sender wwaites at ieee.org before clicking links or attachments 

    > I might have to do some digging to find it, but Dr. Mikel Petty* (UAH) did a proof that you cannot make any assertions about the validity of the composition of two valid models.

    In general, that is very likely true. I’d like to see the proof!

    This gets into the weeds a bit, but in some specific cases we can say some things. Two examples.

    1. If we have a system of differential equations of the form F’(x) = M(x) = A(x) + B(x). Call the two models A and B. We can make propagators φA and φB separately and show that φM(x) = φA(φB(x)) + O(h) where h is a chosen time-step. (A propagator in this context is a function that takes the state of the system at time t and gives you the state at time t+h.) This is a simple result that is quite specific about the properties of the composition and its error. This is a building block that lets you construct integration schemes from the individual propagators that can be cheaper and more stable than the one you get for the big propagator directly. This is used, for example, in calculating trajectories of particles around synchrotrons where there is one model for the behaviour far from the magnets and one for the behaviour near to the magnets.

    2. Models formulated as Petri nets or generalisations like graph rewriting systems can be freely composed to get well-defined behaviour. Whether this is the behaviour that you want is a slightly different question and some care is required, but it is well-defined. This is why we can study systems of chemical reactions for cascades. There is also a connection to differential equations because there is also a completely mechanical way to get the equations for the moments of observables of such systems (which coincide with the standard ODE formulation of many of the SBML models in the limit of large systems). This also applies to some kinds of electronics design and is why we can build circuits out of linear components and they behave as we thing they should. It also works for some of the kinds of models we like to make for infectious diseases.

    Note that both examples require pretty strong conditions on the form of the models to be able to say anything useful about how they compose. It seems likely that there are other kinds of models where we can say useful things. All is not lost, there is some hope!

    I am completely willing to believe, however, that we cannot say much more for general models other than “the interfaces match”, particularly for heterogeneous bunches of models.

    Cheers,
    -w