[Vp-integration-subgroup] Standards for simulation interoperability

[Jacob Barhak]

Thanks William,

Your additions are valuable and I am trying to organize those so they will
not get lost in the communications. When you refer you agree with James
Glazier, I assume you mean to continue this thread you started a month ago:
https://lists.simtk.org/pipermail/vp-reproduce-subgroup/2021-March/000030.html

It seems you are interested in composition of models and organizing it more
that you are interested in the actual standardization process, so it is ok
you are writing to the integration mailing list.

It seems you are avoiding the discussion you saw started about standards in
response to the white paper:
https://lists.simtk.org/pipermail/vp-reproduce-subgroup/2021-April/000069.html

This discussion is less about the composition process itself you are
interested in, it is more about formal standards and still relevant, and is
far from the theoretical concepts you describe.

The current situation is that the research community wants freedom and this
leads to many paths that are unique and hard to generalize. Katherine was
revealing to us that the same situation existed in the Defense community
many years ago and they solved it through proper standards. Let me extend
on her ideas and claim that whatever work we do, if we do not manage to
create clear ways to integrate biological models that can be standardized,
then a lot of the effort we do will be lost - and you already know my
opinions on academic inefficiency.

Your suggestions are inline with what Katherine claims, you are trying to
describe the areas that need fixing so that in the future we will be able
to standardize. However, in my mind, we do need to accept that a
standardization effort will follow the more theoretical work you suggest.
In the past, groups like the SBML/COMBINE rejected the idea they needed to
standardize formally. Those communities valued freedom over formal
processes that require administrative work. There is a place of academic
freedom to create new things, yet at some point if the products of research
are to be used, those have to be formalized into some sort of usable
specification. If we are to integrate pieces of knowledge together, we
better have a method to do so - and this is where Katherine is trying to
help us.

At the same time, we need to work on conceptual aspects like the ones you
suggest.

For example, I want us to agree on terminology for composition methods
used. A lot of the language you use needs examples since people come from
different domains and sometimes call similar things differently  - we need
to start simple for people to comprehend what we mean.

How about starting with a simple composition example and see what people
would call this composition in their domain?

Do you have a very simple example in mind we can start with? If so please
start a thread and call it Composition Example 1 and I suggest we discuss
elements of it.

Do you think such a discussion would be useful, or would you like to divert
it in a different direction?

Yet I do think we should focus our discussions on integration in this
mailing list for now. It is a good observation you made.

               Jacob









On Sun, Apr 25, 2021 at 3:04 AM William Waites <wwaites at ieee.org> wrote:

> Starting a new thread because this is not especially about the paper.
>
> I was a little confused by the recent exchange on standards because it
> seems to mix up a few different things.
>
> It seems to me that for composing models we need to pay attention to the
> interface between them and not to the internal workings. James Glazier is
> right that we often run into situations where the boundary isn’t clear. By
> forcing a clear boundary, we introduce error in the composed model. This is
> probably unavoidable and a big part of what we need to be able to do is
> quantify this error.
>
> Re-reading the exchange, I don’t think that James Sluka meant that a
> particular programming language should be considered a standard — please
> correct me if I am wrong. I think the point was about using annotation to
> document the interface of programs. This is a good idea and is independent
> of the implementation details.
>
> It would be a bad idea for several reasons to mandate a specific
> programming language. First of all, most programming languages do not have
> well-defined semantics (denotational or operational) so the foundation is
> shaky. Second of all, we probably do not want to gratuitously constrain
> research into modelling techniques which might be more convenient to
> implement in one language than another. I think we should stay well clear
> of prescribing particular programming languages apart perhaps from
> insisting that we should use only ones with freely available compilers or
> interpreters.
>
> It is a good idea to try to specify what behaviour a model must have in
> order to be able to compose it. We probably want to say something about
> classes of model whose compositional properties are somewhat better
> understood. We know how to take two ODE models and compose them together in
> a way that we can understand what kind of error to expect. We know how to
> compose chemical reaction networks and rule-based models and derive ODE or
> stochastic simulations from them. We might know what to expect if we throw
> equilibrium processes such as FBA models into the mix. We do not especially
> know what to expect if we compose an arbitrary program with any of these,
> though Jonathan Karr showed that it can, miraculously, work in some
> circumstances at least.
>
> The distinction between syntax and semantics is important too. Syntax
> means things like XML or JSON or packed byte arrays in memory that we need
> to make sure the input and output of models (or simulations, or, better,
> propagators as I described in an earlier message) is well-formed. These are
> low-level things that need to be standard for practical reasons. The higher
> level is semantics, or meaning. When a simulation produces an integer, the
> low-level standard says what the syntax of this is, how it is represented
> in bits, but we also need to know what it means. Maybe it is the copy
> number of a certain protein or the number of recovered individuals.
>
> I think there are several things to think about:
>
>   - Distinction between what happens within a model and what happens
> between models
>   - Low-level plumbing, syntax at the interface of simulations
>   - Semantics of models and annotation
>   - Semantics of simulation and composition
>   - Properties of composite models that can be derived from the
> constituent models
>   - Properties of composite models that can be derived from the
> composition method
>   - Just because a model produces well-formed output with documented
> semantics doesn’t mean that it is correct.
>   - Just because individual models are correct doesn’t mean that a
> composite model made by combining them is.
>
> Best wishes,
> -w
>
> _______________________________________________
> Vp-integration-subgroup mailing list
> Vp-integration-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-integration-subgroup/attachments/20210425/e19e011f/attachment.html>