Test FMI-CS simulation with CVODE

[casella]

@adrpo, OpenModelica/OpenModelica#13582 shows that we probably have serious problems with non-trivial implementations of FMI-CS.

In principle, if we use stiff CVODE as internal solver and a co-simulation time step equal to the Interval annotation (or to the default 500 steps), all the simulations that run with the regular runtime and the stiff DASSL solver should also run successfully. If they don't, it means there's some problem in the internal implementation of the ODE solver in the CS FMU, which should be checked.

I would suggest that we add yet another FMI task where we generate FMI-CS with stiff CVODE and then use OMSimulator to simulate them. Could run once every week or something.

[bilderbuchi]

all the simulations that run with the regular runtime and the stiff DASSL solver should also run successfully

I guess you could extend that with "and give the same results up to a tolerance", to catch differences in solution behaviour caused by bugs in the FMI-CS implementation? Or is that too high a bar to clear (at first)?

[casella]

That was implicit. The CI infrastructure always compares the simulation results if they are provided. Thanks for making that clear.

However, I guess there could be problems verifying the results of models with events, since FMI-CS won't report the additional time points where events take place, I guess.

[bilderbuchi]

since FMI-CS won't report the additional time points where events take place, I guess.

(Disclaimer: I'm still not familiar enough with FMI to answer this confidently myself)
It's unclear to me if you mean that this is a FMI limitation or an OM/tool limitation?
It looks to me to be the latter, as the bouncing ball example at the end of https://fmi-standard.org/docs/3.0.2/#_event_mode seems to very much suggest that additional time points at events are in fact reported.
Or is that an FMI3-only thing?

[adrpo]

Or is that an FMI3-only thing?

Yes, that is only FMI3, in FMI2 one reports the events at the end of the doStep.

[casella]

https://fmi-standard.org/faq/

[casella]

We'll get there, eventually. We are already implementing some FMI 3.0.x features, but we also still need to make sure that FMI 2.0.4 export works well.

[AnHeuermann]

Waiting for the backlog of tests to clear on ryzen-5950x-2 to see if https://test.openmodelica.org/jenkins/blue/organizations/jenkins/LibraryTesting%2FLibraryTest/detail/LibraryTest/9985/pipeline passes.

@adrpo we need some smarter sceduling at some point tu utilize the available machines better.

[casella]

I guess it's cheaper to buy another computer. I mean it, it will cost like a couple of weeks of developer's time.

In general I'm not in favour of brute force, but given how cheap computing power is these days, I wouldn't bother too much.

Yes, we should also consider saving energy and CO2 emissions, but one single 200 W PC running 8000 hrs/year on the Swedish grid, which is mostly powered by hydro, nuclear, and PV, will definitely add negligible emissions to the atmosphere 😃

[AnHeuermann]

The test is now mostly working. Still have some strange output in the overview, e.g. in https://libraries.openmodelica.org/branches/v1.24-cs-fmu-cvode/Modelica_4.1.0/Modelica_4.1.0.html:

FMI tool: b"warning: Failed to load settings: [json.exception.parse_error.101] parse error at line 4, column 1: syntax error while parsing value - unexpected '}'; expected end of input\nOMSimulator v3.0.0.post98-g04448fd-linux-debug\ninfo: 1 warnings\ninfo: 0 errors"

Tested Library: Modelica_4.1.0 4.1.0 warning: Failed to load settings: [json.exception.parse_error.101] parse error at line 4, column 1: syntax error while parsing value - unexpected '}'; expected end of input OMSimulator v3.0.0.post98-g04448fd-linux-debug info: 1 warnings info: 0 errors

But the test itself is working:

[AnHeuermann]

I guess it's cheaper to buy another computer. I mean it, it will cost like a couple of weeks of developer's time.

I think we have plenty of unused compute power. We only need to use it smarter.
We hard code on which machine a test runs, I'm assuming because of some cashing. It would be better to have more machines available with the same tag and Jenkins just chooses one that is free. Maybe using more containerization. But I guess there is a reason why we didn't do that in the first place.

[casella]

@AnHeuermann is the status of the testing machinery good enough that it makes sense to start looking at the results, or is there still something important to be fine-tuned?

[AnHeuermann]

I think it's good enough. I can't reproduce the warnings/errors locally, but the general CS simulation with CVODE is working.

[casella]

OK, then we should probably start looking into it. In principle, the success ratio should be similar to what we obtain by using the C runtime with the cvode solver. For Modelica 4.1.0, it is much higher