Page 8 of 11

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Apr 24, 2017 6:35 am
by Josef Templ
I am now asking for a vote about including the coroutine support as in
issue branch issue-#156.

For the diffs see ... oops "An error occurred when trying to access the repository: Cannot allocate memory - 'git' --version --no-color"

Chairman, feel free to split the vote into any number of sub votes if you think
it would help finding a decision.

Let me summarize the reason and rationale for the proposal.

The short form:
If you don't know what a coroutine is, I can only ask you to trust me.
This is a starting point for a new feature that fits well with the BlackBox design.
It can be refined later if experience shows new requirements or problems.
Without starting it, we will never get experience for refining it.
There are no incompatibilities whatsoever.
This is a pure add-on that is not activated if not used.
Kernel size is increased from 31001 to 32902 bytes, i.e. by 1901 bytes.

The long form:
There seems to be a need by the community for supporting coroutines in
BlackBox as can be seen by the existence of the CPC package named Co_.
I can only confirm the need for coroutine support by my own experience.
It is certainly not used every day but sometimes it can be very helpful, for example
when dealing with long running background tasks or with discrete event simulation.

It is not possible to add coroutine support without Kernel modifications.
Therefore Kernel support for coroutines has been added.
Coroutine support has been added such that it cannot introduce any instabilities
because the new code is only activated when coroutines are really used.
This trades a small amount of code replication for guaranteed stability.
Coroutine support is encapsulated in Kernel because Kernel maintains the core
runtime data structures including the heap, the main stack, the trap handling, the stack overflow detection,
the garbage collection etc. and coroutines are part of that. Any split would break that abstraction
and introduce more complications than simplifications. BlackBox has been designed by ominc
from the beginning as a portable system with support for Windows and MacOS.
Some base modules have been split and implemented in a separate Host module.
Kernel is not among them. I trust the design of ominc in this respect and don't see any
chance and need to 'improve' this.
(personal estimation: with the amount of time spent into looking at alternative approaches,
coroutines could have been ported to 3 platforms including maintenance for 10 years.)

The Kernel's coroutine support makes it possible to implement the Co_ package
without crashing BlackBox occasionally. There have been doubts that this would be possible
but it has been shown that it works by simply replacing the WinApi fiber calls by
the corresponding Kernel calls 1:1.

The Co_ package is designed with specific kinds of applications in mind. It is not
a general purpose coroutine library. Therefore a general purpose module named 'Coroutines'
has been added. In addition to basic general purpose coroutines it supports
two special coroutine patterns named Task and Iterator. These are common patterns that
can appear in many applications. They are special in the sense that they have
additional methods (Transfer wrappers) with checked pre- and post conditions that
restrict their usage to the intended purpose.
Further application specific patterns can be implemented on top of Coroutines.
It would for example be possible to create a library specifically
targeted at discrete event simulation or similar.

ObxCoroutines has been added for showing by example how plain coroutines,
Tasks and Iterators can be used.

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Apr 24, 2017 7:56 am
by Zinn
Josef Templ wrote: For the diffs see ... oops "An error occurred when trying to access the repository: Cannot allocate memory - 'git' --version --no-color"
There is a file blackbox-1.7.1-a1.856-buildlog.htlm but no 856.zip and no 856-setup.exe file in the unstable/issue#156 directory.
-Helmut

P.S. Thank you for your great work.

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Apr 24, 2017 9:26 am
by Josef Templ
Zinn wrote: There is a file blackbox-1.7.1-a1.856-buildlog.htlm but no 856.zip and no 856-setup.exe file in the unstable/issue#156 directory.
-Helmut
Something seems to be wrong with the Edis host.
May be there is a problem with the X-Server or xvfb command.
Also in Redmine I get an "out of memory" error when I try to open the Repository.

Ivan, can you please check this?

Thanks

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Apr 24, 2017 1:28 pm
by Ivan Denisov
We need to make preliminary voting to choose realization composition first.

I will check what's happening with the server after finish my work.

Re: issue-#156 adding Coroutines to BlackBox

Posted: Tue Apr 25, 2017 7:29 am
by Josef Templ
The complete diffs for this issue are now available, see https://redmine.blackboxframework.org/p ... 97faa7e691.

The latest build is http://blackboxframework.org/unstable/i ... a1.857.zip.

Also the Redmine Repository function works again.
Thanks to Ivan for fixing the Edis host problems.

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Tue Apr 25, 2017 9:59 am
by DGDanforth
Question:
How fast can a Coroutine run?
By that I mean that Service comes up for air 20 times a second.
How often can a Coroutine come up for air?
Is the granularity the same as for Services or is it finer?
-Doug

Re: issue-#156 adding Coroutines to BlackBox

Posted: Tue Apr 25, 2017 7:13 pm
by Robert
Josef Templ wrote:The solution is based on Windows fibers...
The Wikipedia coroutines page (https://en.wikipedia.org/wiki/Coroutine) says:
Consequently, the use of the fiber API to switch tasks is currently not a viable option in the .NET Framework.
Is there an issue here?

Re: issue-#156 adding Coroutines to BlackBox

Posted: Wed Apr 26, 2017 5:48 am
by Josef Templ
DGDanforth wrote:Question:
How fast can a Coroutine run?
By that I mean that Service comes up for air 20 times a second.
How often can a Coroutine come up for air?
Is the granularity the same as for Services or is it finer?
-Doug
Plain coroutines are completely independent from Services.Action.
A Transfer takes about 200 - 300 nanoseconds, depending on the hardware.

Tasks use the Services.Action infrastructure and are bound to the same behavior
except for one optimization: Task.Sleep(0) is transferring to main only occasionally,
i.e. when a certain amount of time has passed. Otherwise it is a dummy call
and the coroutine resumes immediately. This delivers close to 100% cpu utilization
and is much faster as can be seen in the Primes example.

The minimum cycle time for a Services.Action is about 62 milliseconds in normal mode and
16 milliseconds in server mode. This is a limitation imposed by the so-called
Windows idle period, which is affected by switching to server mode.
The granularity of Services.Ticks() is always 16 milliseconds, at least on my PC.

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Wed Apr 26, 2017 6:02 am
by Josef Templ
Robert wrote:
Josef Templ wrote:The solution is based on Windows fibers...
The Wikipedia coroutines page (https://en.wikipedia.org/wiki/Coroutine) says:
Consequently, the use of the fiber API to switch tasks is currently not a viable option in the .NET Framework.
Is there an issue here?
BlackBox has nothing in common with the .NET CLR infrastructure.

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Wed Apr 26, 2017 10:03 am
by luowy
Josef,
1, shared stack coroutine is more valuable than independent stack one;
2,sperate scheduler from coroutine is needed,
Q: where the trap handle code for coroutine? what happen when coroutine traped?