Page 10 of 11

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Jul 31, 2017 10:02 am
by Josef Templ
Robert wrote:Regarding Iterators:

1 - Would it be simpler for client code if the signature of Next was a function with a BOOLEAN return value? TRUE would mean the status was 'suspended', FALSE would mean 'returned'. Other statuses are, I think, impossible?

2 - It is possible (correct me if I am wrong) to call Coroutines.Transfer with an Iterator. I suspect that 99% of the time this would be done by accident, and 100% of the time it would lead to hard to read and understand code. So I suggest adding a documented TRAP to prevent this.
Of course it would still be called internally by Yield & Next.
ad 1: I had Next as a function as you suggested in an earlier version. When I applied it to the SameFringe problem
it turned out to be very dangerous because it is a function with a side effect. If called twice in an expression it modifies the state,
when not called due to short-cut evaluation it accidentally did not modify the state, etc.
It took me hours to recognize my stupid mistake.
Combining Next and something like HasNext is not a good idea, I am convinced.

An iterator subclass can introduce its own rule for detecting the end of the iteration if checking the state is unwanted.
It could, for example, return NIL at the end or set its own flag "hasNext", etc.

ad 2: Calling Coroutines.Transfer(iterator) instead of iterator.Next() is possible indeed. But there are already checks in place that
trap if called in a way that violates the semantics of iterators.
Besides that, it is hard to imagine that Coroutines.Transfer is called accidentally.
It is more likely to appear in tests of the robustness of the Coroutines module.

Regarding the docu: it would be easy to insert a link to the coroutine examples (ObxCoroutines) in the docu of module Coroutines.

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Jul 31, 2017 11:01 am
by Robert
Josef Templ wrote:Combining Next and something like HasNext is not a good idea, I am convinced.
Ok.
I still have a problem with the documentation; how does one get started on the documentation trail without using Info->Repository?
Maybe we should put a link to "Coroutines" into the "Map to the BlackBox Core" System Map, then also have a link from Coroutines to ObxCoroutines as you suggest.

I still think a link from a high level "Map" or Menu to the ObxMap is very desirable.

For me, and maybe some other people, an initial introduction to using Coroutines is to use a simple Iterator. I don't regard the current Obx Iterator example as a very simple example; it is more of an advanced application. It does not demonstate a general way of using any iterator as it exploits application specific state to determine end conditions. So I still suggest adding something along the lines of my "Typical design pattern" words to the Coroutines Docu, or at least adding a very simple Iterator example to Obx (as well as keeping the Same Fringe advanced example).

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Jul 31, 2017 11:10 am
by Robert
Josef Templ wrote:Besides that, it is hard to imagine that Coroutines.Transfer is called accidentally.
I think it is really easy to image. The Docu for Iterators does not explicitly say "Do not do this". It is the normal way to use plain Coroutines, and Iterators are Coroutines.
Remember, we are talking about beginners who have not yet understood appropriate design patterns for using these (probably) unfamiliar capabilities.

With hindsight call Coroutines.Transfer may seem silly, but we don't all start off with hindsight.

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Jul 31, 2017 1:41 pm
by Josef Templ
The docu of "Transfer" says explicitly:
"Note that using Transfer can lead to complex control flow and subtle programming errors. In case of using a special subclass (Iterator or Task) it is recommended that any transfer is done with one of the transfer wrapper methods provided by the subclass."

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Wed Apr 10, 2019 8:22 pm
by Robert
For the first time since Coroutines were added to BlackBox I've got a problem where I think Coroutines are the natural solution.

I now have what seems like a simple, natural, even elgant solution using iterators (the process of finding this solution was NOT simple or elegant!).

But I am getting quite a lot of low-level Kernel traps, involving illegal memory reads, cascades of small trap boxes, and BlackBox crashing out to Windows. Might there be something flakey in the implementation? Have other people found Coroutines to be reliable?

Re: issue-#156 adding Coroutines to BlackBox

Posted: Thu Apr 11, 2019 8:10 pm
by Robert
This is getting tiresome!

The sequence seems to be:

- I have a suspended Coroutine.
- I unload its defining module.
- I edit, and sometimes save, the module's source code.
- I compile the module.

At that point I get an infinite series of low-level traps, resulting, in a few seconds, with BlackBox closing down loosing all my data.

The top line of the trap reads: Illegal address, ad = 00, or ad = 04 (I think, it happens too quickly to make accurate notes).

It doesn't happen every time, maybe about 1 in 10 chance. More work lost!

Update: The last TRAP was called from Kernel.MarkLocals.
Update: The last TRAP was called from Kernel.MarkReferencedCoroutines.
Update: This time is was TrapHandler called from MarkLocals.
Update: ... and MarkLocals was called from FastCollect.

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Apr 15, 2019 3:17 pm
by Josef Templ
Robert,
there seems to be an issue with coroutines in combination with module unloading followed by a garbage collector cycle.
Something is not cleaned up correctly.

I hope to find the time to check this out in the near future.
Please remind me if it takes too long.

It would be helpful to have a stripped down test case that exhibits the error.
You can call the garbage collector explicitly for forcing a GC cycle.
This may help to get a reproducible error case.

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Apr 15, 2019 4:46 pm
by Josef Templ
Robert,

are you resuming execution of the suspended and unloaded coroutine somehow?

- Josef

Re: issue-#156 adding Coroutines to BlackBox

Posted: Mon Apr 15, 2019 5:53 pm
by Robert
Josef Templ wrote:are you resuming execution of the suspended and unloaded coroutine somehow?
Not deliberately!

My code is (unfortunately) very far from stripped down, and only crashes infrequently, and not predictably, so creating a simple example test case does not seem to be easy.

The most predictable circumstance seems to be that the crashes occur running the compiler just after unloading my module, but even this is not definite. The last crash occurred running my module.

Re: issue-#156 adding Coroutines to BlackBox

Posted: Tue Apr 16, 2019 6:29 pm
by Josef Templ
And it only happens when you unload the suspended coroutine's module, right?
What about stopping the coroutine(s) before unloading the module?
As a workaround, of course, it should not crash.

One thing that can be reproduced easily is a suspicious trap window when
unloading a suspended coroutine's module and resuming the coroutine later.
The trap window displays strange chinese characters, but it never crashes the process
in my experiments. Nevertheless, it is not looking right.

- Josef