Page 1 of 1

issue-#190 add comment about stack and heap memory config.

Posted: Tue Nov 06, 2018 9:18 pm
by Josef Templ
I have created https://redmine.blackboxframework.org/issues/190 for adding comment about subtle dependencies.

When increasing the stack size in DevLinker for creating an exe file with a larger stack the
heap size must be reduced. Otherwise there may be strange effects for example when opening
a file dialog box because Windows functions are running out of memory.
This fact should be added as a comment in Kernel and DevLinker.

- Josef

Re: issue-#190 add comment about stack and heap memory confi

Posted: Sun Nov 11, 2018 9:20 am
by Josef Templ
I have committed a first draft of comments added.
The comments, or more precisely the info added to P-S-I, reflect my current understanding of the
limits as observed from experiments. The ground truth is not yet known, at least not to me.

For the changes see https://redmine.blackboxframework.org/p ... 4e641e5c4b.

- Josef

Re: issue-#190 add comment about stack and heap memory confi

Posted: Mon Nov 12, 2018 8:26 pm
by Robert
Josef Templ wrote:I have committed a first draft of comments added.
The comments, or more precisely the info added to P-S-I, reflect my current understanding of the
limits as observed from experiments. The ground truth is not yet known, at least not to me.

For the changes see https://redmine.blackboxframework.org/p ... 4e641e5c4b.

- Josef
1 - I don't want to be pedantic about the difference between 2^10 & 10^3, but I do prefer writing "kB" for kilobyte to writing "KB".

2 - Do we have any example numbers for the total of stack size + heapsize that are known to work reliably, and totals that are known to cause problems?

3 - Can we make the reserve sizes as big as we like, and not have problems until the commit size becomes big?

Re: issue-#190 add comment about stack and heap memory confi

Posted: Mon Nov 12, 2018 9:39 pm
by Josef Templ
> 1 - I don't want to be pedantic about the difference between 2^10 & 10^3, but I do prefer writing "kB" for kilobyte to writing "KB".
What about MB? For me it would also be OK to give the precise numbers.

> 2 - Do we have any example numbers for the total of stack size + heapsize that are known to work reliably, and totals that are known to cause problems?
All I know is that the standard values work well. The results of experiments are not always reproducible.
So it is hard to find out what is the real limit. From more testing I have now also observed the following problem:
When I increase the reserved stack size to 9MB (without committing more stack space, without any changes in the Kernel)
and then opening a file dialog, it works well.
After closing the dialog and reopening it again, it shows anomalies. Repeating it, results in an out-of-memory trap.

3 - Can we make the reserve sizes as big as we like, and not have problems until the commit size becomes big?
Yes, it seems so, but there is a common wisdom that there should not be more than about 1.5 GB reserved.
We are very close, if not above, to that informal limit.
May be we should test the rule: reserved heap + reserved stack <= (1536 + 2) * 1024 * 1024

- Josef

Re: issue-#190 add comment about stack and heap memory confi

Posted: Mon Nov 12, 2018 10:07 pm
by Robert
Josef Templ wrote:> 1 - I don't want to be pedantic about the difference between 2^10 & 10^3, but I do prefer writing "kB" for kilobyte to writing "KB".
What about MB? For me it would also be OK to give the precise numbers.
My point is that in the SI system "k" means kilo, "M" means mega, & "K" means Kelvin, so "KB" is not appropriate here.

Saying "kilobyte" when you mean 1024 bytes (and the same for mega) is a common, and accepted, casual way of speaking in this context. Giving the precise numbers would only be necessary if we knew a precise limit at which the memory problem occurred.

Re: issue-#190 add comment about stack and heap memory confi

Posted: Sun Nov 18, 2018 12:34 pm
by Josef Templ
Robert wrote:Saying "kilobyte" when you mean 1024 bytes (and the same for mega) is a common, and accepted, casual way of speaking in this context. Giving the precise numbers would only be necessary if we knew a precise limit at which the memory problem occurred.
kB would imply that we mean 1000, but actually we mean 1024.
See https://en.wikipedia.org/wiki/Kilobyte
Also Windows Explorer is using KB for 1024.

I have done more tests and observed strange results.
With a 32 MB stack I need to reduce the heap to about 150 MB (from 1.5 GB).
With 16 MB stack I need to reduce the heap to about 900 MB.
Therefore I cannot deduce any rule for working numbers.
We would need to know more about memory organization in Windows.
So far the docu can only be taken as a warning that it is best not to change the stack size.

See changes at https://redmine.blackboxframework.org/p ... 56a9c65136


- Josef

Re: issue-#190 add comment about stack and heap memory confi

Posted: Mon Nov 19, 2018 3:57 pm
by Robert
Josef Templ wrote:... Also Windows Explorer is using KB for 1024.
That's horrible, but lets forget that.
Maybe I am the only person here who cares that the SI system redefined the second, metre, kilogram, ampere, kelvin, mole, candela, volt, & ohm last week?

Is this topic ready for voting?

Re: issue-#190 add comment about stack and heap memory confi

Posted: Mon Nov 19, 2018 6:06 pm
by Josef Templ
for me it is ready for voting.

- Josef

Re: issue-#190 add comment about stack and heap memory confi

Posted: Mon Nov 19, 2018 8:23 pm
by cfbsoftware
Just two minor typos:

Code: Select all

squares instead of  icons, missig tree view,
(extra space before icons and missig is missing the 'n')

Re: issue-#190 add comment about stack and heap memory confi

Posted: Tue Nov 20, 2018 8:02 pm
by Josef Templ
cfbsoftware wrote:Just two minor typos:

Code: Select all

squares instead of  icons, missig tree view,
(extra space before icons and missig is missing the 'n')

Thanks for the careful reading. Fixed.
I also removed double spaces at some other locations in this document
and fixed more typos after feeding the text to a spell checker.

- Josef