issue-#190 add comment about stack and heap memory config.

issue-#190 add comment about stack and heap memory config.

Postby Josef Templ » Tue Nov 06, 2018 9:18 pm

I have created https://redmine.blackboxframework.org/issues/190 for adding comment about subtle dependencies.

When increasing the stack size in DevLinker for creating an exe file with a larger stack the
heap size must be reduced. Otherwise there may be strange effects for example when opening
a file dialog box because Windows functions are running out of memory.
This fact should be added as a comment in Kernel and DevLinker.

- Josef
User avatar
Josef Templ
 
Posts: 1921
Joined: Tue Sep 17, 2013 6:50 am

Re: issue-#190 add comment about stack and heap memory confi

Postby Josef Templ » Sun Nov 11, 2018 9:20 am

I have committed a first draft of comments added.
The comments, or more precisely the info added to P-S-I, reflect my current understanding of the
limits as observed from experiments. The ground truth is not yet known, at least not to me.

For the changes see https://redmine.blackboxframework.org/projects/blackbox/repository/diff?utf8=%E2%9C%93&rev=c3d1bb8947eaadfb2d24667bbb62d656a9c65136&rev_to=a940ecb3063c8387b57c69e4496ea14e641e5c4b.

- Josef
User avatar
Josef Templ
 
Posts: 1921
Joined: Tue Sep 17, 2013 6:50 am

Re: issue-#190 add comment about stack and heap memory confi

Postby Robert » Mon Nov 12, 2018 8:26 pm

Josef Templ wrote:I have committed a first draft of comments added.
The comments, or more precisely the info added to P-S-I, reflect my current understanding of the
limits as observed from experiments. The ground truth is not yet known, at least not to me.

For the changes see https://redmine.blackboxframework.org/projects/blackbox/repository/diff?utf8=%E2%9C%93&rev=c3d1bb8947eaadfb2d24667bbb62d656a9c65136&rev_to=a940ecb3063c8387b57c69e4496ea14e641e5c4b.

- Josef


1 - I don't want to be pedantic about the difference between 2^10 & 10^3, but I do prefer writing "kB" for kilobyte to writing "KB".

2 - Do we have any example numbers for the total of stack size + heapsize that are known to work reliably, and totals that are known to cause problems?

3 - Can we make the reserve sizes as big as we like, and not have problems until the commit size becomes big?
User avatar
Robert
 
Posts: 930
Joined: Sat Sep 28, 2013 11:04 am
Location: Edinburgh, Scotland

Re: issue-#190 add comment about stack and heap memory confi

Postby Josef Templ » Mon Nov 12, 2018 9:39 pm

> 1 - I don't want to be pedantic about the difference between 2^10 & 10^3, but I do prefer writing "kB" for kilobyte to writing "KB".
What about MB? For me it would also be OK to give the precise numbers.

> 2 - Do we have any example numbers for the total of stack size + heapsize that are known to work reliably, and totals that are known to cause problems?
All I know is that the standard values work well. The results of experiments are not always reproducible.
So it is hard to find out what is the real limit. From more testing I have now also observed the following problem:
When I increase the reserved stack size to 9MB (without committing more stack space, without any changes in the Kernel)
and then opening a file dialog, it works well.
After closing the dialog and reopening it again, it shows anomalies. Repeating it, results in an out-of-memory trap.

3 - Can we make the reserve sizes as big as we like, and not have problems until the commit size becomes big?
Yes, it seems so, but there is a common wisdom that there should not be more than about 1.5 GB reserved.
We are very close, if not above, to that informal limit.
May be we should test the rule: reserved heap + reserved stack <= (1536 + 2) * 1024 * 1024

- Josef
User avatar
Josef Templ
 
Posts: 1921
Joined: Tue Sep 17, 2013 6:50 am

Re: issue-#190 add comment about stack and heap memory confi

Postby Robert » Mon Nov 12, 2018 10:07 pm

Josef Templ wrote:> 1 - I don't want to be pedantic about the difference between 2^10 & 10^3, but I do prefer writing "kB" for kilobyte to writing "KB".
What about MB? For me it would also be OK to give the precise numbers.

My point is that in the SI system "k" means kilo, "M" means mega, & "K" means Kelvin, so "KB" is not appropriate here.

Saying "kilobyte" when you mean 1024 bytes (and the same for mega) is a common, and accepted, casual way of speaking in this context. Giving the precise numbers would only be necessary if we knew a precise limit at which the memory problem occurred.
User avatar
Robert
 
Posts: 930
Joined: Sat Sep 28, 2013 11:04 am
Location: Edinburgh, Scotland

Re: issue-#190 add comment about stack and heap memory confi

Postby Josef Templ » Sun Nov 18, 2018 12:34 pm

Robert wrote:Saying "kilobyte" when you mean 1024 bytes (and the same for mega) is a common, and accepted, casual way of speaking in this context. Giving the precise numbers would only be necessary if we knew a precise limit at which the memory problem occurred.


kB would imply that we mean 1000, but actually we mean 1024.
See https://en.wikipedia.org/wiki/Kilobyte
Also Windows Explorer is using KB for 1024.

I have done more tests and observed strange results.
With a 32 MB stack I need to reduce the heap to about 150 MB (from 1.5 GB).
With 16 MB stack I need to reduce the heap to about 900 MB.
Therefore I cannot deduce any rule for working numbers.
We would need to know more about memory organization in Windows.
So far the docu can only be taken as a warning that it is best not to change the stack size.

See changes at https://redmine.blackboxframework.org/projects/blackbox/repository/diff?utf8=%E2%9C%93&rev=ded89cb08bba1ffe979c32a2abd698f577b1e55f&rev_to=c3d1bb8947eaadfb2d24667bbb62d656a9c65136


- Josef
User avatar
Josef Templ
 
Posts: 1921
Joined: Tue Sep 17, 2013 6:50 am

Re: issue-#190 add comment about stack and heap memory confi

Postby Robert » Mon Nov 19, 2018 3:57 pm

Josef Templ wrote:... Also Windows Explorer is using KB for 1024.

That's horrible, but lets forget that.
Maybe I am the only person here who cares that the SI system redefined the second, metre, kilogram, ampere, kelvin, mole, candela, volt, & ohm last week?

Is this topic ready for voting?
User avatar
Robert
 
Posts: 930
Joined: Sat Sep 28, 2013 11:04 am
Location: Edinburgh, Scotland

Re: issue-#190 add comment about stack and heap memory confi

Postby Josef Templ » Mon Nov 19, 2018 6:06 pm

for me it is ready for voting.

- Josef
User avatar
Josef Templ
 
Posts: 1921
Joined: Tue Sep 17, 2013 6:50 am

Re: issue-#190 add comment about stack and heap memory confi

Postby cfbsoftware » Mon Nov 19, 2018 8:23 pm

Just two minor typos:

Code: Select all
squares instead of  icons, missig tree view,

(extra space before icons and missig is missing the 'n')
cfbsoftware
 
Posts: 200
Joined: Wed Sep 18, 2013 10:06 pm

Re: issue-#190 add comment about stack and heap memory confi

Postby Josef Templ » Tue Nov 20, 2018 8:02 pm

cfbsoftware wrote:Just two minor typos:

Code: Select all
squares instead of  icons, missig tree view,

(extra space before icons and missig is missing the 'n')



Thanks for the careful reading. Fixed.
I also removed double spaces at some other locations in this document
and fixed more typos after feeding the text to a spell checker.

- Josef
User avatar
Josef Templ
 
Posts: 1921
Joined: Tue Sep 17, 2013 6:50 am


Return to Resolved (Documentation)

Who is online

Users browsing this forum: No registered users and 2 guests