issue-#14.2 Tuning the build engine

User avatar
Josef Templ
Posts: 2047
Joined: Tue Sep 17, 2013 6:50 am

Re: Tuning the build engine

Post by Josef Templ »

Config.CopyLog is too complicated and introduces a limit on the line length.
Simplifications pushed to issue-#14.

ScriptBuild.AppendProperty used to work well so far.
So why did you change it? It is horribly complicated now.

- Josef
User avatar
Josef Templ
Posts: 2047
Joined: Tue Sep 17, 2013 6:50 am

Re: Tuning the build engine

Post by Josef Templ »

regarding the bbscript updates: the problem only arises if a feature is
removed from the language/compiler that is still in use in parallel open branches.
Now we know about this problem and have to serialize such changes, which,
fortunately, are very rare. I do not expect any more for release 1.7.
Fixing this by introducing multiple versions of bbscript is complicated and not worth the effort.
A possible compromise would be to update bbscript.exe only when building 'master'.
This would allow to continue working in parallel on open branches but of course
still requires to merge the incompatible branch as the last one.

- Josef
Ivan Denisov
Posts: 1700
Joined: Tue Sep 17, 2013 12:21 am
Location: Russia

Re: Tuning the build engine

Post by Ivan Denisov »

Josef Templ wrote:ScriptBuild.AppendProperty used to work well so far.
So why did you change it? It is horribly complicated now.
It is TRAPing for my projects because Strings file is bigger and busy while saving, so can not be saved. It is better to open file once during the session and save with different name, it solved the problem. Now I have the fork of Script for my projects. In future I wanted to introduce some manual how to make derivatives build engines in base of Center one. So I want that our build engine works well in various cases — with long System/Rsrc/Strings.odc file and produce readable log while compiling subsystems with Unicode names.
Ivan Denisov
Posts: 1700
Joined: Tue Sep 17, 2013 12:21 am
Location: Russia

Re: Tuning the build engine

Post by Ivan Denisov »

Josef Templ wrote:regarding the bbscript updates: the problem only arises if a feature is
removed from the language/compiler that is still in use in parallel open branches.
Now we know about this problem and have to serialize such changes, which,
fortunately, are very rare. I do not expect any more for release 1.7.
Fixing this by introducing multiple versions of bbscript is complicated and not worth the effort.
A possible compromise would be to update bbscript.exe only when building 'master'.
This would allow to continue working in parallel on open branches but of course
still requires to merge the incompatible branch as the last one.

- Josef
I am not agree with you, we should solve the problem anyway or it will block compiler development. Now it is impossible to make compiler experimental branch without breaking the work of the system. I am ready to perform this changes. That is not that difficult as you expecting.
User avatar
Josef Templ
Posts: 2047
Joined: Tue Sep 17, 2013 6:50 am

Re: Tuning the build engine

Post by Josef Templ »

> It is TRAPing for my projects because Strings file is bugger and busy while saving, so can not be saved.

I don't understand this. This is not English.
Please explain in other words.
Are you responding to CopyLog or AppendProperty?

- Josef
Ivan Denisov
Posts: 1700
Joined: Tue Sep 17, 2013 12:21 am
Location: Russia

Re: Tuning the build engine

Post by Ivan Denisov »

Josef Templ wrote:> It is TRAPing for my projects because Strings file is bugger and busy while saving, so can not be saved.

I don't understand this. This is not English.
Please explain in other words.
Are you responding to CopyLog or AppendProperty?
I am responding about AppendProperty. The problem is that if Strings file is bigger, BlackBox not closing it fast and can not save over it the new version with appended properties. So with old Script/Mod/Build.odc building processes terminating with TRAP. Here I mean not Center version, but my projects based on Center building engine mechanism. So, the solution is to open file once and save with other name.
User avatar
Josef Templ
Posts: 2047
Joined: Tue Sep 17, 2013 6:50 am

Re: Tuning the build engine

Post by Josef Templ »

> The problem is that if Strings file is bigger, BlackBox not closing it fast and can not save over it the new version with appended properties.

I don't understand this. This is not English.
Please explain in other words or revert that change.
It looks highly suspicious.

- Josef
Ivan Denisov
Posts: 1700
Joined: Tue Sep 17, 2013 12:21 am
Location: Russia

Re: Tuning the build engine

Post by Ivan Denisov »

Josef Templ wrote:>Please explain in other words or revert that change.
It looks highly suspicious.
If I am using the System/Rsrc/Strings.odc file that is twice bigger than regular one, I got TRAP during the call of AppendProperty. It can be after first, second or third call.
This irregularity was strange for me also, but then I analyse Build.odc and TRAP position. It became obvious, that the problem is about using of the same filename. If BlackBox has time to free file opened for reading, the saving works well. This happen now with Center build machine, when Strings file is small.
Firstly I tried to call f.Close, but this did not help. So the simplest solution is to use another name.

You can simply reproduce this error! Just increase the size of Strings file (duplicate the content several times) and try to run:

Code: Select all

bbscript.exe /PAR appbuild/appendProps.txt
You will see StdLog.txt:

Code: Select all

invoking DevCompiler.CompileThis
compiling "ScriptBuild"   1080   0
invoking ScriptBuild.AppendProperty
TRAP caught
User avatar
Josef Templ
Posts: 2047
Joined: Tue Sep 17, 2013 6:50 am

Re: Tuning the build engine

Post by Josef Templ »

I cannot reproduce this under Windows.
Seems to be wine related.
If it is a timing problem, there must be some asynchronous operation involved.
This is then probably a bug in wine.
If it is really a timing problem, why don't you simply insert a 'Sleep' call?

In general, if this is a wine related problem it may hit us or other users at
different occasions as well. So we should find out what is going on instead of working around
the bug by introducing inappropriate changes to Build.AppendProperty.

- Josef
User avatar
Josef Templ
Posts: 2047
Joined: Tue Sep 17, 2013 6:50 am

Re: Tuning the build engine

Post by Josef Templ »

Ivan, excellent analysis. It really seems to be a timing problem.
The fix, however, is not so excellent.
A single line does it all.
No need to change the overall structure of AppendProperty to the very strange
form of finishing the work in the CLOSE section!!

It is a timing problem under wine only. So I added:

Code: Select all

IF Dialog.IsWine() THEN WinApi.Sleep(1) END;
Works well in all my tests so far and makes it explicit what the work around is all about.
In your version, it was not visible at all and there was no way to control the introduced delay.
It just happened by accident that it was large enough.
Even my version must be considered a temporary fix, though.
Such a code should not exist at all. If it is a bug in wine, however,
we cannot do much about it except working around this bug.

The file bbscript.exe is now only updated if building for master.
This is VERY simple to do and it has an excellent cost/benefit ratio.
The advantage is that we can do any build now (except for master) without any side effect.
Language restrictions are so rare that it really is a waste of time to even think about a more
complicated version.

From my point of view, the changes in issue-#14 can be merged now to master.

- Josef
Post Reply