usage of undocumented low-level functions

[Josef Templ]

This voting is about a fundamental software engineering question.

The background of this voting is a discussion about the usage of Utf8 conversion functions in the module DevBrowser.
Here is a summary of the discussion:
- In BB1.6 there were no such conversions at all.
- For issue-#19 we added conversions to DevBrowser based on the CPC 1.7 proposal using Kernel.
- For issue-#57 Ivan added additional conversions in DevBrowser using Strings.
- Eventually, issue-#57 will be integrated with issue-#19 because it is a follow up change due to an oversight in issue-#19.
- I proposed to change all Utf8 conversions in DevBrowser such that they use Strings.
- Ivan proposed to change all Utf8 conversions in all modules such that they use Kernel.
I am convinced that this is the wrong way because there is no strong reason for using Kernel
for this purpose in DevBrowser. Kernel is an undocumented low-level module
that should only be used in exceptional cases and if there are good reasons for doing so.
Ivan argues that Strings is more for export, i.e. for users of BB outside the BB development itself
and that the Strings.*Utf8* functions are to be changed by the user such that they do a different form
of conversion and such changes should not interfere with Utf8 conversions within BB itself.

I hope that I have summed up the discussion correctly.
Since the question may arise in a different context as well, it is formulated such that the Utf8 conversion is taken
as an example but the voting is not restricted to this special case.

Feel free to add a comment to your vote.

- Josef

[luowy]

I prefer implement a enhanced utf8 converter in Strings Module,a simple one(current) keep in Kernel.
maybe Winapi.MultiByteToWideChar(CP_UTF8,..) is a good reference for the enhanced one.

Kernel one used by low level modules(base,runtime) only:compiler,debugger,meta...for efficiency.
Strings one used by high level modules :framework and third party modules,for endhanced feature like 4 bytes encoding...

so for DevBrowser I prefer using Kernel One.

luowy

[Ivan Denisov]

Josef, sorry, but you are falsifying the voting idea! "wherever possible" — what is this?

Kernel should be used "only for identifiers conversions" not "wherever possible". You distorted the voting idea

I am voting for the second "a very good reason for using it", because for me "identifiers conversions" is a good reason for using Kernel conversion. So, according Luowe comment and my, you need to make next voting:

1. The "identifiers conversions" is a very good reason for using Kernel conversion

2. The "identifiers conversions" is not a very good reason for using Kernel conversion

3. Abstain

[Robert]

I am not convinced this is

a fundamental software engineering question

. Sometimes low-level is appropriate for efficiency reasons. Sometimes a high-level of abstraction is good for simplicity and being comprehensible. Sometimes Modules form a natural hierarchy and it is clear which is the appropriate route. Sometimes it is a matter of aesthetic judgement, at which some people are more skilled than others. This skill cannot be replaced by simple "rules" or "guidelines".

So on the general question I intend to abstain.

On the particular question I am aware that a lot of work has been done by others, but I have not personally kept close to the details, so again I intend to abstain.

[Josef Templ]

I am glad to see that we all agree on a fundamental level.
There may of course be a discussion of what constitutes a strong reason that
justifies an exception. This must be treated on a case by case basis.
The voting about the DevBrowser changes will include it.

- Josef

[Ivan Denisov]

I am glad to see that we all agree on a fundamental level.
There may of course be a discussion of what constitutes a strong reason that
justifies an exception. This must be treated on a case by case basis.
The voting about the DevBrowser changes will include it.

- Josef

Josef, we are not ready for DevBrowser voting until we not decide about conversion of identifiers. Please make separate voting before.

[Josef Templ]

Josef, we are not ready for DevBrowser voting until we not decide about conversion of identifiers. Please make separate voting before.

Do you mean conversion of identifiers in DevBrowser or conversion of identifiers in general?

- Josef

[Zinn]

I hope that I have summed up the discussion correctly.

Yes, you have. And the answer of your question is yes.
But the question doesn't match the problem which arise this vote.
Let me explain.

The current state of identifier translation is:
We added into module Kernel:

PROCEDURE IsAlpha (ch: CHAR): BOOLEAN;
PROCEDURE IsLower (ch: CHAR): BOOLEAN;
PROCEDURE IsUpper (ch: CHAR): BOOLEAN;

PROCEDURE Lower (ch: CHAR): CHAR;
PROCEDURE Upper (ch: CHAR): CHAR;

PROCEDURE StringToUtf8 (IN in: ARRAY OF CHAR; OUT out: ARRAY OF SHORTCHAR; OUT res: INTEGER);
PROCEDURE Utf8ToString (IN in: ARRAY OF SHORTCHAR; OUT out: ARRAY OF CHAR; OUT res: INTEGER);

And we added into module Strings:

PROCEDURE IsAlpha (ch: CHAR): BOOLEAN;
PROCEDURE IsAlphaNumeric (ch: CHAR): BOOLEAN;
PROCEDURE IsLower (ch: CHAR): BOOLEAN;
PROCEDURE IsNumeric (ch: CHAR): BOOLEAN;
PROCEDURE IsUpper (ch: CHAR): BOOLEAN;

PROCEDURE Lower (ch: CHAR): CHAR;
PROCEDURE Upper (ch: CHAR): CHAR;

PROCEDURE IsIdent (ch: CHAR): BOOLEAN;
PROCEDURE IsIdentStart (ch: CHAR): BOOLEAN;

PROCEDURE StringToUtf8 (IN in: ARRAY OF CHAR; OUT out: ARRAY OF SHORTCHAR; OUT res: INTEGER);
PROCEDURE Utf8ToString (IN in: ARRAY OF SHORTCHAR; OUT out: ARRAY OF CHAR; OUT res: INTEGER);

And module Strings imports Kernel.
Note: In BB 1.6 module Strings doesn't import Kernel.


Back to the history:
The original implementation introduced a new module Characters:

PROCEDURE IsAlpha (ch: CHAR): BOOLEAN;
PROCEDURE IsAlphaNumeric (ch: CHAR): BOOLEAN;
PROCEDURE IsLower (ch: CHAR): BOOLEAN;
PROCEDURE IsNumeric (ch: CHAR): BOOLEAN;
PROCEDURE IsUpper (ch: CHAR): BOOLEAN;

PROCEDURE Lower (ch: CHAR): CHAR;
PROCEDURE Upper (ch: CHAR): CHAR;

PROCEDURE IsIdent (ch: CHAR): BOOLEAN;
PROCEDURE IsIdentStart (ch: CHAR): BOOLEAN;

PROCEDURE StringToUtf8 (IN in: ARRAY OF CHAR; OUT out: ARRAY OF SHORTCHAR; OUT res: INTEGER);
PROCEDURE Utf8ToString (IN in: ARRAY OF SHORTCHAR; OUT out: ARRAY OF CHAR; OUT res: INTEGER);

The module Kernel imports Characters.
And module Strings imports neither Characters nor Kernel.
Module Strings was unchanged.

The question about import Kernel does not arise.
Import Character is used for identifier translation.
All stuff about identifier translation was inside one module called Characters.


Which ugly design decision you won't have?

* Kernel is not the lowest module
* Using Kernel instead of Strings for the variable translation
* Strings should not depend on Kernel
* Linking Characters + Kernel
* Linking Math + Strings + Kernel
* Don't split module Characters into Kernel & Strings
* Don't have the same procedure in Kernel & Strings

Gentlemen cast your vote.

[Ivan Denisov]

Josef, we are not ready for DevBrowser voting until we not decide about conversion of identifiers. Please make separate voting before.

Do you mean conversion of identifiers in DevBrowser or conversion of identifiers in general?

In general.

[Josef Templ]

...
The module Kernel imports Characters.
And module Strings imports neither Characters nor Kernel.
Module Strings was unchanged.

The question about import Kernel does not arise.
Import Character is used for identifier translation.
All stuff about identifier translation was inside one module called Characters.

Which ugly design decision you won't have?

* Kernel is not the lowest module
* Using Kernel instead of Strings for the variable translation
* Strings should not depend on Kernel
* Linking Characters + Kernel
* Linking Math + Strings + Kernel
* Don't split module Characters into Kernel & Strings
* Don't have the same procedure in Kernel & Strings

> Which ugly design decision you won't have?
I don't understand this english? Please explain in other words.

In general, I don't understand the meaning of this posting.

It is clear that if we give up the requirement not to import anything from Kernel,
we have the option of importing Strings (or former Characters) from Kernel.
But then we run into the problem that existing link commands will fail
because the list of linked modules is incomplete and needs to be changed
potentially at many places outside BB itself.
A similar situation is with HostPackedFiles and its relation with DevDependencies.
Helmut, this problem with HostPackedFiles has been pointed out by yourself.

The current design avoids any link problems and leaves Kernel as the lowest level module.
In order to avoid code replication, Strings imports Kernel. This is a compromise.
It could be changed such that import of Kernel is avoided but code replication would then
be introduced. In addition, Strings would then have to import WinApi, which is also kind of ugly.
But is this really the topic of this voting?

- Josef