issue-#9: adding module Characters → #19

[Josef Templ]

As an experiment for starting the integration of the CPC 1.7 changes and for seeing
how our tool chain works and where it needs to be refined I have done the following:

1. I have created a Feature issue in the redmine issue tracker for the first item on the CPC 1.7 change list.
see http://redmine.blackboxframework.org/issues/9

2. Following Git best practices I have created a topic branch for the bbcp repository based on master.
The topic branch is named with respect to the redmine issue.
see https://github.com/BlackBoxCenter/bbcb/ ... Characters

3. I have extracted the changes that refer to this issue from Helmuts change list and the CPC 1.7 rc sources.
Basically there is a new module named Characters. In addition there was a module named Util, which is not used anywhere but
provided for documentation purposes. Since I didn't want to pollute the global module name space with such a module,
I have moved it into the examples as ObxFileUtils. I made some minor improvements and spelling corrections.
I adapted module Kernel for using module Characters according to CPC 1.7.
I have adapted some docu files.

4. I have committed and pushed the changes into the topic branch.

5. I have not touched the other files that are changed in the CPC change number 1 (Config, About) because
they do not have any relation with adding module Characters. They would belong to different topics.
There are many more related files that will be affected (changes 21, 22 in the CPC list) but these
are not yet covered.

You can see the changes in GitHub or in the redmine mirror when selecting the appropriate branch.
In redmine you can diff the changes even for .odc files.
I have not yet created a pull request for merging with master because it is too early for that.
This is intended as a starting point for a review and for getting used to the tool chain.

From this example one can see a number of problems:
1. there are independent issues combined in single CPC change numbers.
2. there are related changes spread over multiple CPC change numbers.
3. it will not be easy to get all the changes step by step but we can try.
4. redmine has a problem with diff for module Kernel. Ivan could you check this?
5. for power users it will be unavoidable to clone the repository to your local machine and work
with a local Git client where you can use BlackBox as a diff viewer.
6. This first example is one of the most complex ones, I guess, because it adds new files and
changes many. In addition, it is not very well-prepared so it may trigger a lot of discussions.

- Josef

[DGDanforth]

Excellent Josef. Exactly what is necessary to shake down the whole process of software
maintenance using git and redmine.

I suspect that the problem of integrating Characters exceeds both of those tools and that
additional tool (or methods) will be needed (or need to be created).

For example I just built a module "MyImports" which finds all of the files which import a given module.
That is a BB specific tool but a useful one and might benefit the maintenance of BB.

-Doug

[Ivan Denisov]

4. redmine has a problem with diff for module Kernel. Ivan could you check this?

I fixed this. It looks nice now, the Kernel.odc converted well.

[Josef Templ]

Center members, please comment on this issue.

It is not only about the issue itself but also about finding out the best way of
doing a code/issue review. Should we do it in this phpBB forum or in GitHub or in redmine?
In GitHub you can add a comment to a commit, for example.
Since we are currently using the redmine issue tracker, there is no way of
adding a comment to the issue in GitHub because there is no such issue.
Adding a comment in the redmine issue tracker is possible.
What is the best way to discuss this issue?

- Josef

[DGDanforth]

My preference is to use a single (unified) system.

[Josef Templ]

Here are some thoughts about the design of the module Characters as proposed in Feature #9.
May be it triggers some discussion.

1. Characters is introduced as a new lowest-level module. It is imported by module Kernel,
which until now was the lowest-level module. This is kind of UGLY, isn't it?
In addition it requires to change compile and link commands in many otherwise unaffected
source and docu files. Simply search for "DevLinker.Link" in Sources and Docu to get the idea.

2. Characters.IsIdentChar(ch) will be used in DevCPS (part of the CP compiler) for scanning identifiers.
It must be tuned for better efficiency because this is a character level operation.

3. Module Strings already contains some character operations.
Couldn't it also contain the functionality of module Characters?
For the part required by module Kernel it could provide a simple wrapper in order to
avoid code replication and the import of Kernel in client modules. Thus, the part of Characters
needed in Kernel could be moved to Kernel, the other part could be moved to
Strings plus some trivial wrapper code to call Kernel functions.

This would avoid the introduction of the new module Characters and the change of
compile and link commands in many many files.
It would also preserve module Kernel as the lowest-level module
and it would create the smallest possible linked application files because not
all the code of Characters is needed by Kernel.

@Helmut: has this approach been considered when introducing module Characters?
Would it still be usable for Informatika-21 as taught in Russia?

- Josef

[Josef Templ]

My preference is to use a single (unified) system.

and which one is this?

[Ivan Denisov]

@Helmut: has this approach been considered when introducing module Characters?
Would it still be usable for Informatika-21 as taught in Russia?

Some of his comments are here: http://forum.oberoncore.ru/viewtopic.ph ... 042#p86932

[Zinn]

Here there is a discussion about two different topics:

1. How to develop with Git
2. Introduce the module Characters.

I have nothing to say about develop with Git. There I have no experience. Now I write about the module Characters. Ivan's link above is only interesting for the history. My comments there are out of date.

For those friends who would like to read the history please do it in the following order:

1. National.odc for BB 1.6
28. 08. 2013
http://forum.oberoncore.ru/viewtopic.php?f=130&t=4468

2. Report illegal characters
11. 09. 2013
http://forum.oberoncore.ru/viewtopic.php?f=130&t=4487

3. Using BB 1.6 for shkola (with code page)
15. 09. 2013
http://forum.oberoncore.ru/viewtopic.php?f=130&t=4490

4. Coming soon: BlackBox Component Builder 1.7 CPC Edition
09. 03. 2014
http://forum.oberoncore.ru/viewtopic.php?f=130&t=5018

5. Module Characters
26. 03. 2014
http://forum.oberoncore.ru/viewtopic.php?f=130&t=5042

6. Adding support for Greek and Cyrillic code pages; CPC 1.7 changes 1, 2, 3, 5, 21, 22, 43 and 63
29. 07. 2014
http://redmine.blackboxframework.org/issues/9

7. Feature #9: adding module Characters
30. 07. 2014
http://forum.blackboxframework.org/view ... f=41&t=109

It was a long journey. The name of this module was first National, later Unicode and final Characters. This month I revised my definition of the module Characters to

DEFINITION Characters;

PROCEDURE IsLetter (ch: CHAR): BOOLEAN;
PROCEDURE IsCap (ch: CHAR): BOOLEAN;
PROCEDURE IsLow (ch: CHAR): BOOLEAN;

PROCEDURE IsFirstIdentChar (ch: CHAR): BOOLEAN;
PROCEDURE IsIdentChar (ch: CHAR): BOOLEAN;

PROCEDURE Cap (ch: CHAR): CHAR;
PROCEDURE Low (ch: CHAR): CHAR;

PROCEDURE LongIdent (IN in: ARRAY OF SHORTCHAR; OUT out: ARRAY OF CHAR);
PROCEDURE ShortIdent (IN in: ARRAY OF CHAR; OUT out: ARRAY OF SHORTCHAR);

END Characters.

Here my answer to the questions which arise so far.

Why do you introduce the module Characters?
Because I would like to merge the east and west community of BlackBox. Without Cyrillic character identifier we divide the community. Furthermore it provides an easy migration from BB 1.5 Russian version to BB 1.7 CPC Edition.

Why do you not insert its procedures into module Strings?
Module Character has a private interface, it is only used internally. I recommend to use module Strings for your application programs. It is a create advantage of BlackBox to have the complete task in a separated module.

Kernel is not the lowest-level module. This is kind of UGLY, isn't it?
Yes, Kernel imports from module Characters the procedures IsCap, Low, ShortIdent & LongIdent. You may add a hook for it. Anyway you must link it to have its functions.

Must Characters.IsIdentChar(ch) be tuned for better efficiency?
No, it is already tuned. I do not use the full Unicode implementation. Only the part which we need is implemented. I redesign the interface for optimisation use.

Is it waste of time to add module characters?
Yes, for me it is. I don't use Cyrillic letters in my programs. I need someone who will test my latest changes and reports me all problems. Is there someone who would like to do it?

What is new in the latest implementation?
I improve the use of Cyrillic letters. The new version reports the loaded modules in the correct encoding. Also the Trap Windows uses the correct encoding. This changes works. I also can create Cyrillic file names. But this future is currently disabled, because it is buggy.

Is the lastest implementation published?
No, because it is not tested yet.

Where can I get the latest implementation?
Send me an e-mail to feedback at zinnamturm dot eu and I will send you a download link.

[Josef Templ]

> Why do you not insert its procedures into module Strings?
> Module Character has a private interface, it is only used internally. I recommend to use module Strings for your application programs. It is a create advantage of BlackBox to have the complete task in a separated module.

Module Characters does not have a private interface because it exports and documents useful features.
The features are used and required in many modules and cannot be regarded as private.

The module Strings already provides Upper and Lower and it is obvious that these procedures overlap
with Characters.Cap / Low. Currently Upper and Lower are limited to Latin-1 characters.
So it is an obvious step to generalize them to support the full Unicode by means of
additional Kernel functions that use the corresponding WinApi functions for that purpose.
Thereby we would avoid the limitation to Cyrillic and Greek and add a truly useful feature
and one more step towards full Unicode support at a very low price.
If for other functions the limitation to Cyrillic and Greek is necessary because there
are no Windows-Api functions (which should be checked first!), so be it. This is a technical problem
and no need for a new lowest level module.

In general, I should add, I am in favour of supporting Cyrillic and Greek code pages
for the reason Helmut pointed out, viz. to unify the east and west distributions.
When we can at the same time add more Unicode support to Strings, so what is the problem?

- Josef