issue-#201 Importer and Exporter for UTF-8 texts

[luowy]

for views, no need; considering rtf2odc , a lot of rulers will be added, e.g. bolding a word text will add begin ruler and end ruler, if they convert them to lines, ....

for para,I rarely use it, not familiar with it.

[Josef Templ]

added as in other exporters:

Code: Select all

IF (ch # TextModels.viewcode) & (ch # TextModels.para) THEN

Improving the treatment of para or rulers is a cross cutting change that applies also to other exporters.
It is not a special problem of the utf-8 text exporter, so it is not optimized in any way.
This could be another issue, but so far nobody has ever found this to be a relevant topic.

For the diffs see https://redmine.blackboxframework.org/projects/blackbox/repository/diff?utf8=%E2%9C%93&rev=af22345d3c2859ccfe38057ffaa1549841f6448b&rev_to=2d2cb269c073b96429fd823701d764dcf5bb0d83.

- Josef

[Zinn]

added as in other exporters:

Code: Select all

IF (ch # TextModels.viewcode) & (ch # TextModels.para) THEN

Josef,
yes it is an other topic. To get the same behaviour as the text exporter we need this line.

But what happens without this line?
The function of para is kept. It is not lost. Do we need a translation of para to an empty line?
The viewcode is still lost. You see that their was a view (02X). It may be a problem to have the control character in the result.

Normally this question is not relevant, because you can edit the text before you save it as you would like to have.
- Helmut

[Josef Templ]

But what happens without this line?
The function of para is kept. It is not lost. Do we need a translation of para to an empty line?
The viewcode is still lost. You see that their was a view (02X). It may be a problem to have the control character in the result.

Normally this question is not relevant, because you can edit the text before you save it as you would like to have.
- Helmut

I assume this question refers to that other issue, i.e. changing the export of viewcode and para in ALL text exporters, right?

A simple strategy would be to treat para as newline and to treat rulers also as newline.
Other views would probably have to be ignored as it is done now.
Inserting viewcode into the output text is not OK, I think, because it is a very special control character
that only causes problems with text editors.

Regarding the usage of utf8 for xml and html importers is definitely worth looking at, I think.
I just tried to export a text with extended ASCII characters to xhtml and it ends up as utf8.
When opening it with the text importer, as it is the default now, the extended characters are garbage.
When opening it as utf8, everything works fine. So this would remove an asymmetry between exporting
and importing an html file.

I have changed this in the Config module.
For the changes see https://redmine.blackboxframework.org/projects/blackbox/repository/diff?utf8=%E2%9C%93&rev=013101735bc4376f4eaff6f6e1745e4c415d7e9d&rev_to=af22345d3c2859ccfe38057ffaa1549841f6448b.

- Josef

[Josef Templ]

If there are no more remarks I think we can vote about this issue.

Here is the link to all the changes:
https://redmine.blackboxframework.org/projects/blackbox/repository/diff?utf8=%E2%9C%93&rev=013101735bc4376f4eaff6f6e1745e4c415d7e9d&rev_to=cd3c020e77648cc9b37ad4cea7677cec172f4dc6

- Josef

[Zinn]

Thank you Josef, your last change in config is a create improvement.

A simple strategy would be to treat para as newline and to treat rulers also as newline.
Other views would probably have to be ignored as it is done now.
Inserting viewcode into the output text is not OK, I think, because it is a very special control character
that only causes problems with text editors.

Why don't you add this changes into the UTF8 Exporter? There is no reason to obey the rules of the txt Exporter.

- Helmut

[Josef Templ]

Thank you Josef, your last change in config is a create improvement.

Why don't you add this changes into the UTF8 Exporter? There is no reason to obey the rules of the txt Exporter.

- Helmut

The para/view/ruler treatment is an orthogonal issue that applies to all text exporters not only to utf8.
It would be a mixture of unrelated changes then in #201.

But there is no point against adding that other issue before we make the release.

- Josef