issue-#19 Externalize & Internalize

[Ivan Denisov]

I found, that Externalize & Internalize are not working correctly for Views and Models from modules with Cyrillic identifiers.

[Zinn]

Can you send me an example with a description of how to produced the error?

[Josef Templ]

The bug is in the usage of ReadXString and WriteXString in module Stores.
Non-ASCII characters in a type name are thereby truncated to ASCII.
We have to use Utf8 conversion, I think.

- Josef

[Ivan Denisov]

Can you send me an example with a description of how to produced the error?

Compile this module, call commander bellow, save this view in a separate window and then try to open.

Code: Select all

MODULE ПроверкаОтображения;

IMPORT Views, Log, Stores, Ports;

TYPE
	Отображение = POINTER TO RECORD (Views.View) END;
	
	
	PROCEDURE (v: Отображение) Restore (f: Views.Frame; l, t, r, b: INTEGER);
	BEGIN
		f.DrawLine(l, t, r, b, 1, Ports.red)
	END Restore;
	
	PROCEDURE Deposit*;
	VAR v: Отображение;
	BEGIN
		NEW(v);
		Views.Deposit(v)
	END Deposit;

END ПроверкаОтображения.


^Q "ПроверкаОтображения.Deposit; StdCmds.PasteView"

[Ivan Denisov]

The bug is in the usage of ReadXString and WriteXString in module Stores.
Non-ASCII characters in a type name are thereby truncated to ASCII.
We have to use Utf8 conversion, I think.

Agree!

[Josef Templ]

The missing Utf-8 conversions have been added to Stores.
See the diffs at http://redmine.blackboxframework.org/pr ... 19cd8cc644.

- Josef

[Ivan Denisov]

The missing Utf-8 conversions have been added to Stores.
See the diffs at http://redmine.blackboxframework.org/pr ... 19cd8cc644.

- Josef

I tried test version:
http://blackboxframework.org/unstable/i ... a1.212.zip

Partially this solves the problem. Now the views with Unicode identifiers can be saved and restored if module name is ASCII. However, if module name is Unicode also, the problem kept...

[Josef Templ]

I see. There is an inappropriate SHORT left in Stores.ThisType,
which truncates Unicode characters in module names to ASCII.

Fixed.
See http://redmine.blackboxframework.org/pr ... 7724be721d.

- Josef

[Ivan Denisov]

Thank you, Josef!
Now it works well.

I tested this version
http://blackboxframework.org/unstable/i ... a1.213.zip
with the example above.

Dear, Helmut, can you please create the voting for including this bug fix in master.

[luowy]

The missing Utf-8 conversions have been added to Stores.

and the TextModels.Attributes.Internalize/Externalize need Utf-8 conversions in ReadXString/WriteXString for the font.typeface also.

Code: Select all

	PROCEDURE (attr: Attributes) Internalize- (VAR rd: Stores.Reader), EXTENSIBLE;
	(** pre: ~attr.init **)
	(** post: attr.init **)
		VAR 
			thisVersion: INTEGER;
			fprint: INTEGER; 
			face: Fonts.Typeface; 
			size: INTEGER; 
			style: SET; 
			weight: INTEGER;
			(*>>*)VAR buf:ARRAY 64 OF SHORTCHAR;res:INTEGER;(*<<*)
	BEGIN
		ASSERT(~attr.init, 20); 
		attr.init := TRUE;
		attr.Internalize^(rd);
		IF rd.cancelled THEN RETURN END;
		rd.ReadVersion(minVersion, maxAttrVersion, thisVersion);
		IF rd.cancelled THEN RETURN END;
		rd.ReadInt(attr.color);
		rd.ReadInt(fprint);
		rd.ReadXString(face);  (*>>*)buf:=SHORT(face$);Kernel.Utf8ToString(buf,face,res);(*<<*)
		rd.ReadInt(size); rd.ReadSet(style); rd.ReadXInt(weight);
		attr.font := Fonts.dir.This(face, size, style, weight);
		IF attr.font.IsAlien() THEN Stores.Report("#System:AlienFont", face, "", "")
(*
		ELSIF attr.font.Fingerprint() # fprint THEN Stores.Report("#System:AlienFontVersion", face, "", "")
*)
		END;
		rd.ReadInt(attr.offset)
	END Internalize;

	PROCEDURE (attr: Attributes) Externalize- (VAR wr: Stores.Writer), EXTENSIBLE;
	(** pre: attr.init **)
		VAR font: Fonts.Font;
		(*>>*)VAR buf:ARRAY 64 OF SHORTCHAR;res:INTEGER;(*<<*)
	BEGIN
		ASSERT(attr.init, 20);
		attr.Externalize^(wr);
		wr.WriteVersion(maxAttrVersion);
		wr.WriteInt(attr.color);
		font := attr.font;
(*
		wr.WriteInt(font.Fingerprint());
*)
		wr.WriteInt(0);
		(*>>*)Kernel.StringToUtf8(font.typeface,buf,res);(*<<*)
		wr.WriteXString((*>>*)LONG(buf)(*font.typeface*)(*<<*)); 
		wr.WriteInt(font.size); wr.WriteSet(font.style); wr.WriteXInt(font.weight);
		wr.WriteInt(attr.offset)
	END Externalize;

luowy