Page 1 of 3

issue-#19 Externalize & Internalize

Posted: Wed Jul 08, 2015 2:53 am
by Ivan Denisov
I found, that Externalize & Internalize are not working correctly for Views and Models from modules with Cyrillic identifiers.

Re: issue-#19 Externalize & Internalize

Posted: Wed Jul 08, 2015 9:20 am
by Zinn
Can you send me an example with a description of how to produced the error?

Re: issue-#19 Externalize & Internalize

Posted: Wed Jul 08, 2015 2:00 pm
by Josef Templ
The bug is in the usage of ReadXString and WriteXString in module Stores.
Non-ASCII characters in a type name are thereby truncated to ASCII.
We have to use Utf8 conversion, I think.

- Josef

Re: issue-#19 Externalize & Internalize

Posted: Wed Jul 08, 2015 2:25 pm
by Ivan Denisov
Zinn wrote:Can you send me an example with a description of how to produced the error?
Compile this module, call commander bellow, save this view in a separate window and then try to open.

Code: Select all

MODULE ПроверкаОтображения;

IMPORT Views, Log, Stores, Ports;

TYPE
	Отображение = POINTER TO RECORD (Views.View) END;
	
	
	PROCEDURE (v: Отображение) Restore (f: Views.Frame; l, t, r, b: INTEGER);
	BEGIN
		f.DrawLine(l, t, r, b, 1, Ports.red)
	END Restore;
	
	PROCEDURE Deposit*;
	VAR v: Отображение;
	BEGIN
		NEW(v);
		Views.Deposit(v)
	END Deposit;

END ПроверкаОтображения.


^Q "ПроверкаОтображения.Deposit; StdCmds.PasteView"



Re: issue-#19 Externalize & Internalize

Posted: Wed Jul 08, 2015 2:26 pm
by Ivan Denisov
Josef Templ wrote:The bug is in the usage of ReadXString and WriteXString in module Stores.
Non-ASCII characters in a type name are thereby truncated to ASCII.
We have to use Utf8 conversion, I think.
Agree!

Re: issue-#19 Externalize & Internalize

Posted: Thu Jul 09, 2015 9:11 am
by Josef Templ
The missing Utf-8 conversions have been added to Stores.
See the diffs at http://redmine.blackboxframework.org/pr ... 19cd8cc644.

- Josef

Re: issue-#19 Externalize & Internalize

Posted: Thu Jul 09, 2015 3:51 pm
by Ivan Denisov
Josef Templ wrote:The missing Utf-8 conversions have been added to Stores.
See the diffs at http://redmine.blackboxframework.org/pr ... 19cd8cc644.

- Josef
I tried test version:
http://blackboxframework.org/unstable/i ... a1.212.zip

Partially this solves the problem. Now the views with Unicode identifiers can be saved and restored if module name is ASCII. However, if module name is Unicode also, the problem kept...

Re: issue-#19 Externalize & Internalize

Posted: Fri Jul 10, 2015 10:10 am
by Josef Templ
I see. There is an inappropriate SHORT left in Stores.ThisType,
which truncates Unicode characters in module names to ASCII.

Fixed.
See http://redmine.blackboxframework.org/pr ... 7724be721d.

- Josef

Re: issue-#19 Externalize & Internalize

Posted: Fri Jul 10, 2015 11:12 am
by Ivan Denisov
Thank you, Josef!
Now it works well.

I tested this version
http://blackboxframework.org/unstable/i ... a1.213.zip
with the example above.

Dear, Helmut, can you please create the voting for including this bug fix in master.

Re: issue-#19 Externalize & Internalize

Posted: Fri Jul 10, 2015 12:33 pm
by luowy
Josef Templ wrote:The missing Utf-8 conversions have been added to Stores.
and the TextModels.Attributes.Internalize/Externalize need Utf-8 conversions in ReadXString/WriteXString for the font.typeface also.

Code: Select all

	PROCEDURE (attr: Attributes) Internalize- (VAR rd: Stores.Reader), EXTENSIBLE;
	(** pre: ~attr.init **)
	(** post: attr.init **)
		VAR 
			thisVersion: INTEGER;
			fprint: INTEGER; 
			face: Fonts.Typeface; 
			size: INTEGER; 
			style: SET; 
			weight: INTEGER;
			(*>>*)VAR buf:ARRAY 64 OF SHORTCHAR;res:INTEGER;(*<<*)
	BEGIN
		ASSERT(~attr.init, 20); 
		attr.init := TRUE;
		attr.Internalize^(rd);
		IF rd.cancelled THEN RETURN END;
		rd.ReadVersion(minVersion, maxAttrVersion, thisVersion);
		IF rd.cancelled THEN RETURN END;
		rd.ReadInt(attr.color);
		rd.ReadInt(fprint);
		rd.ReadXString(face);  (*>>*)buf:=SHORT(face$);Kernel.Utf8ToString(buf,face,res);(*<<*)
		rd.ReadInt(size); rd.ReadSet(style); rd.ReadXInt(weight);
		attr.font := Fonts.dir.This(face, size, style, weight);
		IF attr.font.IsAlien() THEN Stores.Report("#System:AlienFont", face, "", "")
(*
		ELSIF attr.font.Fingerprint() # fprint THEN Stores.Report("#System:AlienFontVersion", face, "", "")
*)
		END;
		rd.ReadInt(attr.offset)
	END Internalize;

	PROCEDURE (attr: Attributes) Externalize- (VAR wr: Stores.Writer), EXTENSIBLE;
	(** pre: attr.init **)
		VAR font: Fonts.Font;
		(*>>*)VAR buf:ARRAY 64 OF SHORTCHAR;res:INTEGER;(*<<*)
	BEGIN
		ASSERT(attr.init, 20);
		attr.Externalize^(wr);
		wr.WriteVersion(maxAttrVersion);
		wr.WriteInt(attr.color);
		font := attr.font;
(*
		wr.WriteInt(font.Fingerprint());
*)
		wr.WriteInt(0);
		(*>>*)Kernel.StringToUtf8(font.typeface,buf,res);(*<<*)
		wr.WriteXString((*>>*)LONG(buf)(*font.typeface*)(*<<*)); 
		wr.WriteInt(font.size); wr.WriteSet(font.style); wr.WriteXInt(font.weight);
		wr.WriteInt(attr.offset)
	END Externalize;
luowy