Encoding problem: utf8 not displayed correctly for properties

[jkuehn]

Hi guys,
its driving my crazy for hours, i just cant figure out a solution:

I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag, then the
output is messed up: Nächste becomes Nächste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??

Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"Nächste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "Nächste" is
pumped out. Why??

The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.

I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8

What makes Grails/Spring think that the messages_de.properties is NOT utf8?

Any help is very appreciated!

[j pimmel]

I think you need to put the unicode character code in the message properties
file, to ensure file eonciding of the saved file doesnt affect output
character
\u0308

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag, then the
output is messed up: NÀchste becomes NÀchste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"NÀchste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "NÀchste" is
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?
Any help is very appreciated!
--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

[Mohamed]

If you save the file, in some other program this can happen. I would suggest
you create a new app, take that properties file, copy the content your file
into that one, and save. Then replace yours with that one.

This happens to me when I save outside intellij.

Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag, then the
output is messed up: Nächste becomes NÀchste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"Nächste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "NÀchste" is
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?
Any help is very appreciated!
--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

[j pimmel]

yes you are right.
we always had this when some developers using Linux saved files with special
characters and then windows people used the same files..
using unicode in message.properties throughout was the way we found
prevented the different file encodings tripping people up

Post by Mohamed
If you save the file, in some other program this can happen. I would
suggest you create a new app, take that properties file, copy the content
your file into that one, and save. Then replace yours with that one.
This happens to me when I save outside intellij.
Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag, then the
output is messed up: NÀchste becomes NÀchste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"NÀchste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "NÀchste" is
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?
Any help is very appreciated!
--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

[jkuehn]

Hi Moe, hi pimmel, thank you for the quick reply.
If i copy messages_de.properties from the fresh project and simply overwrite
mine, then the result doesnt change. So it doesnt seem to be an encoding
issue of the gsp or the properties file.

I also made a diff on both projects, besides the controllers and domains and
DataSource there are no differences.

Then i compared the generated directories in ~/.grails
fresh project:
default.paginate.next=N\u00e4chste
my project:
default.paginate.next=Nächste

So my properties arent escaped to unicode chars properly if i call grails
run-app (both dev and prod). But why not? I dont want to enter the chars as
\u0308 :)

I guess i copy all my stuff to a new project, that would be the quickiest
solution. But would really like to know what happens there.

Greetings,
Juri

Post by Mohamed
If you save the file, in some other program this can happen. I would suggest
you create a new app, take that properties file, copy the content your file
into that one, and save. Then replace yours with that one.
This happens to me when I save outside intellij.
Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag, then the
output is messed up: Nächste becomes Nächste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"Nächste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "Nächste" is
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?
Any help is very appreciated!
--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

[j pimmel]

Well my guess is that the file encoding of the grails default (in fresh
project) are of one encoding and those which get created locally another
one.. so if you copy the file, the encoding gets copied, but if you create a
new file and it adopts your OS encoding... ?
I haven't ever found good tools to show encodings.

You can convert to unix format (probably the one you want) using dos2unix if
that is available on your OS..

Post by jkuehn
Hi Moe, hi pimmel, thank you for the quick reply.
If i copy messages_de.properties from the fresh project and simply overwrite
mine, then the result doesnt change. So it doesnt seem to be an encoding
issue of the gsp or the properties file.
I also made a diff on both projects, besides the controllers and domains and
DataSource there are no differences.
Then i compared the generated directories in ~/.grails
default.paginate.next=N\u00e4chste
default.paginate.next=NÀchste
So my properties arent escaped to unicode chars properly if i call grails
run-app (both dev and prod). But why not? I dont want to enter the chars as
\u0308 :)
I guess i copy all my stuff to a new project, that would be the quickiest
solution. But would really like to know what happens there.
Greetings,
Juri

Post by Mohamed
If you save the file, in some other program this can happen. I would suggest
you create a new app, take that properties file, copy the content your file
into that one, and save. Then replace yours with that one.
This happens to me when I save outside intellij.
Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag, then the
output is messed up: NÀchste becomes NÀchste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"NÀchste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "NÀchste" is
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?
Any help is very appreciated!
--

http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html

Post by Mohamed

Post by jkuehn
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20741269.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

[tramuntanal]

i think that dos2unix only removes \r (carriage returns) chars from
files but the encoding of the file stays the same.

Juri if you are shure you are encoding your file as utf8 maybe you
have changed spring's encoding filter configuration and your browser
doesn't know the encoding of the pages. Is it possible?

regards

Post by j pimmel
Well my guess is that the file encoding of the grails default (in fresh
project) are of one encoding and those which get created locally another
one.. so if you copy the file, the encoding gets copied, but if you create a
new file and it adopts your OS encoding... ?
I haven't ever found good tools to show encodings.
You can convert to unix format (probably the one you want) using dos2unix if
that is available on your OS..

Post by jkuehn
Hi Moe, hi pimmel, thank you for the quick reply.
If i copy messages_de.properties from the fresh project and simply overwrite
mine, then the result doesnt change. So it doesnt seem to be an encoding
issue of the gsp or the properties file.
I also made a diff on both projects, besides the controllers and domains and
DataSource there are no differences.
Then i compared the generated directories in ~/.grails
default.paginate.next=N\u00e4chste
default.paginate.next=Nächste
So my properties arent escaped to unicode chars properly if i call grails
run-app (both dev and prod). But why not? I dont want to enter the chars as
\u0308 :)
I guess i copy all my stuff to a new project, that would be the quickiest
solution. But would really like to know what happens there.
Greetings,
Juri

Post by Mohamed
If you save the file, in some other program this can happen. I would suggest
you create a new app, take that properties file, copy the content your file
into that one, and save. Then replace yours with that one.
This happens to me when I save outside intellij.
Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag,
then
the
output is messed up: Nächste becomes Nächste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"Nächste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "Nächste" is
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?
Any help is very appreciated!
--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20741269.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[jkuehn]

I found the problem, finally. While trying to reproduce the error with newly
created apps i didnt delete all project files in
<home>/.grails/1.0.4/<project>, so not all of my changes took effect, which
led to confusig results while testing. But good to know now :)

In the end it was my silliness. All the file encodings were correct. I put a
new block for the mail plugin at the bottom of Config.groovy:
---
grails {
mail {
host = "***<value cut out>"
port = 25
username = "***"
password = "***"
}
}
---
What i didnt think of, is that it overwrites all grails.* settings, that
were set above. Which also overwrites the grails.enable.native2ascii = true
setting and grails assumes the default value of false
If grails.enable.native2ascii is set to true, then all property-strings are
converted to ascii and all special chars to unicode sequences \uxxxx, what
is my desired behaviour (http://grails.org/Config)

The solution was to put the mail config on top of Config.groovy, so that it
doesnt overwrite succeeding settings. Thats it :)

Took a long time to find out. I put a word about it on the config page.

Thank you for your hints, they were very helpful for digging in!

Greetings,
Juri

Post by tramuntanal
i think that dos2unix only removes \r (carriage returns) chars from
files but the encoding of the file stays the same.
Juri if you are shure you are encoding your file as utf8 maybe you
have changed spring's encoding filter configuration and your browser
doesn't know the encoding of the pages. Is it possible?
regards

Post by j pimmel
Well my guess is that the file encoding of the grails default (in fresh
project) are of one encoding and those which get created locally another
one.. so if you copy the file, the encoding gets copied, but if you create a
new file and it adopts your OS encoding... ?
I haven't ever found good tools to show encodings.
You can convert to unix format (probably the one you want) using dos2unix if
that is available on your OS..

Post by jkuehn
Hi Moe, hi pimmel, thank you for the quick reply.
If i copy messages_de.properties from the fresh project and simply overwrite
mine, then the result doesnt change. So it doesnt seem to be an encoding
issue of the gsp or the properties file.
I also made a diff on both projects, besides the controllers and domains and
DataSource there are no differences.
Then i compared the generated directories in ~/.grails
default.paginate.next=N\u00e4chste
default.paginate.next=Nächste
So my properties arent escaped to unicode chars properly if i call grails
run-app (both dev and prod). But why not? I dont want to enter the chars as
\u0308 :)
I guess i copy all my stuff to a new project, that would be the quickiest
solution. But would really like to know what happens there.
Greetings,
Juri

Post by Mohamed
If you save the file, in some other program this can happen. I would suggest
you create a new app, take that properties file, copy the content your file
into that one, and save. Then replace yours with that one.
This happens to me when I save outside intellij.
Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag,
then
the
output is messed up: Nächste becomes Nächste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of

web-app/index.gsp.

Post by Mohamed

Post by jkuehn
so
"Nächste" is displayed correctly for the german language. If i copy

the

Post by Mohamed

Post by jkuehn
whole file (not the contents, the file) into my app, then "Nächste"

is

Post by Mohamed

Post by jkuehn
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh

app

Post by Mohamed

Post by jkuehn
too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both

projects,

Post by Mohamed

Post by jkuehn
all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?
Any help is very appreciated!
--

http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html

Post by Mohamed

Post by jkuehn
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20741269.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

[Marc Palmer]

Post by jkuehn
I found the problem, finally. While trying to reproduce the error with newly
created apps i didnt delete all project files in
<home>/.grails/1.0.4/<project>, so not all of my changes took
effect, which
led to confusig results while testing. But good to know now :)
In the end it was my silliness. All the file encodings were correct. I put a
---
grails {
mail {
host = "***<value cut out>"
port = 25
username = "***"
password = "***"
}
}
---
What i didnt think of, is that it overwrites all grails.* settings, that
were set above. Which also overwrites the grails.enable.native2ascii = true
setting and grails assumes the default value of false
If grails.enable.native2ascii is set to true, then all property-
strings are
converted to ascii and all special chars to unicode sequences
\uxxxx, what
is my desired behaviour (http://grails.org/Config)
The solution was to put the mail config on top of Config.groovy, so that it
doesnt overwrite succeeding settings. Thats it :)
Took a long time to find out. I put a word about it on the config page.
Thank you for your hints, they were very helpful for digging in!

Can I just clarify this functionality again for everybody.

There are deficiencies in the way .properties files work (assume
ISO_8859-1, no way to indicate different encoding per properties file
etc).

As a result Grails does the only sane thing. It assumes ALL message
bundle .properties files (and all grails app source code and GSPs) are
in UTF-8. You -must- save all files as UTF-8, especially properties
files.

Then, for properties files only, it will run native2ascii at build
time to convert these into pure 7-bit ascii files with unicode
escapes, that Java will always load properly.

So all you have to do to have great i18n support is save as UTF-8
always. Nothing more.

Marc



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Mengu]

sorry for bumping a 2 years old thread but i didn't want to create a new
thread.

i have the same problem with turkish characters. i have created a
messages_tr.properties file and the characters are not shown properly.

- my java_home points to sun jdk.
- my file is saved as unicode text.
- my charset in pages is utf-8.

what would be causing this problem?

[Luke Daley]

Post by Mengu
sorry for bumping a 2 years old thread but i didn't want to create a new
thread.
i have the same problem with turkish characters. i have created a
messages_tr.properties file and the characters are not shown properly.
- my java_home points to sun jdk.
- my file is saved as unicode text.
- my charset in pages is utf-8.
what would be causing this problem?

Make sure your encoding config is correct:

http://grails.org/doc/latest/guide/3.%20Configuration.html#3.1.1%20Built%20in%20options
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Mengü Kağan]

hmm, thanks Luke. I remember commenting the native2ascii config line. i'll
check it out when i get back home and update here.

Post by Luke Daley

Post by Mengu
sorry for bumping a 2 years old thread but i didn't want to create a new
thread.
i have the same problem with turkish characters. i have created a
messages_tr.properties file and the characters are not shown properly.
- my java_home points to sun jdk.
- my file is saved as unicode text.
- my charset in pages is utf-8.
what would be causing this problem?

http://grails.org/doc/latest/guide/3.%20Configuration.html#3.1.1%20Built%20in%20options
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

[Mengu]

I wanted to post an update here.

I had native2ascii setting in my config set to false. I have changed it to
true and all my turkish characters are shown as expected.

thanks luke.

[Marc Paquette]

java.util.Properties is the one reading and writing in ISO 8859-1.
See javadoc for it.

It is interesting to read the javadoc for 1.4.2, 1.5.0 and 1.6.0 :

http://java.sun.com/j2se/1.4.2/docs/api/java/util/Properties.html
http://java.sun.com/j2se/1.5.0/docs/api/java/util/Properties.html
http://java.sun.com/javase/6/docs/api/java/util/Properties.html

As it says, native2ascii is the prefered way to deal with this.
dos2unix will fiddle with end-of-line character, which IS NOT what you
want :-0

As for tools to show encodings, I agree, it is always a hide-and-seek
game .... but you never get to win :-(

For a thorough treatment of the subject, I find http://nlp.cs.nyu.edu/nycnlp/nycnlp-crash-course-on-character-encodings.pdf
entertaining !

Cheers,
--
Marc Paquette
Marcpa Inc.
http://www.marcpa.ca/

Post by j pimmel
Well my guess is that the file encoding of the grails default (in
fresh project) are of one encoding and those which get created
locally another one.. so if you copy the file, the encoding gets
copied, but if you create a new file and it adopts your OS
encoding... ?
I haven't ever found good tools to show encodings.
You can convert to unix format (probably the one you want) using
dos2unix if that is available on your OS..
Hi Moe, hi pimmel, thank you for the quick reply.
If i copy messages_de.properties from the fresh project and simply overwrite
mine, then the result doesnt change. So it doesnt seem to be an encoding
issue of the gsp or the properties file.
I also made a diff on both projects, besides the controllers and domains and
DataSource there are no differences.
Then i compared the generated directories in ~/.grails
default.paginate.next=N\u00e4chste
default.paginate.next=Nächste
So my properties arent escaped to unicode chars properly if i call grails
run-app (both dev and prod). But why not? I dont want to enter the chars as
\u0308 :)
I guess i copy all my stuff to a new project, that would be the quickiest
solution. But would really like to know what happens there.
Greetings,
Juri

Post by Mohamed
If you save the file, in some other program this can happen. I would suggest
you create a new app, take that properties file, copy the content

your

Post by Mohamed
file
into that one, and save. Then replace yours with that one.
This happens to me when I save outside intellij.
Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message

tag, then

Post by Mohamed

Post by jkuehn
the
output is messed up: Nächste becomes Nächste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is

iso??

Post by Mohamed

Post by jkuehn
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/

index.gsp.

Post by Mohamed

Post by jkuehn
so
"Nächste" is displayed correctly for the german language. If i

copy the

Post by Mohamed

Post by jkuehn
whole file (not the contents, the file) into my app, then "NÃ

¤chste" is

Post by Mohamed

Post by jkuehn
pumped out. Why??
The only plugin i installed is mail, so i installed it on the

fresh app

Post by Mohamed

Post by jkuehn
too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both

projects,

Post by Mohamed

Post by jkuehn
all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is

NOT

Post by Mohamed

Post by jkuehn
utf8?
Any help is very appreciated!
--
http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20740968.html
Sent from the grails - user mailing list archive at Nabble.com.

---------------------------------------------------------------------

Post by Mohamed

Post by jkuehn
http://xircles.codehaus.org/manage_email

--
View this message in context: http://www.nabble.com/Encoding-problem%3A-utf8-not-displayed-correctly-for-properties-tp20740968p20741269.html
Sent from the grails - user mailing list archive at Nabble.com.
---------------------------------------------------------------------
http://xircles.codehaus.org/manage_email

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[j pimmel]

As it says, native2ascii is the prefered way to deal with this. dos2unix
will fiddle with end-of-line character, which IS NOT what you want :-0

Oh yeah, thats the one.. got them confused.. Its been a while.. script
pre-processing was never regarded very highly for what we were doing so just
made sure we unicode escaped everything

[Stand Trooper]

While the javadoc spec has always been to use \u+0000 in your
properties file, it is and has always been, IMO, utterly stupid.
Grails does provide that you can use the RBs with native code and it
does the work on the backend.

Since Grails handles all of this for you there's no need to do
anything but add your keys and values to the proper RB. Now, if you
open those in windows with an IDE (such as IntelliJ) and your settings
have been changed, then there is a chance you'll muck up the file with
\u+0000 ascii encodings through the file. I always open mine in
IntelliJ or notepad on Windows systems and have never had a problem.
Moe, however, has issues with IntelliJ and I've known at least one
other person to have issues with Eclipse. The majority of people
don't encounter a problem. I have never had a problem in either case
(and I don't use Eclipse, so... .)

native2ascii has a -reverse option that converts \u+0000 to the native
version for easier reading.

the /.grails folder is where the files are generated from when you're
loading your project and running it... and those are correct with the
\u+0000 encoding. What you need to look at is the grails-app/i18n
folder and the RBs there. Those properties files are the ones to edit
and worry about. Grails will take those files and convert them with
native2ascii and store them in the .grails directory substructure
automatically. This is precisely because of the statements Marc
brought about. The Properties class will read in 8859_1.

Let me know if you're still having issues and I'll work through stuff
with you like I did Moe.

thx

timo

java.util.Properties is the one reading and writing in ISO 8859-1. See
javadoc for it.
http://java.sun.com/j2se/1.4.2/docs/api/java/util/Properties.html
http://java.sun.com/j2se/1.5.0/docs/api/java/util/Properties.html
http://java.sun.com/javase/6/docs/api/java/util/Properties.html
As it says, native2ascii is the prefered way to deal with this. dos2unix
will fiddle with end-of-line character, which IS NOT what you want :-0
As for tools to show encodings, I agree, it is always a hide-and-seek game
.... but you never get to win :-(
For a thorough treatment of the subject, I find
http://nlp.cs.nyu.edu/nycnlp/nycnlp-crash-course-on-character-encodings.pdf entertaining
!
Cheers,
--
Marc Paquette
Marcpa Inc.
http://www.marcpa.ca/

Post by j pimmel
Well my guess is that the file encoding of the grails default (in fresh
project) are of one encoding and those which get created locally another
one.. so if you copy the file, the encoding gets copied, but if you create a
new file and it adopts your OS encoding... ?
I haven't ever found good tools to show encodings.
You can convert to unix format (probably the one you want) using dos2unix
if that is available on your OS..
Hi Moe, hi pimmel, thank you for the quick reply.
If i copy messages_de.properties from the fresh project and simply overwrite
mine, then the result doesnt change. So it doesnt seem to be an encoding
issue of the gsp or the properties file.
I also made a diff on both projects, besides the controllers and domains and
DataSource there are no differences.
Then i compared the generated directories in ~/.grails
default.paginate.next=N\u00e4chste
default.paginate.next=Nächste
So my properties arent escaped to unicode chars properly if i call grails
run-app (both dev and prod). But why not? I dont want to enter the chars as
\u0308 :)
I guess i copy all my stuff to a new project, that would be the quickiest
solution. But would really like to know what happens there.
Greetings,
Juri

Post by Mohamed
If you save the file, in some other program this can happen. I would suggest
you create a new app, take that properties file, copy the content your file
into that one, and save. Then replace yours with that one.
This happens to me when I save outside intellij.
Moe

Post by jkuehn
Hi guys,
I am using german umlauts on my page. If i enter umlaut characters directly
in the gsp, they are shown correctly, but if i use the message tag,
then
the
output is messed up: Nächste becomes Nächste
If i save the properties file as ISO-8859-1 then the strings are displayed
correctly. Why and what component is assuming, that the file is iso??
Now the superweird thing: i created a fresh app and put <g:message
code="default.paginate.next" /> into the body tag of web-app/index.gsp. so
"Nächste" is displayed correctly for the german language. If i copy the
whole file (not the contents, the file) into my app, then "Nächste" is
pumped out. Why??
The only plugin i installed is mail, so i installed it on the fresh app too,
didnt change anything.
Also tried to set -Dfile.encoding="UTF-8" as JAVA_OPTS, no effect.
I didnt change any encoding settings on the project. In both projects, all
files are encoded in UTF8
What makes Grails/Spring think that the messages_de.properties is NOT utf8?