Return to
Globalization_HOWTO.htm
CVS log
Up to [Pegasus] / pegasus / doc
|
Return to
Globalization_HOWTO.htm
CVS log
|
Up to [Pegasus] / pegasus / doc
|
|
File: [Pegasus] / pegasus / doc / Globalization_HOWTO.htm
(download)
/
(as text)
Revision: 1.4, Mon Sep 8 20:32:02 2003 UTC (20 years, 7 months ago) by chuck Branch: MAIN CVS Tags: test, RELEASE_2_3_0-msg-freeze Changes since 1.3: +37 -12 lines IBM-CC, Bug 509, R2.3: Update Globalization HOWTO |
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Mozilla/4.78 [en] (X11; U; Linux 2.4.7-10 i686) [Netscape]">
</head>
<body text="#000000" bgcolor="#FFFFFF" link="#0000EF" vlink="#55188A" alink="#FF0000">
<center><font size=+4>Globalization HOWTO</font>
<p>Release: Pegasus 2.3
<p>Author: Chuck Carmack (carmack@us.ibm.com)
<p>July 28, 2003</center>
<p><br>
<h2>
1.0 Introduction</h2>
<p><br>As part of the Pegasus 2.3 release, functions were added for globalization
support. Globalization involves two major aspects: internationalization
and localization.
<br>
<p>Internationalization is the process of writing a program that is locale-neutral.
In other words, the program should be able to run in any locale without
change. There are several categories in a locale, including the language
of message strings, date format, time format, etc. For release 2.3,
the Pegasus server is concerned with the language of the message strings
it returns to its clients.
<br>
<p>To support internationalization, a program is designed to do the following:
<br>
<blockquote>
<li>
Support character sets that can represent customer data in any language.
Typically, the program supports some variation of Unicode for internal
data. There is usually some conversion between the supported character
sets for external data, and the internal character set. Since Unicode
covers all characters, and usually has converters on the platform, it is
a good choice for the 'normalized' internal character set.
The most 'interoperable' solution for external data is to support UTF-8
(eg. network and file system data). The internal data is usually
UTF-16 (or UCS-2, but that is deprecated).</li>
<br>
<li>
Extract locale-sensitive resources, such as message strings, from the code
to external resource files. Typically, the resources are loaded based
on the locale requested by the end-user, and returned to the end-user for
display.</li>
</blockquote>
<p><br>Localization is the process of customizing a software product to
support particular locales. For example, a product that is internationalized
might want to only localize for certain countries. This would mean
that the localized resources (eg. message files) would only be translated
and shipped for the countries that the product supports. Since the
code for the product is locale-neutral, it will be easy to drop in new
translations as more countries are supported.
<br>
<p>The Pegasus 2.3 release added support for globalization. At a
high-level, the following additions were made to Pegasus 2.3:
<br>
<ul>
<li>
Support UTF-8 for external data.</li>
<br>
<ul>
<li>
The CIM-XML documents contained in the HTTP messages</li>
<li>
The files in the repository</li>
<li>
Note: Pegasus 2.3 does NOT support UTF-8 in the MOF files</li>
<br> </ul>
<li>
Support UTF-16 for internal data.</li>
<br>
<li>
Extract the hardcoded messages from the Pegasus code into message files.
An API was added to load messages from the message files.</li>
<br>
<li>
APIs were added for clients to associate a language with the CIM objects
they are sending to Pegasus. Also, APIs were added for clients to
determine the language of the error message or CIM object that Pegasus
returns.</li>
<br>
<li>
APIs were added for providers to determine the language of CIM objects
sent by the client. Also, APIs were added for providers to associate
a language with the CIM object, or error message, they return to the client.</li>
</ul>
<p><br>Please refer to PEPs 56 and 58 for details about the globalization
design in Pegasus 2.3.
<br>
<p>This document provides a HOWTO guide to be used by developers to globalize
code that is being added to Pegasus. The audience for this document
are:
<br>
<ul>
<li>
Provider developers - both CMPI and C++</li>
<li>
Client developers</li>
<li>
Pegasus developers</li>
</ul>
<p><br>The quickest way to approach this document is to read the General
section, and then the developer section that relates to what you are doing.
<br>
<h2>
2.0 General</h2>
<h3>
2.1 Unicode Support</h3>
<p><br>Pegasus 2.3 supports Unicode throughout the processing of requests.
External data to Pegasus is encoded in UTF-8. Internal data is encoded
in UTF-16.
<br>
<p>UTF-8 support for external data includes the CIM-XML messages passed
over the network, and the repository files. Note: UTF-8 support
was NOT added to the MOF Compiler for MOF files in release 2.3.
For the CIM-XML messages, Pegasus follows section 4.8 of the <a href="http://www.dmtf.org/standards/documents/WBEM/DSP200.html">CIM-HTTP
specification</a> Specifically, Pegasus supports the
"utf-8" setting for the charset parameter of the Content-Type header and
the XML encoding attribute. If no charset is specified, the 7-bit
ASCII is assumed.
<br>
<p>The internal support of UTF-16 is encapsulated in the Pegasus String
class. This class has been updated to contain UTF-16 characters.
Specifically, the Char16 objects inside the String contain UTF-16 characters.
Note: a UTF-16 surrogate pair is contained in two consecutive Char16 objects.
To keep backwards compatibilty, the methods on the String class have not
changed. New methods have been added as needed. The following
describes this in more detail:
<ul>
<li>
The Pegasus 2.2 methods that take a char *, or return char *, are unchanged.
Code written to Pegasus 2.2 may have expected to store 8-bit ASCII (ISO-8859-1)
characters into String. These methods will convert the input to UTF-16
from 8-bit ASCII. (This is simple because UTF-16 is a superset of
8-bit ASCII - simply need to prepend '\0' to each char). The Pegasus
2.2 methods that return char data will attempt to convert from the UTF-16
internal representation to 8-bit ASCII. Characters that cannot be
converted will be replaced with a substitution character.</li>
<br>
<li>
All methods that take or return Char16 data are unchanged. The String
class now supports UTF-16 data in Char16, although surrogate pairs will
require two consecutive Char16 objects. The String class does NO
checking for unmatched surrogate pairs.</li>
<br>
<li>
New methods have been added to take and return UTF-8 data. The String
class will convert between UTF-8 and the UTF-16 internal representation
as needed. These new methods will use char * parameters, but will
be clearly labelled as UTF-8 methods.</li>
<br> </ul>
PROGRAMMING NOTE: Putting EBCDIC data into the String class is dangerous.
The String class is designed for UTF-16, which is a superset of 8-bit ASCII.
Any String object containing EBCDIC data will not work if it is used by
Pegasus to read or write data from external sources, such as the network
or repository files. In other words, any String containing EBCDIC
data should not leave the code using it.
<br>
<br>
<h3>
2.2 Localization Support</h3>
<h4>
2.2.1 Language Headers</h4>
<p><br>Pegasus 2.3 supports clients and providers that wish to localize.
There are two areas to be localized: <a href="http://www.dmtf.org/standards/documents/WBEM/DSP201.html#SecERROR">ERROR</a>
elements in the CIM-XML; and <a href="http://www.dmtf.org/standards/documents/WBEM/DSP201.html#SecObjectDefinitionElements">Object
Definition</a> elements in the CIM-XML. Clients can request
the server to return error messages and CIM objects in a set of languages
of their choosing. Clients can also tag a language to the CIM objects
they are sending to the server. Providers and the server can return
error messages and CIM objects that are tagged with one of languages
requested by the client.
<br>
<p>The localization design is based on section 4.8 of the <a href="http://www.dmtf.org/standards/documents/WBEM/DSP200.html">CIM-HTTP
specification</a> , which refers to <a href="http://www.ietf.org/rfc/rfc2616.txt?number=2616">RFC
2616</a>. The method used to tag a language to the CIM-XML is through
the Accept-Language and Content-Language HTTP headers. These headers
are basically lists of language tags. An HTTP request can contain
an Accept-Language header, which indicates the list of preferred languages
that the client wants in the response. This list can be prioritized
by using the quality numbers. An HTTP request or response can contain
a Content-Language header, which indicates the language(s) of the content
in the message. In the Pegasus case, this would be the CIM-XML.
Note that the Content-Language header is a list of language tags.
This allows the content of an HTTP message to contain more than one translation.
However, in the Pegasus case, there is only one CIM-XML document in the
HTTP message, and thus one translation.
<br>
<p>CIM clients may use the Accept-Language HTTP header to specify the languages
they wish to be returned in the CIM response message. CIM clients
may also use the Content-Language header to tag the language of any CIM
objects they are sending to the server in the CIM request message.
The server, and providers, should attempt to return error messages and
CIM objects in one of the accept languages requested by the client.
The server and providers should set the Content-Language header in the
CIM response message to indicate which of the requested languages they
are returning.
<br>
<p>NOTE: Localization support was not added for the MOF files and
repository in Pegasus 2.3. The #pragma locale, #pragma instancelocale,
and translatable qualifier flavor are not supported in the Pegasus 2.3
MOF compiler. From the client perspective, classes, qualifiers, and
instances stored in the repository are not tagged with a language.
The Accept-Language and Content-Language headers will be ignored for repository
operations. However, since the repository will support UTF-8,
characters for any language may be stored there.
<br>
<p>NOTE: Since the Content-Language header applies to the entire
HTTP message, it applies to the entire CIM-XML document. This includes
all the objects in the document, including enumerated objects, and all
the values in the objects. This is a limitation that will remain
until the CIM standard has been updated to support language tags tied to
individual CIM values. From the client perspective, it is possible
for Pegasus to send a CIM response with NO Content-Language, even if the
client had sent Accept-Language. This can happen if Pegasus
does not know the language of the response. An example is a request
that was sent to a Pegasus 2.2 provider. Another example is an enumerated
response where each provider returned a different language. Please
refer to PEP58 for details on these provider scenarios.
<br>
<p>Pegasus 2.3 has added classes for the localization support. There
are new classes called AcceptLanguages and ContentLanguages that encapsulate
the Accept-Language and Content-Language headers, respectively. These
classes are basically containers of AcceptLanguageElement and ContentLanguageElement,
where a language element represents one language tag. The AcceptLanguages
class will keep the AcceptLanguageElement's prioritized based on quality,
according to RFC 2616.
<br>
<p>AcceptLanguages and ContentLanguages are the objects used by code throughout
the request/response processing, from the client to the server to the providers
and back. The server handles the creation of these objects from the
HTTP headers. Code at each point in the process will have access
to these objects.
<br>
<p>Please refer to the following files for details on the new Pegasus classes.
<br>
<ul>
<li>
pegasus/src/Pegasus/Common/AcceptLanguages.h</li>
<li>
pegasus/src/Pegasus/Common/AcceptLanguageElement.h</li>
<li>
pegasus/src/Pegasus/Common/ContentLanguages.h</li>
<li>
pegasus/src/Pegasus/Common/ContentLanguageElement.h</li>
<li>
pegasus/src/Pegasus/Common/LanguageElementContainer.h</li>
<li>
pegasus/src/Pegasus/Common/LanguageElement.h</li>
</ul>
<p><br>See the sections below for details on how to write clients and providers
to use these classes.
<br>
<br>
<h4>
2.2.2 Message Bundles</h4>
<p><br>One of the goals of globalization for Pegasus 2.3 is the extraction
of hardcoded messages into translated message files, loading translated
messages from those files, and returning those messages to the client.
The topics to be discussed here are: how to create message files,
how to compile message files, and how to load messages into Pegasus.
<br>
<p>At the time of writing, the message loading function in Pegasus 2.3
used the International Components for Unicode (<a href="http://oss.software.ibm.com/icu">ICU)</a>
libraries. This is expected to be the future direction for Pegasus.
<a href="http://oss.software.ibm.com/icu">ICU
</a>uses
a resource bundle format for their message files. In order
to load the messages, ICU requires that the resource bundles are compiled
into a binary form (.res file) using their genrb tool.
<br>
<p>Platform Maintainers Note: Please refer to PEP 58 for information
about how to build Pegasus to use the ICU libraries.
<br>
<p>The documentation for ICU resource bundles is in the <a href="http://oss.software.ibm.com/icu/userguide/ResourceManagement.html">Resource
Management</a> section of the <a href="http://oss.software.ibm.com/icu/userguide/">ICU
User Guide</a> . This section will tell you how to
<br>create and organize your resource bundles for different languages.
Note: your resource bundles should be organized in a tree structure
similiar to the one shown in the Resource Management section, including
the empty bundles in the tree. It is recommended that you ship a
root resource bundle to be used as the fallback in case the client requests
a language that you are not supporting. The Pegasus make files are
set up to automatically create and compile a root resource bundle for you.
For Pegasus 2.3, the make will use your "en" bundle, upper case all the
messages, and then put the uppercased messages into the root bundle.
The uppercasing of the messages is necessary to create a "fallback" root
bundle that contains invariant characters across all EBCIC and ASCII codepages.
<br>
<p>NOTE: When creating your resource bundles, the name of the table
resource should contain the package name. For example,
if you have a bundle with a package name of "xyz", then the "en" bundle
should start like this:
<p>xyz_en:table {
<br>..... messages here
<br>
}
<br>
<p><i>not</i> like this:
<p>en:table {
<br>..... messages here
<br>
}
<p>This is different than some of the examples in the <a href="http://oss.software.ibm.com/icu/userguide/ResourceManagement.html">Resource
Management</a> section, but is needed because the -p option is not used
on genrb by the make files.
<br>
<p>NOTE: Pegasus 2.3 only supports simple string resources in the
ICU resource bundles. String resources may only be loaded by key.
Tables, arrays, and other complex resource types, are not supported.
<br>
<p>In order to compile your resource bundles, support has been added to
the Pegasus make files to run genrb. A new make target, "messages",
has been added that will call genrb and put the compiled bundles (.res)
in a directory of your choosing. An example of ICU resource bundles
and the make files to compile them are located in:
<br>
<ul>
<li>
pegasus/src/Providers/sample/LocalizedProvider/Makefile (just causes the
make to recurse to the msg sub-directory)</li>
<li>
pegasus/src/Providers/sample/LocalizedProvider/msg/Makefile (compiles the
bundles in the msg/ directory)</li>
<li>
pegasus/src/Providers/sample/LocalizedProvider/msg/*.txt (the resource
bundles to compile, using the recommended ICU language tree structure)</li>
</ul>
<p><br>NOTE: At the time of writing, only the Linux make files have
been updated to compile ICU resource bundles.
<br>
<p>It is important to place the compiled resource bundles in a directory
where your code can find them . The make files above compile the
resource bundles into $PEGASUS_HOME/msg/provider/localizedProvider.
The code that loads these messages uses the MessageLoader class (next section)
to load messages from this directory.
<br>
<br>
<h4>
2.2.3 Message Loading</h4>
<p><br>Code that needs to load a message in Pegasus does not call ICU directly.
Two message loading classes were added for Pegasus 2.3: MessageLoader
and MessageLoaderParms. These classes are abstractions designed to
hide of the actual loader used (but note that at the time of writing, only
ICU is supported). The MessageLoader is used to load a message
using a list of preferrred languages. The parameters to MessageLoader
are encapsulated in a MessageLoaderParms object.
<br>
<p>The MessageLoader is the place where the Accept-Language header, Content-Language
header, and the ICU resource bundles, join up. The MessageLoader
class is designed to receive an AcceptLanguages object, and a set of parameters
indicating the bundle base-name and message ID to use. The AcceptLanguages
object contains the list of requested languages sent by the client.
The MessageLoader searches for the message in the set of bundles named
with the base-name, using the AcceptLanguages for the list of specific
translated bundles to search. The MessageLoader returns the message
that it found, along with a ContentLanguages object indicating the language
of the message. The ContentLanguages object should be used to indicate
the language of the response sent back to the client.
<br>
<p>The MessageLoaderParms object contains the parameters to load the message.
There are many parameters, but many can be allowed to default. Here
is a description of the parameters:
<br>
<br>
<table BORDER COLS=3 WIDTH="100%" NOSAVE >
<tr>
<td>String msg_id; </td>
<td>Input.
<br>Required.</td>
<td>Message ID of the message to load from the resource bundle.
This is the key that ICU will use to load the message.</td>
</tr>
<tr>
<td>String default_msg;</td>
<td>Input.
<br>Required</td>
<td>Message to return if the no message can be loaded for msg_id from any
resource bundle. Note: The args parameters below are substituted
into this string.
<br>Note: For the args into this string, use the Pegasus '$'
form, as described in pegasus/src/Pegasus/Common/Formatter.h. Don't
use the ICU substitution format for the default message string.</td>
</tr>
<tr>
<td>String msg_src_path; </td>
<td>Input.
<br>Optional
<br>Default: $PEGASUS_HOME/msg/pegasus/pegasusServer</td>
<td>Path to the root resource bundle file which contains the msg_id.
<br>Note: Only specify the path down to the bundle base-name. Do
not append a language tag, such as "_root" or "_en". Do not append
a file extension.
<br>Note: relative paths start at $PEGASUS_HOME/msg.
<br>Note: defaults to the bundle containing the Pegasus server messages.</td>
</tr>
<tr>
<td>AcceptLanguages acceptlanguages;</td>
<td>Input.
<br>Optional
<br>Default: AcceptLanguages::EMPTY</td>
<td>Contains the list of preferred languages, in priority order.
This is combined with msg_src_path to determine which resource bundles
to search for for the msg_id. If not empty, overrides useThreadLocale
and useProcessLocale.</td>
</tr>
<tr>
<td>ContentLanguages contentlanguages;</td>
<td>Output</td>
<td>Contains the language that MessageLoader found for the msg_id. </td>
</tr>
<tr>
<td>Boolean useProcessLocale;</td>
<td>Input
<br>Optional
<br>Default = false</td>
<td>If true, MessageLoader will use the default locale of the process.
If true, overrides useThreadLocale.</td>
</tr>
<tr>
<td>Boolean useThreadLocale;</td>
<td>Input
<br>Optional
<br>Default = <font color="#FF0000">true</font></td>
<td>If true, MessageLoader will use the AcceptLanguages set by Pegasus
into the caller's Thread. See the Note below for details. </td>
</tr>
<tr>
<td>Boolean useICUfallback</td>
<td>Input
<br>Optional
<br>Default = false</td>
<td>If true, use ICU's fallback mechnism to search more general resource
bundles if the msg_id cannot be found. Note: the recommended setting
is false if you are using an AcceptLanguages from a CIM client. The
Accept-Languages HTTP header from the client contains the fallback specifications.</td>
</tr>
<tr>
<td>Formatter::Arg arg0;
<br> Formatter::Arg arg1;
<br> Formatter::Arg arg2;
<br> Formatter::Arg arg3;
<br> Formatter::Arg arg4;
<br> Formatter::Arg arg5;
<br> Formatter::Arg arg6;
<br> Formatter::Arg arg7;
<br> Formatter::Arg arg8;
<br> Formatter::Arg arg9;</td>
<td>Input
<br>Optional
<br>Default: Formatter::Arg( ) // empty arg</td>
<td>These are the substitution variables, using the Pegasus Formatter::Arg
class.</td>
</tr>
</table>
<p>Notes:
<br>
<p>The "useThreadLocale" parameter defaults to true. This flag indicates
to use the AcceptLanguages object set by Pegasus into the Pegasus Thread
in which the caller's code is running. This AcceptLanguages object
reflects the languages requested by the client. This is useful for
code that may not have access to the AcceptLanguages from the client.
Pegasus sets this AcceptLanguages object into the Thread of providers and
internal Pegasus code. For this reason, it is recommended that provider
and internal Pegasus code use the "useThreadLocale" flag instead of explicity
passing in an AcceptLanguages object. See the Provider Developer
and Pegasus Developer sections for details.
<br>
<p>The "useProcessLocale" flag can be used to tell MessageLoader to use
the default locale of the process, as determined by ICU. This is
useful for situations where the caller is not localizing for a client request.
The caller may itself be a client (eg. cimconfig), or may need to log messages
to the system log in the locale of the Pegasus server process. See
the CLI Messages and Logger Messages sections below.
<br>
<p>"Master switch"
<br>The MessageLoader class has a public static Boolean variable called
_useProcessLocale that may be used to override all the AcceptLanguages
and useThreadLocale settings in the MessageLoaderParms objects passed in.
This is useful for CLI code (eg cimconfig) that needs to localize its messages
based on the locale of its process, which refects the locale set by the
user running the CLI (eg. $LANG on Unix). The CLI code may call Pegasus
APIs that are coded to use the Thread's AcceptLanguages, which will not
be set in this case. The _useProcessLocale static variable tells
the MessageLoader to ignore the AcceptLanguages, useThreadLocale, and useProcessLocale
settings in MessageLoaderParms that it gets. The MessageLoader will
use the default process locale, as determined by ICU, in this case.
<br>
<p><i>Important Note:</i> The MessageLoader defaults to <i>not </i>use
the "fallback" mechanism described in the ICU Resource Management section.
This is because the Accept-Language header itself describes the fallback
that the client wants. However, the MessageLoader does "fallback"
to the root resource bundle if none of the languages in AcceptLanguages
can be found. If the root resource bundle cannot be found, then the
default_msg is returned. The "useICUFallback" flag can be set to
have MessageLoader use ICU fallback on all message load attempts.
However, usage of this flag for client requests may lead to incorrect results.
For example, a client sets Accept-Language to french, german, and spanish,
in that order, but there is no french resource bundle. A call to
MessageLoader with useICUfallback == true would cause the root resource
bundle string to be returned on the attempt to load from the french bundle.
But the client requested german to be the fallback after french.
<br>
<p>Please refer to the following files for details on the new Pegasus classes.
<br>
<ul>
<li>
pegasus/src/Pegasus/Common/MessageLoader.h</li>
</ul>
<h4>
2.2.4 Message Loading Example</h4>
<p><br>The following example shows how a message may be loaded using the
classes described above. Note: this a generic example. Each
of the developer sections below have 'real-life' examples that are better
suited to each type of code.
<p>// Build an AcceptLanguages with some language elements
<br>AcceptLanguages acceptLangs;
<br>acceptLangs.add(AcceptLanguageElement("fr", 0.5));
<br>acceptLangs.add(AcceptLanguageElement("de", 0.8));
<br>acceptLangs.add(AcceptLanguageElement("es", 0.4));
<p>// Construct a MessageLoaderParms
<br>MessageLoaderParms parms("msgID", "default message");
<br>parms. msg_src_path = "/my_msg_dir/my_bundle";
<br>parms.acceptlanguages = acceptLangs;
<p>// Note: If you have args, set them into MessageLoaderParms
<p>// Load the localized String
<br>String localizedMsg = MessageLoader::getMessage(parms);
<br>
<br>
<h4>
2.2.4 Message Writing Guidelines</h4>
<p><br>Here are some basic rules for writing messages:
<br>
<ul>
<li>
If you want to claim that you are globalized, no hardcoded messages!</li>
<li>
Avoid combining messages in the code from other messages. When you
do this you are assuming that you know the grammar for every language.</li>
<li>
String substitutions into messages are generally untranslated, ie. not
loaded from the resource bundle. Example: a file name.</li>
<li>
Avoid jargon, humour, and cultural idioms. Use full sentences.
Have your messages reviewed by your globalization team. Your messages
need to make sense to the translators, and ultimately the customer.</li>
<li>
<b>TODO </b>- find a good message writing guide to link to</li>
</ul>
<h4>
2.2.5 Localized Exceptions</h4>
<p><br>The base Exception class, and derived classes, have been updated
to support localization. Constructors have been added that take a
MessageLoaderParms object. These constructors will use the MessageLoaderParms
object to call the MessageLoader to load the localized exception message.
The localized message is saved in the Exception. The ContentLanguages
object returned by MessageLoader is also saved in the Exception.
This indicates the language of the message. The ContentLanguages
object is used later to set the Content-Language header in the HTTP message
to the client.
<br>
<p>The old Exception constructors that take a String will remain.
These should be used in cases where the code throwing the exception is
not localized, or the String is not localized (for example, a file name).
Also, there are several exceptions in Pegasus where the String parameter
is meant to be a non-localized substitution in a localized message owned
by the Exception (see InternalException.h, ClassNotResolved for an example).
The old constructors for these have been kept.
<br>
<br>
<h2>
3.0 Provider Developers</h2>
<h3>
3.1 Design Issues</h3>
<p><br>Providers that wish to globalize should consider the following in
their design:
<br>
<ul>
<li>
Are there localized string properties that need to be supported?
If so, then the client will use Accept-Language to request specific languages
for these properties. If the properties are read-only, use MessageLoader
to load the localized strings for the properties.</li>
<li>
If you have a localized read/write string property, then the client will
use Content-Language to set the property with an associated language.
The client will expect to be able to retrieve the property in that same
language later (using Accept-Language).</li>
<li>
Note: only the string property types in CIM are candidates for localization.
The other types, including datetime, are locale-neutral.</li>
<li>
Are there error messages that need to returned to the client in different
languages? The client will use Accept-Language to request specific
languages for the error messages.</li>
<li>
What resource bundle translations, if any, will be shipped with the provider?</li>
<li>
Do any codepage conversions need to be done between the UTF-16 characters
in the String objects and the codepage of data stored on the system?
This is a concern for EBCDIC platforms. All EBCDIC data needs to
be converted to at least 7-bit ASCII before it is passed into the String
object.</li>
</ul>
<p><br>To help providers handle the situations described above, Pegasus
2.3 will pass the Accept-Language received from the client to the provider.
The provider should load strings from its resource bundle based on the
client's Accept-Language. The client's Accept-Language is passed
to the provider in two ways:
<br>
<ul>
<li>
Pegasus will set the Accept-Language from the client into the thread in
which the provider is running. By using the useThreadLocale setting
in MessageLoaderParms, providers can easily load strings using the client's
requested Accept-Language. The provider does not need to know what
the Accept-Language is. This is the recommended method to load messages
based on the client's request.</li>
<br>
<li>
The OperationContext will contain an AcceptLanguages object that has the
Accept-Language requested by the client. The provider can use this
AcceptLanguages object to load strings with MessageLoader.</li>
</ul>
<p><br>The OperationContext will also contain a ContentLanguages object
that is set from the Content-Language in the client request. This
is the language of the CIM objects being passed to the provider on that
request. A localized provider should store the content language along
with the data from the CIM objects. This will allow the client to
use Accept-Language later to retreive the data in that language.
<br>
<p>The provider should indicate the language of CIM objects it is returning
by calling setLanguage( ) on the ResponseHandler. This will be used
to set the Content-Language in the CIM response message sent back to the
client. If setLanguage( ) is not called, then no Content-Language
will be returned to the client. setLanguage( ) should only be called
once per response.
<br>
<h3>
3.2 Sample Code</h3>
<p><br>The following sample code shows a localized getInstance( ) where
the instance returned is localized based on the Accept-Language of the
client request. Note that this example also throws a localized exception.
<br>
<p>void LocalizedProvider::getInstance(
<br> const OperationContext & context,
<br> const CIMObjectPath & instanceReference,
<br> const Boolean includeQualifiers,
<br> const Boolean includeClassOrigin,
<br> const CIMPropertyList & propertyList,
<br> InstanceResponseHandler & handler)
<br>{
<br> // convert a potential fully qualified reference
into a local reference
<br> // (class name and keys only).
<br> CIMObjectPath localReference = CIMObjectPath(
<br> String(),
<br> String(),
<br> instanceReference.getClassName(),
<br> instanceReference.getKeyBindings());
<p> // begin processing the request
<br> handler.processing();
<p> // Find the instance to be returned.
<br> Uint32 i;
<br> Uint32 n = _instances.size();
<br> for (i = 0; i < n; i++)
<br> {
<br> if(localReference
== _instanceNames[i])
<br> {
<br>
// We found the instance to return
<p>
// Build the parameters for loading the localized string property.
<br>
// We are going to let the message loader parameters default to use the
<br>
// AcceptLanguages that Pegasus set into our thread.
<br>
// (this equals the AcceptLanguages requested by the client)
<br>
// Note: This parms object could be constructed once and
<br>
// reused.
<br>
MessageLoaderParms parms("myMsgID", "myDefaultString");
<br>
parms.msg_src_path = "/myprovider/msg/myResourceBundle";
<p>
// Load the string for the localized property from the resource bundle
<br>
String localizedString = MessageLoader::getMessage(parms);
<p>
// Remove the old property from the instance to be returned
<br>
Uint32 index = instances[i].findProperty("myProperty");
<br>
if (index != PEG_NOT_FOUND)
<br>
{
<br>
_instances[i].removeProperty(index);
<br>
}
<p>
// Add the localized string property to the instance
<br>
instances[i].addProperty(CIMProperty("myProperty", localizedString));
<p>
// The MessageLoader set the contentlanguages member
<br>
// of parms to the language that it found for the message.
<br>
ContentLanguages rtnLangs = parms.contentlanguages;
<p>
// We need to tag the instance we are returning with the
<br>
// the content language.
<br>
handler.setLanguages(rtnLangs);
<p>
// deliver requested instance
<br>
handler.deliver(_instances[i]);
<p>
break;
<br>
} // end if
<br> }
// end for
<p> // throw an exception if
the instance wasn't found
<br> if (i == n)
<br> {
<br>
// Build the parameters for loading the localized error message.
<br>
// We are going to let the message loader parameters default to use the
<br>
// AcceptLanguages that Pegasus set into our thread.
<br>
// (this equals the AcceptLanguages requested by the client)
<br>
// Note: This parms object could be constructed once and
<br>
// reused.
<br>
MessageLoaderParms errParms("myErrorMsgID", "myErrorDefaultString");
<br>
errParms.msg_src_path = "/myprovider/msg/myResourceBundle";
<p>
// Note: the exception calls MessageLoader::getMessage( )
<br>
// Note: no need to call handler.setLanguages( ) in this case
<br>
throw CIMObjectNotFoundException(errParms);
<br> }
<br>
<p> // complete processing the
request
<br> handler.complete();
<br>}
<br>
<p>NOTE: A sample provider has been written that fully demonstates the
design issues described above. This provider is located at:
<br>
<ul>
<li>
pegasus/src/Providers/sample/LocalizedProvider/</li>
</ul>
<p><br>This sample provider also demonstrates how some of the special issues
can be handled. The special issues are caused by having a read/only
localized property and a read/write localized property. What happens
if the client sets the read/write property with a Content-Language that
is not one of the supported languages for the read/only property?
This provider allows the client to set any language into the read/write
property, and get that property back in the same language. This becomes
an issue when the client does a getInstance( ) later, because the Content-Language
on the returned instance applies to all the properties. A related
issue is what to return for Content-Language when the client does enumerateInstances,
but the instances have different languages. Recall that Content-Language
applies to the entire response (a limitation in the CIM specification).
<br>
<p>NOTE: Indication Providers have other special considerations for
language support. Please refer to PEP58.
<br>
<p>NOTE: The CMPI interface has been updated for language support.
Please refer to the CMPI documentation for details.
<br>
<p>NOTE: SPECIAL ISSUES FOR OS/400 PROVIDERS:
<ul>
<li>
Convert between UTF-16 in the String objects and EBCDIC system data as
needed. The converters in Pegasus/Common/OS400ConvertChar.h may be
used to convert between EBCDIC CCSID 37 and ASCII CCSID 819 (a subset of
UTF-16).</li>
<li>
The Pegasus program, and all bound service programs, will run in
a UTF-8 locale even though the job CCSID is 37. The C-runtime library
(printf, fopen, isalpha, strcmp, etc) will expect UTF-8, or at least 7-bit
ASCII, characters.</li>
<li>
Consideration should be given to the codepage for the compiled string literals.
Use #pragma convert as needed. But, remember that the C-runtime will
expect UTF-8.</li>
<li>
For more details, refer to "Unicode support" in chapter 3 of the <u>ILE
C/C++ for iSeries Run-Time Functions, Version 5</u> publication for V5R3
(SC41-5607-02). The Pegasus string literals will be compiled with
the UTF-8 compile switch described in this section. OS/400 provider
developers should strongly consider using the same compile switch for their
string literals. This would allow the literals to match the UTF-8
encoding expected by the C-runtime.</li>
</ul>
<h2>
4. 0 Client Developers</h2>
<p><br>Methods have been added to CIMClient to set the Accept-Language
and Content-Language on the request, and retrieve Content-Language on the
response. The language tags in the Accept-Language header must meet
the ISO-639 and ISO-3166 standards.
<br>
<p>Please refer to
<br>
<ul>
<li>
pegasus/src/Pegasus/Client/CIMClient.h</li>
<br> </ul>
for the new methods on CIMClient.
<br>
<p>Here is a code fragment that uses the new methods on CIMClient
<p> //
<br> // Get a localized instance in French
<br> //
<p> // Language priority is martian, pig-latin, and french.
We should
<br> // get french back, even though its the lowest priority
<br> AcceptLanguages acceptLangs;
<br> acceptLangs.add(AcceptLanguageElement("x-martian"));
<br> acceptLangs.add(AcceptLanguageElement("fr", 0.1));
<br> acceptLangs.add(AcceptLanguageElement("x-pig-latin", 0.4));
<p> // Set the requested languages into the CIMClient
<br> client.setRequestAcceptLanguages(acceptLangs);
<p> // Get the instance
<br> CIMInstance instance = client.getInstance(
<br> NAMESPACE,
<br> cimNInstances[0].buildPath(sampleClass),
<br> localOnly,
<br> includeQualifiers,
<br> includeClassOrigin);
<p> // Get the string property that should be french
<br> String returnedString;
<br> instance.getProperty (
<br> instance.findProperty("myProp")).
<br>
getValue().
<br>
get(returnedString);
<p> // Check that we got back french
<br> ContentLanguages CL_FR("fr");
<br> String expectedFRString = "oui";
<br> PEGASUS_ASSERT(CL_FR == client.getResponseContentLanguages());
<br> PEGASUS_ASSERT(expectedFRString == returnedString);
<p> //
<br> // Create an instance in French
<br> //
<p> String oui = "Oui";
<br> CIMInstance frInstance(CLASSNAME);
<br> frInstance.addProperty(CIMProperty(
<br>
CIMName("myProp"),
<br>
oui));
<p> CIMObjectPath frInstanceName = frInstance.buildPath(sampleClass);
<p> client.setRequestContentLanguages(CL_FR);
<p> client.createInstance(NAMESPACE, frInstance);
<br>
<br>
<br>
<p>Also, refer to
<ul>
<li>
pegasus/src/Clients/g11ntest/</li>
</ul>
for more examples of a client that uses Accept-Language and Content-Language.
<br>
<p>NOTE: Consideration should be given for converting the UTF-16
characters in the String objects passed over the CIMClient interface to
a platform codepage. This is especially needed for EBCDIC platforms.
See the Provider developer section for details of the EBCDIC considerations.
<br>
<br>
<h3>
4.1 Default Process Locale</h3>
<p><br>A method has been added to CIMClient to set the Accept-Language
for the requests based on the default locale of the process, as determined
by ICU. If ICU is installed on the client system then CIMClient will
set the Accept-Language from the default ICU process locale. If ICU
is not installed then the caller is required to set an AcceptLanguages
into CIMClient that meets the ISO-639 and IS0-3166 standards. Note:
this is useful for local clients, such as the Pegasus CLIs, where ICU would
be installed on both the client and server sides.
<br>
<br>
<h2>
5. 0 Pegasus Developers</h2>
<p><br>The design for Pegasus releases beyond 2.3 is to avoid using hardcoded
messages. All new messages should be loaded from a Pegasus resource
bundle. This section describes the process to follow if you are creating
a new message. The process depends on where you are in the code.
<br>
<br>
<h3>
<b>5.1 Pegasus Resource Bundles</b></h3>
<p><br>Place any new Pegasus messages into one of the following resource
bundles:
<br>
<ul>
<li>
pegasus/src/Pegasus/msg/Server/pegasusServer_*.txt for server and
MOF compiler (cimmof, cimmofl) messages</li>
<li>
pegasus/src/Pegasus/msg/CLI/pegasusCLI_*.txt for all CLI messages (except
the MOF compiler)</li>
</ul>
<p><br>The make messages target will compile these resource bundles.
<p>Note: As described above, the resource bundle path in MessageLoaderParms
defaults to the server resource bundle. For CLI messages, you will
need to specify the bundle for your CLI.
<br>
<h3>
5.2 Server Messages</h3>
<p><br>For messages returned from one of the services in the Pegasus server
(eg. CIMOperationRequestDispatcher, or ProviderManagerService), the goal
is to make it easy for any code in the call chain to throw an exception
with a localized error string. The code throwing the exception will
not need to know the Accept-Language that the client requested. To
understand how this works, some design points need to described:
<br>
<p><b>Server Design Points:</b>
<br>
<p>The CIMMessage object has been expanded to include an AcceptLanguages
object and a ContentLanguages object. For CIMRequestMessage, these
objects contain the Accept-Language and Content-Language headers that were
built from the client request. For CIMResponseMessage, the ContentLanguages
object is used to build the Content-Language header associated with the
CIM <i>objects </i>in the response message. The AcceptLanguages object
in the CIMResponseMessage is ignored.
<br>
<p>The localization of the cimException object in the CIMResponseMessage
is handled separately from the CIM objects. The message string in
the cimException object is assumed to have been localized by the time it
is built into the XML. For this reason, the localization of the exception
is the responsibility of the code throwing the exception. (The goal
of the design is to make that easy - see below). The ContentLanguages
object in the CIMResponseMessage has NO relation to this exception.
The cimException object keeps its own localization information once it
is created.
<br>
<p>To enable exceptions to be localized, the ability was added to set a
global language for all the code running from a Pegasus Thread object.
The top level code for a Thread can set a global AcceptLanguages object
that can be accessed by all the low-level functions that it calls.
This will allow an exception thrown by the low-level function to be localized
based on this global AcceptLanguages object. Note: This applies
only to Threads that are managed by a ThreadPool.
<br>
<p>Each service in the request path of the Pegasus server sets the AcceptLanguages
into its Thread from the AcceptLanguages in the CIMRequestMessage object
that it dequeues. This sets the global langauge for all the functions
in the same thread that are called below handleEnqueue. <i>If you
are writing a new service that processes requests, or discover a request
service that was missed, please do this. </i> The CIMOperationRequestDispatcher
service is an example.
<br>
<p><b>How to Throw a Localized Exception from Server code:</b>
<br>
<p>With all that background, here is how code running in a Pegasus service
can throw a localized exception:
<br>This example assumes that the top-level code in the service had set
the global thread AcceptLanguages beforehand. As described above,
every service in Pegasus should do that. The code here may be buried
several layers deep in the call chain, but does not need to know the AcceptLanguage
of the current client request.
<p>// First, construct a MessageLoaderParms
<br>//
<br>// Notes:
<br>// 1) The errorMessageID must be in the Pegasus server resource
bundle.
<br>// 2) The default message is the old "hardcoded" message.
<br>// 3) The MessageLoaderParms will default to use the Pegasus
server resource bundle
<br>// 4) The MessageLoaderParms will default to use the AcceptLanguages
set into the current Thread. Don't change this!
<br>// 5) You might need to set the arguments for the message into
the MessageLoaderParms
<br>MessageLoaderParms parms("errorMessageID", "default message");
<p>// Second, throw the Exception
<br>// Note: this applies to all the derived classes from Exception, including
the CIMException's
<br>throw new Exception(parms);
<br>
<p>NOTE: If you are throwing an Exception with un-localized data,
use the constructor that takes a String. An example of this would
be an Exception where you are passing in a file name. Most of the
"non-CIM" exceptions defined in Exception.h and InternalException.h take
un-localized data.
<br>
<p><b>The Exception Macros</b>
<br>
<p>There are many spots in the server code that use the PEGASUS_CIM_EXCEPTION
macro to throw a TraceableCIMException. The use of this macro in
the code like the following example presented a design problem:
<p>....
<br>} catch (Exception & e)
<br>{
<br> throw PEGASUS_CIM_EXCEPTION(CIM_ERR_FAILED, e.getMessage());
<br>}
<br>
<p>This type of code would have lost the ContentLanguages saved in "e",
so that the Content-Language would not be set in HTTP response to the client.
<br>
<p>For Pegasus 2.3, these types of macro calls can stay. The TraceableCIMException
constructed by the macro will "re-localize". That is, the "CIM" part
of the message (the part based on the error code) will be localized at
throw time, and the ContentLanguages re-established. A key is to
avoid a "language mismatch" problem between the CIM part of the message
and the extra part of the message. The design point here is that
all internal exceptions thrown by Pegasus code are localized using the
global AcceptLanguages of the Thread...see above.
<br>
<p>In the future, it will be safer and more maintainable to use of the
new "localized" flavors of the macro. For example:
<br>
<p>When the message from a caught Exception needs to be become the
extra message in a thrown CIMException:
<p>....
<br>} catch (Exception & e)
<br>{
<br> throw PEGASUS_CIM_EXCEPTION_LANG(e.getContentLanguages(
),
<br>
CIM_ERR_FAILED,
<br>
e.getMessage( ));
<br>}
<br>
<p>This guarantees that the ContentLanguages in "e" is copied to the newly
created TraceableCIMException.
<br>
<p>In the case where the extra message for the CIMException is determined
by the throwing code:
<br>
<p>throw PEGASUS_CIM_EXCEPTION_L(CIM_ERR_FAILED,
<br>
MessageLoaderParms("Repository.CIMRepository.COMPACT_FAILED", "compact
failed"));
<br>
<p>(example from CIMRepository.cpp)
<br>This uses a MessageLoaderParms object to localize the extra message
in the newly created TraceableCIMException.
<br>
<h3>
5.2 Logger Messages</h3>
<p><br>New methods have been added to Logger to take a message ID of a
message to be loaded from the Pegasus server resource bundle. The
caller is only required to pass in the message ID, the old "hardcoded"
message, and the args. The Logger will use MessageLoader to load
the message in the locale of the Pegasus server <i>process</i>, using the
hardcoded message as the default string. Please refer to pegasus/src/Pegasus/Logger.h.
<p>Note: Messages sent to the "logs", whether the system logs or
the Pegasus log file, are converted to UTF-8 before being sent.
<br>
<h3>
5.3 CLI Messages</h3>
<p><br>The goal for messages returned by the Pegasus CLIs is to localize
in the locale of the user running the CLI. This should be automatic
-- the user should not be required to tell the CLI what the locale is.
For the CLIs that are CIM clients (cimconfing, cimprovider) there are two
sets of messages to localize -- messages generated in the CLI process
itself, and messages returned from the Pegasus server . For CLIs
that are directly linked into Pegasus (cimmofl), all the messages are generated
in the CLI's process, but the CLI may call Pegasus APIs that are coded
to localize based on a client's requested languages.
<br>
<p>Code in the client side of the client/server CLIs (eg. cimconfig, cimmof),
or in directly linked CLIs (cimmofl), should use the _useProcessLocale
"master switch" described in the Message Loading section. This will
cause all messages, including exceptions thrown by Pegasus APIs,
to be loaded in the locale based on the environment in which the program
is running. This locale can be set by the user before running the
program.
<br>
<p>Code in the client side of the client/server CLIs need to send an Accept-Language
to the Pegasus server that reflects the default locale of the CLI's process.
See the Client Developer section for details.
<br>
<p>An example of these considerations can be seen in the source code for
cimconfig.
<br>
<p>
<hr>
<p><i>Copyright (c) 2003 BMC Software; Hewlett-Packard Development Company,
L.P.; IBM Corp.; The Open Group</i>
<p><i>Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following conditions:</i>
<p><i>THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE SHALL BE INCLUDED
IN ALL COPIES OR SUBSTANTIAL PORTIONS OF THE SOFTWARE. THE SOFTWARE IS
PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.</i>
<br>
<br>
</body>
</html>
| No CVS admin address has been configured |
Powered by ViewCVS 0.9.2 |