pegasus/doc/Globalization_HOWTO.htm - diff

Return to Globalization_HOWTO.htm CVS log

Up to [Pegasus] / pegasus / doc

Diff for /pegasus/doc/Globalization_HOWTO.htm between version 1.1.2.1 and 1.6.38.1

version 1.1.2.1, 2003/07/16 21:45:25

version 1.6.38.1, 2007/01/30 18:59:01

Line 1

<!doctype html public "-//w3c//dtd html 4.0 transitional//en">

<!DOCTYPE doctype PUBLIC "-//w3c//dtd html 4.0 transitional//en">

<html>

<head>

content="text/html; charset=iso-8859-1">

content="Mozilla/4.78 [en] (X11; U; Linux 2.4.7-10 i686) [Netscape]">

</head>

<body text="#000000" bgcolor="#ffffff" link="#0000ef" vlink="#55188a"

alink="#ff0000">

<center>Globalization HOWTO

Release: Pegasus 2.3

<big><big><big>Globalization HOWTO</big></big></big>

Author: Chuck Carmack (carmack@us.ibm.com)

Release: Pegasus 2.3

July 2, 2003</center>

Author: Chuck Carmack (carmack@us.ibm.com)

December 1, 2003

</center>

Change History:

<table cellpadding="2" cellspacing="2" border="1"

style="text-align: left; width: 100%; margin-left: auto; margin-right: auto;">

<tbody>

<tr>

<td style="vertical-align: top;">01/12/03

</td>

<td style="vertical-align: top;">carmack

</td>

<td style="vertical-align: top;">Section 2.2.2.  Changed how

the package name parameter should be used.  It should no longer be

used as part of the table name inside the bundle.

</td>

</tr>

<tr>

<td style="vertical-align: top;">01/23/07

</td>

<td style="vertical-align: top;">Sushma Fernandes

</td>

<td style="vertical-align: top;">Section 2.2.5.   Added information on special considerations while creating a new message.

</td>

</tr>

</tbody>

</table>

<h2> 1.0 Introduction</h2>

As part of the Pegasus 2.3 release, functions were added for

globalization support.   Globalization involves two major

aspects:  internationalization and localization.

Internationalization is the process of writing a program that is

locale-neutral.  In other words, the program should be able to run

in any locale without change.  There are several categories in a

locale, including the language of message strings, date format, time

format, etc.  For release 2.3, the Pegasus server is concerned with

the language of the message strings it returns to its clients.

To support internationalization, a program is designed to do the

following:

<blockquote> <li> Support character sets that can represent customer

data in any language.  Typically, the program supports some

variation of Unicode for internal data.  There is usually some

conversion between the supported character sets for external data, and

the internal character set.  Since Unicode covers all characters,

and usually has converters on the platform, it is a good choice for the

'normalized' internal character set.    The most

'interoperable' solution for external data is to support UTF-8 (eg.

network and file system data).  The internal data is usually UTF-16

(or UCS-2, but that is deprecated).</li>

<li> Extract locale-sensitive resources, such as message

strings, from the code to external resource files.  Typically, the

resources are loaded based on the locale requested by the end-user, and

returned to the end-user for display.</li>

NOTE:  THIS IS A WORK-IN-PROGRESS

<h2>

1.0 Introduction</h2>

As part of the Pegasus 2.3 release, functions were added for globalization

support.   Globalization involves two major aspects:  internationalization

and localization.

Internationalization is the process of writing a program that is locale-neutral.

In other words, the program should be able to run in any locale without

change.  There are several categories in a locale, including the language

of message strings, date format, time format, etc.  For release 2.3,

the Pegasus server is concerned with the language of the message strings

it returns to its clients.

To support internationalization, a program is designed to do the following:

<li>

Support character sets that can represent customer data in any language. 

Typically, the program supports some variation of Unicode for internal

data.  There is usually some conversion between the supported character

sets for external data, and the internal character set.  Since Unicode

covers all characters, and usually has converters on the platform, it is

a good choice for the 'normalized' internal character set.   

The most 'interoperable' solution for external data is to support UTF-8

(eg. network and file system data).  The internal data is usually

UTF-16 (or UCS-2, but that is deprecated).</li>

<li>

Extract locale-sensitive resources, such as message strings, from the code

to external resource files.  Typically, the resources are loaded based

on the locale requested by the end-user, and returned to the end-user for

display.</li>

</blockquote>

Localization is the process of customizing a software product to

Localization is the process of customizing a software product to

support particular locales.  For example, a product that is internationalized

support particular locales.  For example, a product that is

might want to only localize for certain countries.  This would mean

internationalized might want to only localize for certain

that the localized resources (eg. message files) would only be translated

countries.  This would mean that the localized resources (eg.

and shipped for the countries that the product supports.  Since the

message files) would only be translated and shipped for the countries

code for the product is locale-neutral, it will be easy to drop in new

that the product supports.  Since the code for the product is

translations as more countries are supported.

locale-neutral, it will be easy to drop in new translations as more

countries are supported.

The Pegasus 2.3 release added support for globalization.  At a

high-level, the following additions were made to Pegasus 2.3:

high-level, the following additions were made to Pegasus 2.3:

<ul>

<li>

<li> Support UTF-8 for external data.</li>

Support UTF-8 for external data.</li>

<ul>

<li>

<li> The CIM-XML documents contained in the HTTP messages</li>

The CIM-XML documents contained in the HTTP messages</li>

<li> The files in the repository</li>

<li> Note:  Pegasus 2.3 does NOT support UTF-8 in the MOF

<li>

files</li>

Repository files and MOF files  (TODO - remove MOF files if

we can't get this into 2.3)</li>

</ul>

<li> Support UTF-16 for internal data.</li>

<li>

Support UTF-16 for internal data.</li>

<li> Extract the hardcoded messages from the Pegasus code into

message files.  An API was added to load messages from the message

files.</li>

<li>

Extract the hardcoded messages from the Pegasus code into message files. 

<li> APIs were added for clients to associate a language with

An API was added to load messages from the message files.</li>

the CIM objects they are sending to Pegasus.  Also, APIs were added

for clients to determine the language of the error message or CIM

object that Pegasus returns.</li>

<li>

APIs were added for clients to associate a language with the CIM objects

<li> APIs were added for providers to determine the language of

they are sending to Pegasus.  Also, APIs were added for clients to

CIM objects sent by the client.  Also, APIs were added for

determine the language of the error message or CIM object that Pegasus

providers to associate a language with the CIM object, or error message,

returns.</li>

they return to the client.</li>

<li>

APIs were added for providers to determine the language of CIM objects

sent by the client.  Also, APIs were added for providers to associate

a language with the CIM object, or error message, they return to the client.</li>

</ul>

Please refer to PEPs 56 and 58 for details about the globalization

Please refer to PEPs 56 and 58 for details about the globalization

support in Pegasus 2.3.

design in Pegasus 2.3.

This document provides a HOWTO guide to be used by developers to globalize

This document provides a HOWTO guide to be used by developers to

code that is being added to Pegasus.  The audience for this document

globalize code that is being added to Pegasus.  The audience for

are:

this document are:

<ul>

<li>

<li> Provider developers - both CMPI and C++</li>

Provider developers - both CMPI and C++</li>

<li> Client developers</li>

<li> Pegasus developers</li>

<li>

Client developers</li>

<li>

Pegasus developers</li>

</ul>

The quickest way to approach this document is to read the General

The quickest way to approach this document is to read the General

section, and then the developer section that relates to what you are doing.

section, and then the developer section that relates to what you are

doing.

<h2>

2.0 General</h2>

<h2> 2.0 General</h2>

<h3>

<h3> 2.1 Unicode Support</h3>

2.1 Unicode Support</h3>

Pegasus 2.3 supports Unicode throughout the processing of

Pegasus 2.3 supports Unicode throughout the processing of requests.

requests.  External data to Pegasus is encoded in UTF-8. 

External data to Pegasus is encoded in UTF-8.  Internal data is encoded

Internal data is encoded in UTF-16.

in UTF-16.

UTF-8 support for external data includes the CIM-XML messages passed

External data includes the CIM-XML messages passed over the network,

over the network, and the repository files.  Note:  UTF-8

the repository files, and the MOF files.  For the CIM-XML messages,

support was NOT added to the MOF Compiler for MOF files in release

Pegasus follows section 4.8 of the  <a href="http://www.dmtf.org/standards/documents/WBEM/DSP200.html">CIM-HTTP

2.3.  For the CIM-XML messages, Pegasus follows section 4.8 of

the  <a

href="http://www.dmtf.org/standards/documents/WBEM/DSP200.html">CIM-HTTP

specification</a>    Specifically, Pegasus supports the

"utf-8" setting for the charset parameter of the Content-Type header and

the XML encoding attribute.  If no charset is specified, the 7-bit

ASCII is assumed.  The Pegasus MOF compiler supports UTF-8 encoding

ASCII is assumed.

in the MOF files.  (TODO - remove this statement if this is

not in 2.3)

The internal support of UTF-16 is encapsulated in the Pegasus String

class.  This class has been updated to contain UTF-16 characters. 

class.  This class has been updated to contain UTF-16

Specifically, the Char16 objects inside the String contain UTF-16 characters. 

characters.  Specifically, the Char16 objects inside the String

Note: a UTF-16 surrogate pair is contained in two consecutive Char16 objects. 

contain UTF-16 characters.  Note: a UTF-16 surrogate pair is

To keep backwards compatibilty, the methods on the String class have not

contained in two consecutive Char16 objects.  To keep backwards

changed.  New methods have been added as needed.  The following

compatibilty, the methods on the String class have not changed. 

describes this in more detail:

New methods have been added as needed.  The following describes

this in more detail:

<ul>

<li>

<li> The Pegasus 2.2 methods that take a char *, or return char *, are

The Pegasus 2.2 methods that take a char *, or return char *, are unchanged. 

unchanged.  Code written to Pegasus 2.2 may have expected to store

Code written to Pegasus 2.2 may have expected to store 8-bit ASCII (ISO-8859-1)

8-bit ASCII (ISO-8859-1) characters into String.  These methods

characters into String.  These methods will convert the input to UTF-16

will convert the input to UTF-16 from 8-bit ASCII.  (This is simple

from 8-bit ASCII.  (This is simple because UTF-16 is a superset of

because UTF-16 is a superset of 8-bit ASCII - simply need to prepend

8-bit ASCII - simply need to prepend '\0' to each char).  The Pegasus

'\0' to each char).  The Pegasus 2.2 methods that return char data

2.2 methods that return char data will attempt to convert from the UTF-16

will attempt to convert from the UTF-16 internal representation to

internal representation to 8-bit ASCII.  Characters that cannot be

8-bit ASCII.  Characters that cannot be converted will be replaced

converted will be replaced with a substitution character.</li>

with a substitution character.</li>

<li> All methods that take or return Char16 data are

<li>

unchanged.  The String class now supports UTF-16 data in Char16,

All methods that take or return Char16 data are unchanged.  The String

although surrogate pairs will require two consecutive Char16

class now supports UTF-16 data in Char16, although surrogate pairs will

objects.  The String class does NO checking for unmatched surrogate

require two consecutive Char16 objects.  The String class does NO

pairs.</li>

checking for unmatched surrogate pairs.</li>

<li> New methods have been added to take and return UTF-8

data.  The String class will convert between UTF-8 and the UTF-16

<li>

internal representation as needed.  These new methods will use char

New methods have been added to take and return UTF-8 data.  The String

* parameters, but will be clearly labelled as UTF-8 methods.</li>

class will convert between UTF-8 and the UTF-16 internal representation

as needed.  These new methods will use char * parameters, but will

be clearly labelled as UTF-8 methods.</li>

</ul>

PROGRAMMING NOTE:  Putting EBCDIC data into the String class is dangerous. 

The String class is designed for UTF-16, which is a superset of 8-bit ASCII. 

Any String object containing EBCDIC data will not work if it is used by

Pegasus to read or write data from external sources, such as the network

or repository files.  In other words, any String containing EBCDIC

data should not leave the code using it.

<h3>

2.2 Localization Support</h3>

<h4>

</ul>

2.2.1 Language Headers</h4>

PROGRAMMING NOTE:  Putting EBCDIC data into the String class is

dangerous.  The String class is designed for UTF-16, which is a

Pegasus 2.3 supports clients and providers that wish to localize.

superset of 8-bit ASCII.  Any String object containing EBCDIC data

There are two areas to be localized:  <a href="http://www.dmtf.org/standards/documents/WBEM/DSP201.html#SecERROR">ERROR</a>

will not work if it is used by Pegasus to read or write data from

elements in the CIM-XML; and  <a href="http://www.dmtf.org/standards/documents/WBEM/DSP201.html#SecObjectDefinitionElements">Object

external sources, such as the network or repository files.  In

Definition</a>  elements in the CIM-XML.  Clients can request

other words, any String containing EBCDIC data should not leave the code

the server to return error messages and CIM objects in a set of languages

using it.

of their choosing.  Clients can also tag a language to the CIM objects

they are sending to the server.  Providers and the server can return

error messages and CIM objects that are tagged with one of  languages

<h3> 2.2 Localization Support</h3>

requested by the client.

<h4> 2.2.1 Language Headers</h4>

The localization design is based on section 4.8 of the <a href="http://www.dmtf.org/standards/documents/WBEM/DSP200.html">CIM-HTTP

specification</a> , which refers to <a href="http://www.ietf.org/rfc/rfc2616.txt?number=2616">RFC

Pegasus 2.3 supports clients and providers that wish to localize. 

2616</a>.  The method used to tag a language to the CIM-XML is through

There are two areas to be localized:  <a

the Accept-Language and Content-Language HTTP headers.  These headers

href="http://www.dmtf.org/standards/documents/WBEM/DSP201.html#SecERROR">ERROR</a>

elements in the CIM-XML; and  <a

href="http://www.dmtf.org/standards/documents/WBEM/DSP201.html#SecObjectDefinitionElements">Object

Definition</a>  elements in the CIM-XML.  Clients can

request  the server to return error messages and CIM objects in a

set of languages of their choosing.  Clients can also tag a

language to the CIM objects they are sending to the server. 

Providers and the server can return error messages and CIM objects that

are tagged with one of  languages requested by the client.

The localization design is based on section 4.8 of the <a

href="http://www.dmtf.org/standards/documents/WBEM/DSP200.html">CIM-HTTP

specification</a> , which refers to <a

href="http://www.ietf.org/rfc/rfc2616.txt?number=2616">RFC 2616</a>.

The method used to tag a language to the CIM-XML is through the

Accept-Language and Content-Language HTTP headers.  These headers

are basically lists of language tags.  An HTTP request can contain

an Accept-Language header, which indicates the list of preferred languages

an Accept-Language header, which indicates the list of preferred

that the client wants in the response.  This list can be prioritized

languages that the client wants in the response.  This list can be

by using the quality numbers.  An HTTP request or response can contain

prioritized by using the quality numbers.  An HTTP request or

a Content-Language header, which indicates the language(s) of the content

response can contain a Content-Language header, which indicates the

in the message.  In the Pegasus case, this would be the CIM-XML. 

language(s) of the content in the message.  In the Pegasus case,

Note that the Content-Language header is a list of language tags. 

this would be the CIM-XML.  Note that the Content-Language header

This allows the content of an HTTP message to contain more than one translation. 

is a list of language tags.  This allows the content of an HTTP

However, in the Pegasus case, there is only one CIM-XML document in the

message to contain more than one translation.  However, in the

HTTP message, and thus one translation.

Pegasus case, there is only one CIM-XML document in the HTTP message,

and thus one translation.

CIM clients may use the Accept-Language HTTP header to specify the languages

they wish to be returned in the CIM response message.  CIM clients

CIM clients may use the Accept-Language HTTP header to specify the

may also use the Content-Language header to tag the language of any CIM

languages they wish to be returned in the CIM response message. 

objects they are sending to the server in the CIM request message. 

CIM clients may also use the Content-Language header to tag the language

The server, and providers, should attempt to return error messages and

of any CIM objects they are sending to the server in the CIM request

CIM objects in one of the accept languages requested by the client. 

message.  The server, and providers, should attempt to return

The server and providers should set the Content-Language header in the

error messages and CIM objects in one of the accept languages requested

CIM response message to indicate which of the requested languages they

by the client.  The server and providers should set the

are returning.

Content-Language header in the CIM response message to indicate which of

the requested languages they are returning.

NOTE:  Localization support was not added for the MOF files and

repository in Pegasus 2.3.  The #pragma locale, #pragma instancelocale,

repository in Pegasus 2.3.  The #pragma locale, #pragma

and translatable qualifier flavor are not supported in the Pegasus 2.3

instancelocale, and translatable qualifier flavor are not supported in

MOF compiler.  From the client perspective, classes, qualifiers, and

the Pegasus 2.3 MOF compiler.  From the client perspective,

instances stored in the repository as not tagged with a language. 

classes, qualifiers, and instances stored in the repository are not

The Accept-Language and Content-Language headers will be ignored for repository

tagged with a language.  The Accept-Language and Content-Language

operations.  However, since the repository will support UTF-8, 

headers will be ignored for repository operations.  However, since

characters for any language may be stored there.

the repository will support UTF-8,  characters for any language may

be stored there.

NOTE:  Since the Content-Language header applies to the entire

HTTP message, it applies to the entire CIM-XML document.  This includes

HTTP message, it applies to the entire CIM-XML document.  This

all the objects in the document, including enumerated objects, and all

includes all the objects in the document, including enumerated objects,

the values in the objects.  This is a limitation that will remain

and all the values in the objects.  This is a limitation that will

until the CIM standard has been updated to support language tags tied to

remain until the CIM standard has been updated to support language tags

individual CIM values.  From the client perspective, it is possible

tied to individual CIM values.  From the client perspective, it is

for Pegasus to send a CIM response with NO Content-Language, even if the

possible for Pegasus to send a CIM response with NO Content-Language,

client had sent Accept-Language.   This can happen if Pegasus

even if the client had sent Accept-Language.   This can happen

does not know the language of the response.  An example is a request

if Pegasus does not know the language of the response.  An example

that was sent to a Pegasus 2.2 provider.  Another example is an enumerated

is a request that was sent to a Pegasus 2.2 provider.  Another

response where each provider returned a different language.  Please

example is an enumerated response where each provider returned a

refer to PEP58 for details on these provider scenarios.

different language.  Please refer to PEP58 for details on these

provider scenarios.

Pegasus 2.3 has added classes for the localization support.  There

are new classes called AcceptLanguages and ContentLanguages that encapsulate

Pegasus 2.3 has added classes for the localization support.

the Accept-Language and Content-Language headers, respectively.  These

There are new classes called AcceptLanguages and ContentLanguages that

classes are basically containers of AcceptLanguageElement and ContentLanguageElement,

encapsulate the Accept-Language and Content-Language headers,

where a language element represents one language tag.  The AcceptLanguages

respectively.  These classes are basically containers of

class will keep the AcceptLanguageElement's prioritized based on quality,

AcceptLanguageElement and ContentLanguageElement, where a language

according to RFC 2616.

element represents one language tag.  The AcceptLanguages class

will keep the AcceptLanguageElement's prioritized based on quality,

AcceptLanguages and ContentLanguages are the objects used by code throughout

according to RFC 2616.

the request/response processing, from the client to the server to the providers

and back.  The server handles the creation of these objects from the

AcceptLanguages and ContentLanguages are the objects used by code

HTTP headers.  Code at each point in the process will have access

throughout the request/response processing, from the client to the

to these objects.

server to the providers and back.  The server handles the creation

of these objects from the HTTP headers.  Code at each point in the

Please refer to the following files for details on the new Pegasus classes.

process will have access to these objects.

Please refer to the following files for details on the new Pegasus

classes.

<ul>

<li>

<li> pegasus/src/Pegasus/Common/AcceptLanguages.h</li>

pegasus/src/Pegasus/Common/AcceptLanguages.h</li>

<li> pegasus/src/Pegasus/Common/AcceptLanguageElement.h</li>

<li> pegasus/src/Pegasus/Common/ContentLanguages.h</li>

<li>

<li> pegasus/src/Pegasus/Common/ContentLanguageElement.h</li>

pegasus/src/Pegasus/Common/AcceptLanguageElement.h</li>

<li> pegasus/src/Pegasus/Common/LanguageElementContainer.h</li>

<li> pegasus/src/Pegasus/Common/LanguageElement.h</li>

<li>

pegasus/src/Pegasus/Common/ContentLanguages.h</li>

<li>

pegasus/src/Pegasus/Common/ContentLanguageElement.h</li>

<li>

pegasus/src/Pegasus/Common/LanguageElementContainer.h</li>

<li>

pegasus/src/Pegasus/Common/LanguageElement.h</li>

</ul>

See the sections below for details on how to write clients and providers

See the sections below for details on how to write clients and

to use these classes.

providers to use these classes.

<h4>

<h4> 2.2.2 Message Bundles</h4>

2.2.2 Message Loading</h4>

One of the goals of globalization for Pegasus 2.3 is the extraction of

One of the goals of globalization for Pegasus 2.3 is the extraction

hardcoded messages  into translated message files, loading

of hardcoded messages  into external message files, and loading messages

translated messages from those files, and returning those messages to

from those files.  The topics in this section are:  how to create

the client.  The topics to be discussed here are:  how to

message files, and how to load messages.

create message files, how to compile message files, and how to load

messages into Pegasus.

At the time of writing, the message loading function in Pegasus 2.3

used the International Components for Unicode (<a href="http://oss.software.ibm.com/icu">ICU)</a>

used the International Components for Unicode (<a

libraries.  This is expected to be the future direction for Pegasus.

href="http://oss.software.ibm.com/icu">ICU)</a> libraries.  This

<a href="http://oss.software.ibm.com/icu">ICU

is expected to be the future direction for Pegasus. <a

</a>uses

href="http://oss.software.ibm.com/icu">ICU </a>uses a resource bundle

a resource bundle format for their message files.   In order

format for their message files.   In order to load the

to load the messages, ICU requires that the resource bundles are compiled

messages, ICU requires that the resource bundles are compiled into a

into a binary form (.res file) using their genrb tool.

binary form (.res file) using their genrb tool.

The documentation for ICU resource bundles is in the <a href="http://oss.software.ibm.com/icu/userguide/ResourceManagement.html">Resource

Platform Maintainers Note:  Please refer to PEP 58 for

Management</a>  section of the <a href="http://oss.software.ibm.com/icu/userguide/">ICU

information about how to build Pegasus to use the ICU libraries.

User Guide</a> .  This section will tell you how to

create, organize, and compile your resource bundles for different languages.

The documentation for ICU resource bundles is in the <a

Note:  your resource bundles should be organized in a tree structure

href="http://oss.software.ibm.com/icu/userguide/ResourceManagement.html">Resource

similiar to the one shown in the Resource Management section, including

Management</a>  section of the <a

the empty bundles in the tree.

href="http://oss.software.ibm.com/icu/userguide/">ICU User Guide</a>

.  This section will tell you how to create and organize your

resource bundles for different languages.  Note:  your

resource bundles should be organized in a tree structure similiar to

the one shown in the Resource Management section, including the empty

bundles in the tree.

It is recommended that you ship a root resource bundle to be used as

the fallback in case the client requests a language that you are not

supporting.  The Pegasus make files are set up to automatically

create and compile a root resource bundle for you.  For Pegasus

2.3, the make will use your "en" bundle, upper case all the messages,

and then put the uppercased messages into the root bundle.  The

uppercasing of the messages is necessary to create a "fallback" root

bundle that contains invariant characters across all EBCDIC and

ASCII codepages.

NOTE:  When creating your resource bundles, the name of the

table resource should not

contain the package name.    For example, if you

have a bundle with a package name of "xyz", then the "en" bundle should

start like this:

en:table {

..... messages here

}

not like this:

xyz_en:table {

..... messages here

}

This is needed because the package name (-p) option is used by the

Pegasus make files on the call to genrb.

NOTE:  Pegasus 2.3 only supports simple string resources in the

ICU resource bundles.  String resources may only be loaded by key. 

ICU resource bundles.  String resources may only be loaded by

Tables, arrays, and other complex resource types, are not supported.

key.  Tables, arrays, and other complex resource types, are not

supported.

Code that needs to load a message in Pegasus does not call ICU directly.

Two message loading classes were added for Pegasus 2.3:  MessageLoader

In order to compile your resource bundles, support has been added to

and MessageLoaderParms.  These classes are abstractions designed to

the Pegasus make files to run genrb.  A new make target,

hide of the actual loader used.   The MessageLoader is used to

"messages", has been added that will call genrb and put the compiled

load a message using a list of preferrred languages.  The parameters

bundles (.res) in a directory of your choosing.  An example of ICU

to MessageLoader are encapsulated in a MessageLoaderParms object.

resource bundles and the make files to compile them are located in:

The MessageLoaderParms object contains the parameters to load the message.

<ul>

There are many parameters, but many can be allowed to default.  Here

<li> pegasus/src/Providers/sample/LocalizedProvider/Makefile (just

is a description of the parameters:

causes the make to recurse to the msg sub-directory)</li>

<li> pegasus/src/Providers/sample/LocalizedProvider/msg/Makefile

NOTE:  WORK-IN- PROGRESS

(compiles the bundles in the msg/ directory)</li>

<li> pegasus/src/Providers/sample/LocalizedProvider/msg/*.txt (the

resource bundles to compile, using the recommended ICU language tree

structure)</li>

</ul>

NOTE:  At the time of writing, only the Linux make files have been

updated to compile ICU resource bundles.

It is important to place the compiled resource bundles in a

directory where your code can find them .  The make files above

compile the resource bundles into

$PEGASUS_HOME/msg/provider/localizedProvider.  The code that loads

these messages uses the MessageLoader class (next section) to load

messages from this directory.

<h4> 2.2.3 Message Loading</h4>

Code that needs to load a message in Pegasus does not call ICU

directly.  Two message loading classes were added for Pegasus

2.3:  MessageLoader and MessageLoaderParms.  These classes are

abstractions designed to hide of the actual loader used (but note that

at the time of writing, only ICU is supported).   The

MessageLoader is used to load a message using a list of preferrred

languages.  The parameters to MessageLoader are encapsulated in a

MessageLoaderParms object.

The MessageLoader is the place where the Accept-Language header,

Content-Language header, and the ICU resource bundles, join up. 

The MessageLoader class is designed to receive an AcceptLanguages

object, and a set of parameters indicating the bundle base-name and

message ID to use.  The AcceptLanguages object contains the list of

requested languages sent by the client.  The MessageLoader

searches for the message in the set of bundles named with the base-name,

using the AcceptLanguages for the list of specific translated bundles

to search.  The MessageLoader returns the message that it found,

along with a ContentLanguages object indicating the language of the

message.  The ContentLanguages object should be used to indicate

the language of the response sent back to the client.

The MessageLoaderParms object contains the parameters to load the

message.  There are many parameters, but many can be allowed to

default.  Here is a description of the parameters:

<tbody>

<tr>

<td>String msg_id; </td>

<td>Input.

<td>Input.

Required.</td>

Required.</td>

<td>Message ID  of the message to load from the resource

bundle.  This is the key that ICU will use to load the message.</td>

<td>Message ID  of the message to load from the resource bundle.

This is the key that ICU will use to load the message.</td>

</tr>

<tr>

<td>String default_msg;</td>

<td>Input.

<td>Input.

Required</td>

Required</td>

<td>Message to return if the no message can be loaded for msg_id

from any resource bundle.  Note:  The args parameters below

<td>Message to return if the no message can be loaded for msg_id from a

are substituted into this string.

resource bundle.  Note:  The args parameters below are substituted

Note:  For the args into this  string, use the Pegasus '$'

into this string. 

form, as described in pegasus/src/Pegasus/Common/Formatter.h. 

Note:  For the args into this  string, use the Pegasus '$'

Don't use the ICU substitution format for the default message string.</td>

form, as described in pegasus/src/Pegasus/Common/Formatter.h.  Don't

use the ICU substitution format for the default message string.</td>

</tr>

<tr>

<td>String msg_src_path; </td>

<td>Input.

<td>Input.

Optional

Optional

Default: $PEGASUS_HOME/msg/pegasus/pegasusServer</td>

Default: $PEGASUS_HOME/msg/pegasus/pegasusServer</td>

<td>Path to the resource bundle file which contains the

msg_id.

<td>Path to the root resource bundle file which contains the msg_id.

Note: Only specify the path down to the bundle base-name.  Do not

Do not include the language or file extension as part of the path.

append a language tag, such as "_root" or "_en".  Do not append a

Note: relative paths start at $PEGASUS_HOME/msg. </td>

file extension.

Note: relative paths start at $PEGASUS_HOME/msg.

Note: defaults to the bundle containing the Pegasus server messages.</td>

</tr>

<tr>

<td>AcceptLanguages acceptlanguages;</td>

<td>Input.

<td>Input.

Optional

Optional

Default: AcceptLanguages::EMPTY</td>

Default: AcceptLanguages::EMPTY</td>

<td>Contains the list of preferred languages, in priority

order.  This is combined with msg_src_path to determine which

<td>Contains the list of preferred languages, in priority order.

resource bundles to search for for the msg_id.   If not empty,

This is combined with msg_src_path to determine which resource bundles

overrides useThreadLocale and useProcessLocale.</td>

to search for for the msg_id.   If not EMPTY, overrides useThreadLocale

and useProcessLocale.</td>

</tr>

<tr>

<td>ContentLanguages contentlanguages;</td>

<td>Output</td>

<td>Contains the language that MessageLoader found for the

<td>Contains the language that MessageLoader found for the msg_id. </td>

msg_id. </td>

</tr>

<tr>

<td>Boolean useProcessLocale;</td>

<td>Input

<td>Input

Optional

Optional

Default = false</td>

Default = false</td>

<td>If true, MessageLoader will use the default locale of the

process.  If true, overrides useThreadLocale.</td>

<td>If true, MessageLoader will use the default locale of the process.

If true, overrides useThreadLocale.</td>

</tr>

<tr>

<td>Boolean useThreadLocale;</td>

<td>Input

<td>Input

Optional

Optional

Default = true</td>

Default = true</td>

<td>If true, MessageLoader will use the AcceptLanguages set by

Pegasus into the caller's Thread.   See the Note below for

<td>If true, MessageLoader will use the locale of the caller's thread. </td>

details. </td>

</tr>

<tr>

<td>Boolean useICUfallback</td>

<td>Input

<td>Input

Optional

Optional

Default = false</td>

Default = false</td>

<td>If true, use ICU's fallback mechnism to search more general

resource bundles if the msg_id cannot be found.  Note: the

<td>If true, use ICU's fallback mechnism to search more general resource

recommended setting is false if you are using an AcceptLanguages from a

bundles if the msg_id cannot be found.  Note: the recommended setting

CIM client.  The Accept-Languages HTTP header from the client

is false if you are using an AcceptLanguages from a CIM client.  The

contains the fallback specifications.  Using ICU's fallback in this

Accept-Languages HTTP header from the client contains the fallback specifications.</td>

case may lead to returning a language that the client didn't ask for.</td>

</tr>

<tr>

<td>Formatter::Arg arg0;

<td>Formatter::Arg arg0;

Formatter::Arg arg1;

Formatter::Arg arg1;

Formatter::Arg arg2;

Formatter::Arg arg2;

Formatter::Arg arg3;

Formatter::Arg arg3;

Formatter::Arg arg4;

Formatter::Arg arg4;

Formatter::Arg arg5;

Formatter::Arg arg5;

Formatter::Arg arg6;

Formatter::Arg arg6;

Formatter::Arg arg7;

Formatter::Arg arg7;

Formatter::Arg arg8;

Formatter::Arg arg8;

Formatter::Arg arg9;</td>

Formatter::Arg arg9;</td>

<td>Input

<td>Input

Optional

Optional

Default: Formatter::Arg( ) // empty arg</td>

Default: Formatter::Arg( ) // empty arg</td>

<td>These are the substitution variables, using the Pegasus

Formatter::Arg class.</td>

<td>These are the substitution variables, using the Pegasus Formatter::Arg

class.</td>

</tr>

</tbody>

</table>

Please refer to the following files for details on the new Pegasus classes.

Notes:

The "useThreadLocale" parameter defaults to true.  This flag

indicates to use the AcceptLanguages object set by Pegasus into the

Pegasus Thread in which the caller's code is running.  This

AcceptLanguages object reflects the languages requested by the

client.  This is useful for code that may not have access to the

AcceptLanguages from the client.  Pegasus sets this AcceptLanguages

object into the Thread of providers and internal Pegasus code. 

For this reason, it is recommended that provider and internal Pegasus

code use the "useThreadLocale" flag instead of explicity passing in an

AcceptLanguages object.  See the Provider Developer and Pegasus

Developer sections for details.

The "useProcessLocale" flag can be used to tell MessageLoader to use

the default locale of the process, as determined by ICU.  This is

useful for situations where the caller is not localizing for a client

request.  The caller may itself be a client (eg. cimconfig), or may

need to log messages to the system log in the locale of the Pegasus

server process.  See the CLI Messages and Logger Messages sections

below.

"Master switch"

The MessageLoader class has a public static Boolean variable called

_useProcessLocale that may be used to override all the AcceptLanguages

and useThreadLocale settings in the MessageLoaderParms objects passed

in.  This is useful for CLI code (eg cimconfig) that needs to

localize its messages based on the locale of its process, which refects

the locale set by the user running the CLI (eg. $LANG on Unix). 

The CLI code may call Pegasus APIs that are coded to use the Thread's

AcceptLanguages, which will not be set in this case.  The

_useProcessLocale static variable tells the MessageLoader to ignore the

AcceptLanguages, useThreadLocale, and useProcessLocale settings in

MessageLoaderParms that it gets.  The MessageLoader will use the

default process locale, as determined by ICU, in this case.

Important Note:  The MessageLoader defaults to not use

the "fallback" mechanism described in the ICU Resource Management

section.  This is because the Accept-Language header itself

describes the fallback that the client wants.  However, the

MessageLoader does "fallback" to the root resource bundle if none of the

languages in AcceptLanguages can be found.  If the root resource

bundle cannot be found, then the default_msg is returned.  The

"useICUFallback" flag can be set to have MessageLoader use ICU fallback

on all message load attempts.  However, usage of this flag for

client requests may lead to incorrect results.  For example, a

client sets Accept-Language to french, german, and spanish, in that

order, but there is no french resource bundle.  A call to

MessageLoader with useICUfallback == true would cause the root resource

bundle string to be returned on the attempt to load from the french

bundle.  But the client requested german to be the fallback after

french.

Please refer to the following files for details on the new Pegasus

classes.

<ul>

<li>

<li> pegasus/src/Pegasus/Common/MessageLoader.h</li>

pegasus/src/Pegasus/Common/MessageLoader.h</li>

</ul>

<h4> 2.2.4 Message Loading Example</h4>

<h4>

2.2.3 Message Loading Example</h4>

The following example shows how a message may be loaded using the

classes described above.  Note: this a generic example.  Each

of the developer sections below have 'real-life' examples that are

better suited to each type of code.

// Build an AcceptLanguages with some language elements

AcceptLanguages acceptLangs;

acceptLangs.add(AcceptLanguageElement("fr", 0.5));

acceptLangs.add(AcceptLanguageElement("de", 0.8));

acceptLangs.add(AcceptLanguageElement("es", 0.4));

// Construct a MessageLoaderParms

MessageLoaderParms parms("msgID", "default message");

parms. msg_src_path = "/my_msg_dir/my_bundle";

parms.acceptlanguages = acceptLangs;

The following example shows how a message may be loaded using the classes

// Note: If you have args, set them into MessageLoaderParms

described above.  Note: this a generic example.  Each of the

// Load the localized String

developer sections below have 'real-life' examples that are better suited

String localizedMsg = MessageLoader::getMessage(parms);

to each type of code.

// Build an AcceptLanguages with some language elements

AcceptLanguages acceptLangs;

<h4> 2.2.5 Message Writing Guidelines</h4>

acceptLangs.add(AcceptLanguageElement("fr", 0.5));

acceptLangs.add(AcceptLanguageElement("de", 0.8));

Here are some basic rules for writing messages:

acceptLangs.add(AcceptLanguageElement("es", 0.4));

// Construct a MessageLoaderParms

MessageLoaderParms parms("msgID", "default message");

parms. msg_src_path = "/my_msg_dir/my_bundle";

parms.acceptlanguages = acceptLangs;

// Note: If you have args, set them into MessageLoaderParms

// Load the localized String

String localizedMsg = MessageLoader::getMessage(parms);

<h4>

2.2.4 Message Writing Guidelines</h4>

Here are some basic rules for writing messages:

<ul>

<li>

<li> If you want to claim that you are globalized, no hardcoded

If you want to claim that you are globalized, no hardcoded messages!</li>

messages!</li>

<li> Avoid creating a message in the code by combining other

<li>

messages.  When you do this you are assuming that you know the

Avoid combining messages in the code from other messages.  When you

grammar for every language.</li>

do this you are assuming that you know the grammar for every language.</li>

<li> String substitutions into messages are generally untranslated,

ie. not loaded from the resource bundle.   Example: a file

<li>

name.</li>

String substitutions into messages are generally untranslated, ie. not

<li> Avoid jargon, humour, and cultural idioms.  Use full

loaded from the resource bundle.   Example: a file name.</li>

sentences.  Have your messages reviewed by your globalization

team.  Your messages need to make sense to the translators, and

<li>

ultimately the customer.</li>

Avoid jargon, humour, and cultural idioms.  Use full sentences. 

<li> TODO - find a good message writing guide to link to</li>

Have your messages reviewed by your globalization team.  Your messages

need to make sense to the translators, and ultimately the customer.</li>

<li>

TODO - find a good message writing guide to link to</li>

</ul>

<h2>

Are there any special considerations while creating a new message ?

3.0 Provider Developers</h2>

<h3>

3.1 Design Issues</h3>

Providers that wish to globalize should consider the following in

their design:

<ul>

<li>

<li>If a message definition contains text within a single quote

Are there localized string properties that need to be supported? 

it is not interpreted in any way.

If so, then the client will use Accept-Language to request specific languages

for these properties.  If the properties are read-only, use MessageLoader

to load the localized strings for the properties.</li>

<li>

Example:

If you have a localized read/write string property, then the client will

Server.CIMOperationRequestAuthorizer.NOT_IN_AUTHORIZED_GRP:

use Content-Language to set the property with an associated language. 

string {"PGS05202: User '{0}' is not authorized to access CIM data."}

The client will expect to be able to retrieve the property in that same

language later (using Accept-Language).</li>

Processed message:

User {0} is not authorized to access CIM data.

</li>

<li> For a single quote to appear in a processed message, it needs to be preceded by

another single quote.

Example:

Server.CIMOperationRequestAuthorizer.NOT_IN_AUTHORIZED_GRP:

string {"PGS05202: User ''{0}'' is not authorized to access CIM data."}

Processed message:

User 'wbemuser' is not authorized to access CIM data.

</li>

<li>

<li> For a double quote to appear in a processed message, it needs to be preceded by

Note: only the string property types in CIM are candidates for localization. 

a back slash.

The other types, including datetime, are locale-neutral.</li>

<li>

Example:

Are there error messages that need to returned to the client in different

languages?  The client will use Accept-Language to request specific

ControlProviders.ProviderRegistrationProvider.ProviderRegistrationProvider.

languages for the error messages.</li>

UNSUPPORTED_USERCONTEXT_VALUE:string {"PGS03029: Unsupported UserContext

value: \"{0}\"."}

Processed message:

Unsupported UserContext value: "10".

</li>

</ul>

<li>

What resource bundle translations, if any, will be shipped with the provider?</li>

<li>

<h4> 2.2.6 Localized Exceptions</h4>

Do any codepage conversions need to be done between the UTF-16 characters

in the String objects and the codepage of data stored on the system? 

The base Exception class, and derived classes, have been updated to

This is a concern for EBCDIC platforms.  All EBCDIC data needs to

support localization.  Constructors have been added that take a

be converted to at least 7-bit ASCII before it is passed into the String

MessageLoaderParms object.  These constructors will use the

object.</li>

MessageLoaderParms object to call the MessageLoader to load the

localized exception message.  The localized message is saved in the

Exception.  The ContentLanguages object returned by MessageLoader

is also saved in the Exception.  This indicates the language of

the message.  The ContentLanguages object is used later to set the

Content-Language header in the HTTP message to the client.

The old Exception constructors that take a String will remain.

These should be used in cases where the code throwing the exception is

not localized, or the String is not localized (for example, a file

name).  Also, there are several exceptions in Pegasus where the

String parameter is meant to be a non-localized substitution in a

localized message owned by the Exception (see InternalException.h,

ClassNotResolved for an example).  The old constructors for these

have been kept.

<h2> 3.0 Provider Developers</h2>

<h3> 3.1 Design Issues</h3>

Providers that wish to globalize should consider the following in their

design:

<ul>

<li> Are there localized string properties that need to be

supported?  If so, then the client will use Accept-Language to

request specific languages for these properties.  If the properties

are read-only, use MessageLoader to load the localized strings for the

properties.</li>

<li> If you have a localized read/write string property, then the

client will use Content-Language to set the property with an associated

language.  The client will expect to be able to retrieve the

property in that same language later (using Accept-Language).</li>

<li> Note: only the string property types in CIM are candidates for

localization.  The other types, including datetime, are

locale-neutral.</li>

<li> Are there error messages that need to returned to the client in

different languages?  The client will use Accept-Language to

request specific languages for the error messages.</li>

<li> What resource bundle translations, if any, will be shipped with

the provider?</li>

<li> Do any codepage conversions need to be done between the UTF-16

characters in the String objects and the codepage of data stored on the

system?  This is a concern for EBCDIC platforms.  All EBCDIC

data needs to be converted to at least 7-bit ASCII before it is passed

into the String object.</li>

</ul>

To help providers handle the situations described above, Pegasus

To help providers handle the situations described above, Pegasus 2.3

2.3 will pass the Accept-Language received from the client to the provider. 

will pass the Accept-Language received from the client to the

The provider should load strings from its resource bundle based on the

provider.  The provider should load strings from its resource

client's Accept-Language.  The client's Accept-Language is passed

bundle based on the client's Accept-Language.  The client's

to the provider in two ways:

Accept-Language is passed to the provider in two ways:

<ul>

<li>

<li> Pegasus will set the Accept-Language from the client into the

Pegasus will set the Accept-Language from the client into the thread in

thread in which the provider is running.  By using the

which the provider is running.  By using the useThreadLocale setting

useThreadLocale setting in MessageLoaderParms, providers can easily load

in MessageLoaderParms, providers can easily load strings using the client's

strings using the client's requested Accept-Language.  The

requested Accept-Language.  The provider does not need to know what

provider does not need to know what the Accept-Language is.  This

the Accept-Language is.  This is the recommended method to load messages

is the recommended method to load messages based on the client's request.</li>

based on the client's request.</li>

<li> The OperationContext will contain an AcceptLanguages object

that has the Accept-Language requested by the client.  The provider

<li>

can use this AcceptLanguages object to load strings with MessageLoader.</li>

The OperationContext will contain an AcceptLanguages object that has the

Accept-Language requested by the client.  The provider can use this

AcceptLanguages object to load strings with MessageLoader.</li>

</ul>

The OperationContext will also contain a ContentLanguages object

The OperationContext will also contain a ContentLanguages object that

that is set from the Content-Language in the client request.  This

is set from the Content-Language in the client request.  This is

is the language of the CIM objects being passed to the provider on that

the language of the CIM objects being passed to the provider on that

request.  A localized provider should store the content language along

request.  A localized provider should store the content language

with the data from the CIM objects.  This will allow the client to

along with the data from the CIM objects.  This will allow the

use Accept-Language later to retreive the data in that language.

client to use Accept-Language later to retreive the data in that

language.

The provider should indicate the language of CIM objects it is returning

by calling setLanguage( ) on the ResponseHandler.  This will be used

The provider should indicate the language of CIM objects it is

to set the Content-Language in the CIM response message sent back to the

returning by calling setContext( ) on the ResponseHandler.  This

client.  If setLanguage( ) is not called, then no Content-Language

will be used to set the Content-Language in the CIM response message

will be returned to the client.  setLanguage( ) should only be called

sent back to the client.  If setContext( ) is not called, then no

once per response.

Content-Language will be returned to the client.  The setContext( )

function should only be called once per response.

<h3>

3.2 Sample Code</h3>

<h3> 3.2 Sample Code</h3>

The following sample code shows a localized getInstance( ) where

The following sample code shows a localized getInstance( ) where the

the instance returned is localized based on the Accept-Language of the

instance returned is localized based on the Accept-Language of the

client request.  Note that this example also throws a localized exception.

client request.  Note that this example also throws a localized

exception.

void LocalizedProvider::getInstance(

const OperationContext & context,

void LocalizedProvider::getInstance(

const CIMObjectPath & instanceReference,

const OperationContext & context,

const Boolean includeQualifiers,

const CIMObjectPath & instanceReference,

const Boolean includeClassOrigin,

const Boolean includeQualifiers,

const CIMPropertyList & propertyList,

const Boolean includeClassOrigin,

InstanceResponseHandler & handler)

const CIMPropertyList & propertyList,

{

InstanceResponseHandler & handler)

// convert a potential fully qualified reference

{

into a local reference

     // convert a potential fully qualified

// (class name and keys only).

reference into a local reference

CIMObjectPath localReference = CIMObjectPath(

// (class name and keys only).

String(),

CIMObjectPath localReference = CIMObjectPath(

String(),

String(),

instanceReference.getClassName(),

String(),

instanceReference.getKeyBindings());

// begin processing the request

instanceReference.getClassName(),

handler.processing();

// Find the instance to be returned.

instanceReference.getKeyBindings());

Uint32 i;

// begin processing the request

Uint32 n = _instances.size();

handler.processing();

for (i = 0;  i < n;  i++)

// Find the instance to be returned.

{

Uint32 i;

if(localReference

Uint32 n = _instances.size();

== _instanceNames[i])

for (i = 0;  i < n;  i++)

{

{

// We found the instance to return

if(localReference == _instanceNames[i])

{

// We found the instance to return

// Build the parameters for loading the localized string property.

// Build the parameters for loading the localized string property.

// We are going to let the message loader parameters default to use the

// We are going to let the message loader parameters default to use the

// AcceptLanguages that Pegasus set into our thread.

// AcceptLanguages that Pegasus set into our thread.

// (this equals the AcceptLanguages requested by the client)

// (this equals the AcceptLanguages requested by the client)

// Note: This parms object could be constructed once and

// Note: This parms object could be constructed once and

// reused.

// reused.

MessageLoaderParms parms("myMsgID", "myDefaultString");

MessageLoaderParms parms("myMsgID", "myDefaultString");

parms.msg_src_path = "/myprovider/msg/myResourceBundle";

parms.msg_src_path = "/myprovider/msg/myResourceBundle";

// Load the string for the localized property from the resource bundle

// Load the string for the localized property from the resource bundle

String localizedString = MessageLoader::getMessage(parms);

String localizedString = MessageLoader::getMessage(parms);

// Remove the old property from the instance to be returned

// Remove the old property from the instance to be returned

Uint32 index = instances[i].findProperty("myProperty");

Uint32 index = instances[i].findProperty("myProperty");

if (index != PEG_NOT_FOUND)

if (index != PEG_NOT_FOUND)

{

{

_instances[i].removeProperty(index);

_instances[i].removeProperty(index);

}

}

// Add the localized string property to the instance

// Add the localized string property to the instance

instances[i].addProperty(CIMProperty("myProperty", localizedString));

instances[i].addProperty(CIMProperty("myProperty", localizedString));

// The MessageLoader set the contentlanguages member

// The MessageLoader set the contentlanguages member

// of parms to the language that it found for the message.

// of parms to the language that it found for the message.

ContentLanguages rtnLangs = parms.contentlanguages;

ContentLanguages rtnLangs = parms.contentlanguages;

// We need to tag the instance we are returning with the

// We need to tag the instance we are returning with the

                 // the

// the content language.

content language.

handler.setLanguages(rtnLangs);

OperationContext context;

// deliver requested instance

context.insert(ContentLanguageListContainer(rtnLangs));

handler.setContext(context);

// deliver requested instance

handler.deliver(_instances[i]);

break;

break;

}        //  end if

}        //  end if

}

// end for

}            //

end for

// throw an exception if

the instance wasn't found

the instance wasn't found

if (i == n)

if (i == n)

{

{

// Build the parameters for loading the localized error message.

// Build the parameters for loading the localized error message.

// We are going to let the message loader parameters default to use the

// We are going to let the message loader parameters default to use the

// AcceptLanguages that Pegasus set into our thread.

// AcceptLanguages that Pegasus set into our thread.

// (this equals the AcceptLanguages requested by the client)

// (this equals the AcceptLanguages requested by the client)

// Note: This parms object could be constructed once and

// Note: This parms object could be constructed once and

// reused.

// reused.

MessageLoaderParms errParms("myErrorMsgID", "myErrorDefaultString");

MessageLoaderParms errParms("myErrorMsgID", "myErrorDefaultString");

errParms.msg_src_path = "/myprovider/msg/myResourceBundle";

errParms.msg_src_path = "/myprovider/msg/myResourceBundle";

// Note: the exception calls MessageLoader::getMessage( )

// Note: the exception calls MessageLoader::getMessage( )

// Note: no need to call handler.setLanguages( ) in this case

// Note: no need to call handler.setContext( ) in this case

throw CIMObjectNotFoundException(errParms);

throw CIMObjectNotFoundException(errParms);

}

}

// complete processing the

// complete processing

request

the request

handler.complete();

handler.complete();

}

}

NOTE: A sample provider has been written that fully demonstates the

design issues described above.  This provider is located at:

design issues described above.  This provider is located at:

<ul>

<li>

<li> pegasus/src/Providers/sample/LocalizedProvider/</li>

pegasus/src/Providers/sample/LocalizedProvider/</li>

</ul>

This sample provider also demonstrates how some of the special issues

This sample provider also demonstrates how some of the special issues

can be handled.  The special issues are caused by having a read/only

can be handled.  The special issues are caused by having a

localized property and a read/write localized property.  What happens

read/only localized property and a read/write localized property. 

if the client sets the read/write property with a Content-Language that

What happens if the client sets the read/write property with a

is not one of the supported languages for the read/only property? 

Content-Language that is not one of the supported languages for the

This provider allows the client to set any language into the read/write

read/only property?  This provider allows the client to set any

property, and get that property back in the same language.  This becomes

language into the read/write property, and get that property back in the

an issue when the client does a getInstance( ) later, because the Content-Language

same language.  This becomes an issue when the client does a

on the returned instance applies to all the properties.  A related

getInstance( ) later, because the Content-Language on the returned

issue is what to return for Content-Language when the client does enumerateInstances,

instance applies to all the properties.  A related issue is what to

but the instances have different languages.  Recall that Content-Language

return for Content-Language when the client does enumerateInstances,

applies to the entire response (a limitation in the CIM specification).

but the instances have different languages.  Recall that

Content-Language applies to the entire response (a limitation in the CIM

NOTE:  Indication Providers have other special considerations for

specification).

language support.  Please refer to  PEP58.

NOTE:  Indication Providers have other special considerations

NOTE:  The CMPI interface has been updated for language support.

for language support.  Please refer to  PEP58.

Please refer to the CMPI documentation for details.

NOTE:  The CMPI interface has been updated for language

NOTE: SPECIAL ISSUES FOR OS/400 PROVIDERS:

support.  Please refer to the CMPI documentation for details.

NOTE: SPECIAL ISSUES FOR OS/400 PROVIDERS:

<ul>

<li>

<li> Convert between UTF-16 in the String objects and EBCDIC system

Convert between UTF-16 in the String objects and EBCDIC system data as

data as needed.  The converters in

needed.  The converters in Pegasus/Common/OS400ConvertChar.h may be

Pegasus/Common/OS400ConvertChar.h may be used to convert between EBCDIC

used to convert between EBCDIC CCSID 37 and ASCII CCSID 819 (a subset of

CCSID 37 and ASCII CCSID 819 (a subset of UTF-16).</li>

UTF-16).</li>

<li> The Pegasus program, and all bound service programs,  will

run in a UTF-8 locale even though the job CCSID is 37.  The

<li>

C-runtime library (printf, fopen, isalpha, strcmp, etc) will expect

The Pegasus program, and all bound service programs,  will run in

UTF-8, or at least 7-bit ASCII, characters.</li>

a UTF-8 locale even though the job CCSID is 37.  The C-runtime library

<li> Consideration should be given to the codepage for the compiled

(printf, fopen, isalpha, strcmp, etc) will expect UTF-8, or at least 7-bit

string literals.  Use #pragma convert as needed.  But,

ASCII, characters.</li>

remember that the C-runtime will expect UTF-8.</li>

<li> For more details, refer to "Unicode support" in chapter 3 of the ILE

<li>

Consideration should be given to the codepage for the compiled string literals. 

Use #pragma convert as needed.  But, remember that the C-runtime will

expect UTF-8.</li>

<li>

For more details, refer to "Unicode support" in chapter 3 of the ILE

C/C++ for iSeries Run-Time Functions, Version 5 publication for V5R3

(SC41-5607-02).  The Pegasus string literals will be compiled with

the UTF-8 compile switch described in this section.  OS/400 provider

the UTF-8 compile switch described in this section.  OS/400

developers should strongly consider using the same compile switch for their

provider developers should strongly consider using the same compile

string literals.  This would allow the literals to match the UTF-8

switch for their string literals.  This would allow the literals to

encoding expected by the C-runtime.</li>

match the UTF-8 encoding expected by the C-runtime.</li>

</ul>

<h2> 4. 0 Client Developers</h2>

<h2>

4. 0 Client Developers</h2>

Methods have been added to CIMClient to set the Accept-Language and

Content-Language on the request, and retrieve Content-Language on the

Methods have been added to CIMClient to set the Accept-Language

response.  The language tags in the Accept-Language header must

and Content-Language on the request, and retrieve Content-Language on the

meet the ISO-639 and ISO-3166 standards.

response.

Please refer to

Please refer to

<ul>

<li>

<li> pegasus/src/Pegasus/Client/CIMClient.h</li>

pegasus/src/Pegasus/Client/CIMClient.h</li>

</ul>

</ul>

for the new methods on CIMClient.

for the new methods on CIMClient.

Here is a code fragment that uses the new methods on CIMClient

Here is a code fragment that uses the new methods on CIMClient

//

//

// Get a localized instance in French

// Get a localized instance in French

//

//

// Language priority is martian, pig-latin, and french.

// Language priority is martian, pig-latin, and

We should

french.  We should

// get french back, even though its the lowest priority

// get french back, even though its the lowest priority

AcceptLanguages acceptLangs;

AcceptLanguages acceptLangs;

acceptLangs.add(AcceptLanguageElement("x-martian"));

acceptLangs.add(AcceptLanguageElement("x-martian"));

acceptLangs.add(AcceptLanguageElement("fr", 0.1));

acceptLangs.add(AcceptLanguageElement("fr", 0.1));

acceptLangs.add(AcceptLanguageElement("x-pig-latin", 0.4));

acceptLangs.add(AcceptLanguageElement("x-pig-latin", 0.4));

// Set the requested languages into the CIMClient

// Set the requested languages into the CIMClient

client.setRequestAcceptLanguages(acceptLangs);

client.setRequestAcceptLanguages(acceptLangs);

// Get the instance

// Get the instance

CIMInstance instance = client.getInstance(

CIMInstance instance = client.getInstance(

NAMESPACE,

NAMESPACE,

cimNInstances[0].buildPath(sampleClass),

cimNInstances[0].buildPath(sampleClass),

localOnly,

localOnly,

includeQualifiers,

includeQualifiers,

includeClassOrigin);

includeClassOrigin);

// Get the string property that should be french

// Get the string property that should be french

String returnedString;

String returnedString;

instance.getProperty (

instance.getProperty (

instance.findProperty("myProp")).

instance.findProperty("myProp")).

getValue().

getValue().

get(returnedString);

get(returnedString);

// Check that we got back french

// Check that we got back french

ContentLanguages CL_FR("fr");

ContentLanguages CL_FR("fr");

String expectedFRString = "oui";

String expectedFRString = "oui";

PEGASUS_ASSERT(CL_FR == client.getResponseContentLanguages());

PEGASUS_ASSERT(CL_FR == client.getResponseContentLanguages());

PEGASUS_ASSERT(expectedFRString == returnedString);

PEGASUS_ASSERT(expectedFRString == returnedString);

//

//

// Create an instance in French

// Create an instance in French

//

//

String oui = "Oui";

String oui = "Oui";

CIMInstance frInstance(CLASSNAME);

CIMInstance frInstance(CLASSNAME);

frInstance.addProperty(CIMProperty(

frInstance.addProperty(CIMProperty(

CIMName("myProp"),

CIMName("myProp"),

oui));

oui));

CIMObjectPath frInstanceName = frInstance.buildPath(sampleClass);

CIMObjectPath frInstanceName =

client.setRequestContentLanguages(CL_FR);

frInstance.buildPath(sampleClass);

client.createInstance(NAMESPACE, frInstance);

client.setRequestContentLanguages(CL_FR);

client.createInstance(NAMESPACE, frInstance);

Also, refer to

Also, refer to

<ul>

<li>

<li> pegasus/src/Clients/g11ntest/</li>

pegasus/src/Clients/g11ntest/</li>

</ul>

for more examples of a client that uses Accept-Language and Content-Language.

for more examples of a client that uses Accept-Language and

Content-Language.

NOTE:  Consideration should be given for converting the UTF-16

characters in the String objects passed over the CIMClient interface to

a platform codepage.  This is especially needed for EBCDIC platforms. 

a platform codepage.  This is especially needed for EBCDIC

See the Provider developer section for details of the EBCDIC considerations.

platforms.  See the Provider developer section for details of the

EBCDIC considerations.

TODO - some info on how CIMClient defaults the Accept-Languages.

<h2>

<h3> 4.1 Default Process Locale</h3>

5. 0 Pegasus Developers</h2>

A method has been added to CIMClient to set the Accept-Language for the

The design for Pegasus releases beyond 2.3 is to avoid using hardcoded

requests based on the default locale of the process, as determined by

messages.  All new messages should be loaded from a Pegasus resource

ICU.  If ICU is installed on the client system then CIMClient will

bundle.  This section describes the process to follow if you are creating

set the Accept-Language from the default ICU process locale.  If

a new message.  The process depends on where you are in the code.

ICU is not installed then the caller is required to set an

AcceptLanguages into CIMClient that meets the ISO-639 and IS0-3166

standards.  Note:  this is useful for local clients, such as

<h3>

the Pegasus CLIs, where ICU would be installed on both the client and

5.1 Pegasus Resource Bundles</h3>

server sides.

Place any new Pegasus messages into one of the following resource

bundles:

<h2> 5. 0 Pegasus Developers</h2>

The design for Pegasus releases beyond 2.3 is to avoid using hardcoded

messages.  All new messages should be loaded from a Pegasus

resource bundle.  This section describes the process to follow if

you are creating a new message.  The process depends on where you

are in the code.

<h3> 5.1 Pegasus Resource Bundles</h3>

Place any new Pegasus messages into one of the following resource

bundles:

<ul>

<li>

<li> pegasus/src/Pegasus/msg/Server/pegasusServer_*.txt  for

pegasus/src/Pegasus/msg/Server/pegasusServer.txt  for server messages</li>

server and MOF compiler (cimmof, cimmofl) messages</li>

<li> pegasus/src/Pegasus/msg/CLI/pegasusCLI_*.txt for all CLI messages

<li>

(except the MOF compiler)</li>

pegasus/src/Clients/<cli_name>/msg/<cli_name>.txt for CLI messages</li>

</ul>

Note:  As described above, the resource bundle path in MessageLoaderParms

defaults to the server resource bundle.  For CLI messages, you will

The make messages target will compile these resource bundles.

need to specify the bundle for your CLI.

Note:  As described above, the resource bundle path in

MessageLoaderParms defaults to the server resource bundle.  For CLI

<h3>

messages, you will need to specify the bundle for your CLI.

5.2 Server Messages</h3>

<h3> 5.2 Server Messages</h3>

For messages returned from one of the services in the Pegasus server

For messages returned from one of the services in the Pegasus server

(eg. CIMOperationRequestDispatcher, or ProviderManagerService), the goal

is to make it easy for any code in the call chain to throw an exception

with a localized error string.  The code throwing the exception will

with a localized error string.  The code throwing the exception

not need to know the Accept-Language that the client requested.  To

will not need to know the Accept-Language that the client

understand how this works, some design points need to described:

requested.  To understand how this works, some design points need

to described:

Server Design Points:

Server Design Points:

The CIMMessage object has been expanded to include an AcceptLanguages

object and a ContentLanguages object.  For CIMRequestMessage, these

The CIMMessage object has been expanded to include an

objects contain the Accept-Language and Content-Language headers that were

AcceptLanguages object and a ContentLanguages object.  For

built from the client request.  For CIMResponseMessage, the ContentLanguages

CIMRequestMessage, these objects contain the Accept-Language and

object is used to build the Content-Language header associated with the

Content-Language headers that were built from the client request. 

CIM objects in the response message.  The AcceptLanguages object

For CIMResponseMessage, the ContentLanguages object is used to build the

in the CIMResponseMessage is ignored.

Content-Language header associated with the CIM objects in the

response message.  The AcceptLanguages object in the

The localization of the cimException object in the CIMResponseMessage

CIMResponseMessage is ignored.

is handled separately from the CIM objects.  The message string in

the cimException object is assumed to have been localized by the time it

The localization of the cimException object in the

is built into the XML.  For this reason, the localization of the exception

CIMResponseMessage is handled separately from the CIM objects.  The

is the responsibility of the code throwing the exception.  (The goal

message string in the cimException object is assumed to have been

of the design is to make that easy - see below).  The ContentLanguages

localized by the time it is built into the XML.  For this reason,

object in the CIMResponseMessage has NO relation to this exception. 

the localization of the exception is the responsibility of the code

The cimException object keeps its own localization information once it

throwing the exception.  (The goal of the design is to make that

is created.

easy - see below).  The ContentLanguages object in the

CIMResponseMessage has NO relation to this exception.  The

cimException object keeps its own localization information once it is

created.

To enable exceptions to be localized, the ability was added to set a

global language for all the code running from a Pegasus Thread object. 

global language for all the code running from a Pegasus Thread

The top level code for a Thread can set a global AcceptLanguages object

object.  The top level code for a Thread can set a global

that can accessed by all the low-level functions that it calls.  This

AcceptLanguages object that can be accessed by all the low-level

will allow an exception thrown by low-level code to be localized based

functions that it calls.  This will allow an exception thrown by

on this global AcceptLanguages object.  Note:  This applies only

the low-level function to be localized based on this global

to Threads that are managed by a ThreadPool.

AcceptLanguages object.  Note:  This applies only to Threads

that are managed by a ThreadPool.

Each service in the request path of the Pegasus server sets the AcceptLanguages

into its Thread from the AcceptLanguages in the CIMRequestMessage object

Each service in the request path of the Pegasus server sets the

that it dequeues.  This sets the global langauge for all the functions

AcceptLanguages into its Thread from the AcceptLanguages in the

in the same thread that are called below handleEnqueue.  If you

CIMRequestMessage object that it dequeues.  This sets the global

are writing a new service that processes requests, or discover a request

langauge for all the functions in the same thread that are called below

service that was missed, please do this.  The CIMOperationRequestDispatcher

handleEnqueue.  If you are writing a new service that processes

service is an example.

requests, or discover a request service that was missed, please do

this.  The CIMOperationRequestDispatcher service is an example.

How to Throw a Localized Exception from Server code:

How to Throw a Localized Exception from Server code:

With all that background, here is how code running in a Pegasus service

can throw a localized exception:

With all that background, here is how code running in a Pegasus

This example assumes that the top-level code in the service had set

service can throw a localized exception:

the global thread language beforehand.  As described above, every

This example assumes that the top-level code in the service had set the

service in Pegasus should do that.

global thread AcceptLanguages beforehand.  As described above,

// First, construct a MessageLoaderParms

every service in Pegasus should do that.  The code here may be

//

buried several layers deep in the call chain, but does not need to know

// Notes:

the AcceptLanguage of the current client request.

//  1) The errorMessageID must be in the Pegasus server resource

// First, construct a MessageLoaderParms

bundle.

//

//  2) The default message is the old "hardcoded" message.

// Notes:

//  3) The MessageLoaderParms will default to use the Pegasus

//  1) The errorMessageID must be in the Pegasus server resource

server resource bundle

bundle.

//  4) The MessageLoaderParms will default to use the locale of

//  2) The default message is the old "hardcoded" message.

the current thread.  Don't change this!

//  3) The MessageLoaderParms will default to use the Pegasus

//  5) You might need to set the arguments for the message into

server resource bundle

the MessageLoaderParms

//  4) The MessageLoaderParms will default to use the

MessageLoaderParms parms("errorMessageID", "default message");

AcceptLanguages set into the current Thread.  Don't change this!

// Second, throw the Exception

//  5) You might need to set the arguments for the message into

// Note: this applies to all the derived classes from Exception, including

the MessageLoaderParms

the CIMException's

MessageLoaderParms parms("errorMessageID", "default message");

throw new Exception(parms);

// Second, throw the Exception

// Note: this applies to all the derived classes from Exception,

including the CIMException's

throw new Exception(parms);

NOTE:  If you are throwing an Exception with un-localized data,

use the constructor that takes a String.  An example of this would

be an Exception where you are passing in a file name.  Most of the

"non-CIM" exceptions defined in Exception.h and InternalException.h take

un-localized data.

un-localized data.

How to Load a Localized Message

The Exception Macros

For code that may not be running in a Thread with the global

language set, but has access to the AcceptLanguages object from the CIMMessage,

There are many spots in the server code that use the

the code is simple:

PEGASUS_CIM_EXCEPTION macro to throw a TraceableCIMException.  The

// Construct a MessageLoaderParms

use of this macro in the code like the following example presented a

//

design problem:

// Notes:

....

//  1) The errorMessageID must be in the Pegasus server resource

} catch (Exception & e)

bundle.

{

//  2) The default message is the old "hardcoded" message.

    throw PEGASUS_CIM_EXCEPTION(CIM_ERR_FAILED,

//  3) The MessageLoaderParms will default to use the Pegasus

e.getMessage());

server resource bundle

}

//  4) The MessageLoaderParms will default to use the locale of

the current thread.  You will change this below.

This type of code would have lost the ContentLanguages saved in "e",

//  5) You might need to set the arguments for the message into

so that the Content-Language would not be set in HTTP response to the

the MessageLoaderParms

client.

MessageLoaderParms parms("errorMessageID", "default message");

// Tell the MessageLoaderParms which languages to search for.

For Pegasus 2.3, these types of macro calls can stay.  The

// MessageLoaderParms will not use the thread locale in this case.

TraceableCIMException constructed by the macro will "re-localize". 

parms.acceptlanguages = <pass in the AcceptLanguages object>

That is, the "CIM" part of the message (the part based on the error

// Load the localized String

code) will be localized at throw time, and the ContentLanguages

String localizedMsg = MessageLoader::getMessage(parms);

re-established.  A key is to avoid a "language mismatch" problem

between the CIM part of the message and the extra part of the

<h3>

message.  The design point here is that all internal exceptions

5.2 Logger Messages</h3>

thrown by Pegasus code are localized using the global AcceptLanguages

of the Thread...see above.

New methods have been added to Logger to take a message ID of a

message to be loaded from the Pegasus server resource bundle.  The

In the future, it will be safer and more maintainable to use of

caller is only required to pass in the message ID, and the old "hardcoded"

the  new "localized" flavors of the macro.  For example:

message, and the args.  The Logger will use MessageLoader to load

the message in the locale of the Pegasus server process, using the

When the message from a caught  Exception needs to be become

hardcoded message as the default string.  Please refer to pegasus/src/Pegasus/Logger.h

the extra message in a thrown CIMException:

....

<h3>

} catch (Exception & e)

5.3 CLI Messages</h3>

{

        throw

Code in the client side of the client/server CLIs (eg. cimconfig,

PEGASUS_CIM_EXCEPTION_LANG(e.getContentLanguages( ),

cimmof), or in directly linked CLIs (cimmofl), should use the useProcessLocale

setting in MessageLoaderParms.  This will cause the messages to be

CIM_ERR_FAILED,

loaded in the locale based on the environment in which the program is running. 

This locale can be set by the user before running the program.

e.getMessage( ));

TODO - describe how CIMClient will default the Accept-Language

}

from the process locale.

This guarantees that the ContentLanguages in "e" is copied to the

newly created TraceableCIMException.

In the case where the extra message for the CIMException is

determined by the throwing code:

throw PEGASUS_CIM_EXCEPTION_L(CIM_ERR_FAILED,

MessageLoaderParms("Repository.CIMRepository.COMPACT_FAILED", 

"compact failed"));

(example from CIMRepository.cpp)

This uses a MessageLoaderParms object to localize the extra message in

the newly created TraceableCIMException.

<h3> 5.2 Logger Messages</h3>

New methods have been added to Logger to take a message ID of a message

to be loaded from the Pegasus server resource bundle.  The caller

is only required to pass in the message ID, the old "hardcoded" message,

and the args.  The Logger will use MessageLoader to load the

message in the locale of the Pegasus server process, using the

hardcoded message as the default string.  Please refer to

pegasus/src/Pegasus/Logger.h.

Note:  Messages sent to the "logs", whether the system logs or

the Pegasus log file, are converted to UTF-8 before being sent.

<h3> 5.3 CLI Messages</h3>

The goal for messages returned by the Pegasus CLIs is to localize in

the locale of the user running the CLI.  This should be automatic

-- the user should not be required to tell the CLI what the locale

is.   For the CLIs that are CIM clients (cimconfing,

cimprovider) there are two sets of messages to localize  --

messages generated in the CLI process itself, and messages returned from

the Pegasus server .  For CLIs that are directly linked into

Pegasus (cimmofl), all the messages are generated in the CLI's process,

but the CLI may call Pegasus APIs that are coded to localize based on a

client's requested languages.

Code in the client side of the client/server CLIs (eg. cimconfig,

cimmof), or in directly linked CLIs (cimmofl), should use the

_useProcessLocale "master switch" described in the Message Loading

section.  This will cause all messages, including exceptions thrown

by Pegasus APIs,  to be loaded in the locale based on the

environment in which the program is running.  This locale can be

set by the user before running the program.

Code in the client side of the client/server CLIs need to send an

Accept-Language to the Pegasus server that reflects the default locale

of the CLI's process.  See the Client Developer section for

details.

An example of these considerations can be seen in the source code

for cimconfig.

<hr>

L.P.; IBM Corp.; The Open Group

Company, L.P.; IBM Corp.; The Open Group

Permission is hereby granted, free of charge, to any person obtaining

Permission is hereby granted, free of charge, to any person

a copy  of this software and associated documentation files (the "Software"),

obtaining a copy  of this software and associated documentation

to deal in the Software without restriction, including without limitation

files (the "Software"), to deal in the Software without restriction,

the rights to use, copy, modify, merge, publish, distribute, sublicense,

including without limitation the rights to use, copy, modify, merge,

and/or sell copies of the Software, and to permit persons to whom the Software

publish, distribute, sublicense, and/or sell copies of the Software, and

is furnished to do so, subject to the following conditions:

to permit persons to whom the Software is furnished to do so, subject

THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE SHALL BE INCLUDED

to the following conditions:

IN ALL COPIES OR SUBSTANTIAL PORTIONS OF THE SOFTWARE. THE SOFTWARE IS

THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE SHALL BE

PROVIDED  "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,

INCLUDED IN ALL COPIES OR SUBSTANTIAL PORTIONS OF THE SOFTWARE. THE

INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS

SOFTWARE IS PROVIDED  "AS IS", WITHOUT WARRANTY OF ANY KIND,

FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS

EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF

OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,

MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY

OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,

SOFTWARE.

TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE

SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

</body>

</html>

Legend:

Removed from v.1.1.2.1
changed lines
	Added in v.1.6.38.1

No CVS admin address has been configured