pegasus/doc/Globalization_HOWTO.htm - diff

Return to Globalization_HOWTO.htm CVS log

Up to [Pegasus] / pegasus / doc

Diff for /pegasus/doc/Globalization_HOWTO.htm between version 1.1.2.1 and 1.1.2.2

version 1.1.2.1, 2003/07/16 21:45:25

version 1.1.2.2, 2003/08/13 19:39:49

Line 2

<html>

<head>

</head>

<center>Globalization HOWTO

Release: Pegasus 2.3

Author: Chuck Carmack (carmack@us.ibm.com)

July 2, 2003</center>

July 28, 2003</center>

NOTE:  THIS IS A WORK-IN-PROGRESS

<h2>

1.0 Introduction</h2>

Line 107

Line 96

</ul>

Please refer to PEPs 56 and 58 for details about the globalization

support in Pegasus 2.3.

design in Pegasus 2.3.

This document provides a HOWTO guide to be used by developers to globalize

code that is being added to Pegasus.  The audience for this document

Line 233

Line 222

repository in Pegasus 2.3.  The #pragma locale, #pragma instancelocale,

and translatable qualifier flavor are not supported in the Pegasus 2.3

MOF compiler.  From the client perspective, classes, qualifiers, and

instances stored in the repository as not tagged with a language. 

instances stored in the repository are not tagged with a language. 

The Accept-Language and Content-Language headers will be ignored for repository

operations.  However, since the repository will support UTF-8, 

characters for any language may be stored there.

Line 292

Line 281

<h4>

2.2.2 Message Loading</h4>

2.2.2 Message Bundles</h4>

One of the goals of globalization for Pegasus 2.3 is the extraction

of hardcoded messages  into external message files, and loading messages

of hardcoded messages  into translated message files, loading translated

from those files.  The topics in this section are:  how to create

messages from those files, and returning those messages to the client. 

message files, and how to load messages.

The topics to be discussed here are:  how to create message files,

how to compile message files, and how to load messages into Pegasus.

At the time of writing, the message loading function in Pegasus 2.3

used the International Components for Unicode (<a href="http://oss.software.ibm.com/icu">ICU)</a>

Line 308

Line 298

to load the messages, ICU requires that the resource bundles are compiled

into a binary form (.res file) using their genrb tool.

Platform Maintainers Note:  Please refer to PEP 58 for information

about how to build Pegasus to use the ICU libraries.

The documentation for ICU resource bundles is in the <a href="http://oss.software.ibm.com/icu/userguide/ResourceManagement.html">Resource

Management</a>  section of the <a href="http://oss.software.ibm.com/icu/userguide/">ICU

User Guide</a> .  This section will tell you how to

create, organize, and compile your resource bundles for different languages.

create and organize your resource bundles for different languages.

Note:  your resource bundles should be organized in a tree structure

similiar to the one shown in the Resource Management section, including

the empty bundles in the tree.

the empty bundles in the tree.  It is recommended that you ship a

root resource bundle to be used as the fallback in case the client requests

a language that you are not supporting.

NOTE:  Pegasus 2.3 only supports simple string resources in the

ICU resource bundles.  String resources may only be loaded by key. 

Tables, arrays, and other complex resource types, are not supported.

Code that needs to load a message in Pegasus does not call ICU directly.

In order to compile your resource bundles, support has been added to

the Pegasus make files to run genrb.  A new make target, "messages",

has been added that will call genrb and put the compiled bundles (.res)

in a directory of your choosing.  An example of ICU resource bundles

and the make files to compile them are located in:

<ul>

<li>

pegasus/src/Providers/sample/LocalizedProvider/Makefile (just causes the

make to recurse to the msg sub-directory)</li>

<li>

pegasus/src/Providers/sample/LocalizedProvider/msg/Makefile (compiles the

bundles in the msg/ directory)</li>

<li>

pegasus/src/Providers/sample/LocalizedProvider/msg/*.txt (the resource

bundles to compile, using the recommended ICU language tree structure)</li>

</ul>

NOTE:  At the time of writing, only the Linux make files have

been updated to compile ICU resource bundles.

It is important to place the compiled resource bundles in a directory

where your code can find them .  The make files above compile the

resource bundles into $PEGASUS_HOME/msg/provider/localizedProvider. 

The code that loads these messages uses the MessageLoader class (next section)

to load messages from this directory.

<h4>

2.2.3 Message Loading</h4>

Code that needs to load a message in Pegasus does not call ICU directly.

Two message loading classes were added for Pegasus 2.3:  MessageLoader

and MessageLoaderParms.  These classes are abstractions designed to

hide of the actual loader used.   The MessageLoader is used to

hide of the actual loader used (but note that at the time of writing, only

load a message using a list of preferrred languages.  The parameters

ICU is supported).   The MessageLoader is used to load a message

to MessageLoader are encapsulated in a MessageLoaderParms object.

using a list of preferrred languages.  The parameters to MessageLoader

are encapsulated in a MessageLoaderParms object.

The MessageLoader is the place where the Accept-Language header, Content-Language

header, and the ICU resource bundles, join up.  The MessageLoader

class is designed to receive an AcceptLanguages object, and a set of parameters

indicating the bundle base-name and message ID to use.  The AcceptLanguages

object contains the list of requested languages sent by the client. 

The MessageLoader searches for the message in the set of bundles named

with the base-name, using the AcceptLanguages for the list of specific

translated bundles to search.  The MessageLoader returns the message

that it found, along with a ContentLanguages object indicating the language

of the message.  The ContentLanguages object should be used to indicate

the language of the response sent back to the client.

The MessageLoaderParms object contains the parameters to load the message.

There are many parameters, but many can be allowed to default.  Here

is a description of the parameters:

NOTE:  WORK-IN- PROGRESS

<tr>

Line 351

Line 390

<td>Input.

Required</td>

<td>Message to return if the no message can be loaded for msg_id from a

<td>Message to return if the no message can be loaded for msg_id from any

resource bundle.  Note:  The args parameters below are substituted

into this string. 

Note:  For the args into this  string, use the Pegasus '$'

Line 367

Line 406

Default: $PEGASUS_HOME/msg/pegasus/pegasusServer</td>

<td>Path to the root resource bundle file which contains the msg_id.

Do not include the language or file extension as part of the path.

Note: Only specify the path down to the bundle base-name.  Do

Note: relative paths start at $PEGASUS_HOME/msg. </td>

not append a language tag, such as "_root" or "_en".  Do not append

a file extension.

Note: relative paths start at $PEGASUS_HOME/msg.

Note: defaults to the bundle containing the Pegasus server messages.</td>

</tr>

<tr>

Line 380

Line 422

<td>Contains the list of preferred languages, in priority order.

This is combined with msg_src_path to determine which resource bundles

to search for for the msg_id.   If not EMPTY, overrides useThreadLocale

to search for for the msg_id.   If not empty, overrides useThreadLocale

and useProcessLocale.</td>

</tr>

Line 410

Line 452

Optional

Default = true</td>

<td>If true, MessageLoader will use the locale of the caller's thread. </td>

<td>If true, MessageLoader will use the AcceptLanguages set by Pegasus

into the caller's Thread.   See the Note below for details. </td>

</tr>

<tr>

Line 447

Line 490

</tr>

</table>

Notes:

The "useThreadLocale" parameter defaults to true.  This flag indicates

to use the AcceptLanguages object set by Pegasus into the Pegasus Thread

in which the caller's code is running.  This AcceptLanguages object

reflects the languages requested by the client.  This is useful for

code that may not have access to the AcceptLanguages from the client. 

Pegasus sets this AcceptLanguages object into the Thread of providers and

internal Pegasus code.  For this reason, it is recommended that provider

and internal Pegasus code use the "useThreadLocale" flag instead of explicity

passing in an AcceptLanguages object.  See the Provider Developer

and Pegasus Developer sections for details.

The "useProcessLocale" flag can be used to tell MessageLoader to use

the default locale of the process, as determined by ICU.  This is

useful for situations where the caller is not localizing for a client request. 

The caller may itself be a client (eg. cimconfig), or may need to log messages

to the system log in the locale of the Pegasus server process.  See

the CLI Messages and Logger Messages sections below.

"Master switch"

The MessageLoader class has a public static Boolean variable called

_useProcessLocale that may be used to override all the AcceptLanguages

and useThreadLocale settings in the MessageLoaderParms objects passed in. 

This is useful for CLI code (eg cimconfig) that needs to localize its messages

based on the locale of its process, which refects the locale set by the

user running the CLI (eg. $LANG on Unix).  The CLI code may call Pegasus

APIs that are coded to use the Thread's AcceptLanguages, which will not

be set in this case.  The _useProcessLocale static variable tells

the MessageLoader to ignore the AcceptLanguages, useThreadLocale, and useProcessLocale

settings in MessageLoaderParms that it gets.  The MessageLoader will

use the default process locale, as determined by ICU, in this case.

Important Note:  The MessageLoader defaults to not use

the "fallback" mechanism described in the ICU Resource Management section. 

This is because the Accept-Language header itself describes the fallback

that the client wants.  However, the MessageLoader does "fallback"

to the root resource bundle if none of the languages in AcceptLanguages

can be found.  If the root resource bundle cannot be found, then the

default_msg is returned.  The "useICUFallback" flag can be set to

have MessageLoader use ICU fallback on all message load attempts. 

However, usage of this flag for client requests may lead to incorrect results. 

For example, a client sets Accept-Language to french, german, and spanish,

in that order, but there is no french resource bundle.  A call to

MessageLoader with useICUfallback == true would cause the root resource

bundle string to be returned on the attempt to load from the french bundle. 

But the client requested german to be the fallback after french.

Please refer to the following files for details on the new Pegasus classes.

<ul>

Line 455

Line 546

</ul>

<h4>

2.2.3 Message Loading Example</h4>

2.2.4 Message Loading Example</h4>

The following example shows how a message may be loaded using the

classes described above.  Note: this a generic example.  Each

of the developer sections below have 'real-life' examples that are better

suited to each type of code.

The following example shows how a message may be loaded using the classes

described above.  Note: this a generic example.  Each of the

developer sections below have 'real-life' examples that are better suited

to each type of code.

// Build an AcceptLanguages with some language elements

AcceptLanguages acceptLangs;

acceptLangs.add(AcceptLanguageElement("fr", 0.5));

Line 512

Line 592

TODO - find a good message writing guide to link to</li>

</ul>

<h4>

2.2.5 Localized Exceptions</h4>

The base Exception class, and derived classes, have been updated

to support localization.  Constructors have been added that take a

MessageLoaderParms object.  These constructors will use the MessageLoaderParms

object to call the MessageLoader to load the localized exception message. 

The localized message is saved in the Exception.  The ContentLanguages

object returned by MessageLoader is also saved in the Exception. 

This indicates the language of the message.  The ContentLanguages

object is used later to set the Content-Language header in the HTTP message

to the client.

The old Exception constructors that take a String will remain.

These should be used in cases where the code throwing the exception is

not localized, or the String is not localized (for example, a file name). 

Also, there are several exceptions in Pegasus where the String parameter

is meant to be a non-localized substitution in a localized message owned

by the Exception (see InternalException.h, ClassNotResolved for an example). 

The old constructors for these have been kept.

<h2>

3.0 Provider Developers</h2>

Line 777

Line 879

Methods have been added to CIMClient to set the Accept-Language

and Content-Language on the request, and retrieve Content-Language on the

response.

response.  The language tags in the Accept-Language header must meet

the ISO-639 and ISO-3166 standards.

Please refer to

Line 849

Line 952

a platform codepage.  This is especially needed for EBCDIC platforms. 

See the Provider developer section for details of the EBCDIC considerations.

TODO - some info on how CIMClient defaults the Accept-Languages.

<h3>

4.1 Default Process Locale</h3>

A method has been added to CIMClient to set the Accept-Language

for the requests based on the default locale of the process, as determined

by ICU.  If ICU is installed on the client system then CIMClient will

set the Accept-Language from the default ICU process locale.  If ICU

is not installed then the caller is required to set an AcceptLanguages

into CIMClient that meets the ISO-639 and IS0-3166 standards.  Note: 

this is useful for local clients, such as the Pegasus CLIs, where ICU would

be installed on both the client and server sides.

<h2>

5. 0 Pegasus Developers</h2>

Line 868

Line 983

<ul>

<li>

pegasus/src/Pegasus/msg/Server/pegasusServer.txt  for server messages</li>

pegasus/src/Pegasus/msg/Server/pegasusServer_*.txt  for server and

MOF compiler (cimmof, cimmofl) messages</li>

<li>

pegasus/src/Clients/<cli_name>/msg/<cli_name>.txt for CLI messages</li>

pegasus/src/Pegasus/msg/CLI/pegasusCLI_*.txt for all CLI messages (except

the MOF compiler)</li>

</ul>

Note:  As described above, the resource bundle path in MessageLoaderParms

The make messages target will compile these resource bundles.

Note:  As described above, the resource bundle path in MessageLoaderParms

defaults to the server resource bundle.  For CLI messages, you will

need to specify the bundle for your CLI.

Line 910

Line 1029

To enable exceptions to be localized, the ability was added to set a

global language for all the code running from a Pegasus Thread object. 

The top level code for a Thread can set a global AcceptLanguages object

that can accessed by all the low-level functions that it calls.  This

that can be accessed by all the low-level functions that it calls. 

will allow an exception thrown by low-level code to be localized based

This will allow an exception thrown by the low-level function to be localized

on this global AcceptLanguages object.  Note:  This applies only

based on this global AcceptLanguages object.  Note:  This applies

to Threads that are managed by a ThreadPool.

only to Threads that are managed by a ThreadPool.

Each service in the request path of the Pegasus server sets the AcceptLanguages

into its Thread from the AcceptLanguages in the CIMRequestMessage object

Line 928

Line 1047

With all that background, here is how code running in a Pegasus service

can throw a localized exception:

This example assumes that the top-level code in the service had set

the global thread language beforehand.  As described above, every

the global thread AcceptLanguages beforehand.  As described above,

service in Pegasus should do that.

every service in Pegasus should do that.  The code here may be buried

several layers deep in the call chain, but does not need to know the AcceptLanguage

of the current client request.

// First, construct a MessageLoaderParms

//

// Notes:

Line 938

Line 1059

//  2) The default message is the old "hardcoded" message.

//  3) The MessageLoaderParms will default to use the Pegasus

server resource bundle

//  4) The MessageLoaderParms will default to use the locale of

//  4) The MessageLoaderParms will default to use the AcceptLanguages

the current thread.  Don't change this!

set into the current Thread.  Don't change this!

//  5) You might need to set the arguments for the message into

the MessageLoaderParms

MessageLoaderParms parms("errorMessageID", "default message");

Line 954

Line 1075

"non-CIM" exceptions defined in Exception.h and InternalException.h take

un-localized data.

How to Load a Localized Message

The Exception Macros

For code that may not be running in a Thread with the global

language set, but has access to the AcceptLanguages object from the CIMMessage,

There are many spots in the server code that use the PEGASUS_CIM_EXCEPTION

the code is simple:

macro to throw a TraceableCIMException.  The use of this macro in

// Construct a MessageLoaderParms

the code like the following example presented a design problem:

//

....

// Notes:

} catch (Exception & e)

//  1) The errorMessageID must be in the Pegasus server resource

{

bundle.

throw PEGASUS_CIM_EXCEPTION(CIM_ERR_FAILED, e.getMessage());

//  2) The default message is the old "hardcoded" message.

}

//  3) The MessageLoaderParms will default to use the Pegasus

server resource bundle

This type of code would have lost the ContentLanguages saved in "e",

//  4) The MessageLoaderParms will default to use the locale of

so that the Content-Language would not be set in HTTP response to the client.

the current thread.  You will change this below.

//  5) You might need to set the arguments for the message into

For Pegasus 2.3, these types of macro calls can stay.  The TraceableCIMException

the MessageLoaderParms

constructed by the macro will "re-localize".  That is, the "CIM" part

MessageLoaderParms parms("errorMessageID", "default message");

of the message (the part based on the error code) will be localized at

// Tell the MessageLoaderParms which languages to search for.

throw time, and the ContentLanguages re-established.  A key is to

// MessageLoaderParms will not use the thread locale in this case.

avoid a "language mismatch" problem between the CIM part of the message

parms.acceptlanguages = <pass in the AcceptLanguages object>

and the extra part of the message.  The design point here is that

// Load the localized String

all internal exceptions thrown by Pegasus code are localized using the

String localizedMsg = MessageLoader::getMessage(parms);

global AcceptLanguages of the Thread...see above.

In the future, it will be safer and more maintainable to use of the

new "localized" flavors of the macro.  For example:

When the message from a caught  Exception needs to be become the

extra message in a thrown CIMException:

....

} catch (Exception & e)

{

throw PEGASUS_CIM_EXCEPTION_LANG(e.getContentLanguages(

CIM_ERR_FAILED,

e.getMessage( ));

}

This guarantees that the ContentLanguages in "e" is copied to the newly

created TraceableCIMException.

In the case where the extra message for the CIMException is determined

by the throwing code:

throw PEGASUS_CIM_EXCEPTION_L(CIM_ERR_FAILED,

MessageLoaderParms("Repository.CIMRepository.COMPACT_FAILED",  "compact

failed"));

(example from CIMRepository.cpp)

This uses a MessageLoaderParms object to localize the extra message

in the newly created TraceableCIMException.

<h3>

5.2 Logger Messages</h3>

New methods have been added to Logger to take a message ID of a

message to be loaded from the Pegasus server resource bundle.  The

caller is only required to pass in the message ID, and the old "hardcoded"

caller is only required to pass in the message ID, the old "hardcoded"

message, and the args.  The Logger will use MessageLoader to load

the message in the locale of the Pegasus server process, using the

hardcoded message as the default string.  Please refer to pegasus/src/Pegasus/Logger.h

hardcoded message as the default string.  Please refer to pegasus/src/Pegasus/Logger.h.

Note:  Messages sent to the "logs", whether the system logs or

the Pegasus log file, are converted to UTF-8 before being sent.

<h3>

5.3 CLI Messages</h3>

Code in the client side of the client/server CLIs (eg. cimconfig,

The goal for messages returned by the Pegasus CLIs is to localize

cimmof), or in directly linked CLIs (cimmofl), should use the useProcessLocale

in the locale of the user running the CLI.  This should be automatic

setting in MessageLoaderParms.  This will cause the messages to be

-- the user should not be required to tell the CLI what the locale is.  

loaded in the locale based on the environment in which the program is running. 

For the CLIs that are CIM clients (cimconfing, cimprovider) there are two

This locale can be set by the user before running the program.

sets of messages to localize  -- messages generated in the CLI process

TODO - describe how CIMClient will default the Accept-Language

itself, and messages returned from the Pegasus server .  For CLIs

from the process locale.

that are directly linked into Pegasus (cimmofl), all the messages are generated

in the CLI's process, but the CLI may call Pegasus APIs that are coded

to localize based on a client's requested languages.

Code in the client side of the client/server CLIs (eg. cimconfig, cimmof),

or in directly linked CLIs (cimmofl), should use the _useProcessLocale

"master switch" described in the Message Loading section.  This will

cause all messages, including exceptions thrown by Pegasus APIs, 

to be loaded in the locale based on the environment in which the program

is running.  This locale can be set by the user before running the

program.

Code in the client side of the client/server CLIs need to send an Accept-Language

to the Pegasus server that reflects the default locale of the CLI's process. 

See the Client Developer section for details.

An example of these considerations can be seen in the source code for

cimconfig.

<hr>

Legend:

Removed from v.1.1.2.1
changed lines
	Added in v.1.1.2.2

No CVS admin address has been configured