Change History:

1.0 Introduction

As part of the Pegasus 2.3 release, functions were added for globalization support.   Globalization involves two major aspects:  internationalization and localization.
 

Internationalization is the process of writing a program that is locale-neutral.  In other words, the program should be able to run in any locale without change.  There are several categories in a locale, including the language of message strings, date format, time format, etc.  For release 2.3, the Pegasus server is concerned with the language of the message strings it returns to its clients.
 

To support internationalization, a program is designed to do the following:
 

Support character sets that can represent customer data in any language.  Typically, the program supports some variation of Unicode for internal data.  There is usually some conversion between the supported character sets for external data, and the internal character set.  Since Unicode covers all characters, and usually has converters on the platform, it is a good choice for the 'normalized' internal character set.    The most 'interoperable' solution for external data is to support UTF-8 (eg. network and file system data).  The internal data is usually UTF-16 (or UCS-2, but that is deprecated).

 

Extract locale-sensitive resources, such as message strings, from the code to external resource files.  Typically, the resources are loaded based on the locale requested by the end-user, and returned to the end-user for display.

Localization is the process of customizing a software product to support particular locales.  For example, a product that is internationalized might want to only localize for certain countries.  This would mean that the localized resources (eg. message files) would only be translated and shipped for the countries that the product supports.  Since the code for the product is locale-neutral, it will be easy to drop in new translations as more countries are supported.
 

The Pegasus 2.3 release added support for globalization.  At a high-level, the following additions were made to Pegasus 2.3:
 

• Support UTF-8 for external data.

 
• The CIM-XML documents contained in the HTTP messages
• The files in the repository
• Note:  Pegasus 2.3 does NOT support UTF-8 in the MOF files

 
• Support UTF-16 for internal data.

 
• Extract the hardcoded messages from the Pegasus code into message files.  An API was added to load messages from the message files.

 
• APIs were added for clients to associate a language with the CIM objects they are sending to Pegasus.  Also, APIs were added for clients to determine the language of the error message or CIM object that Pegasus returns.

 
• APIs were added for providers to determine the language of CIM objects sent by the client.  Also, APIs were added for providers to associate a language with the CIM object, or error message, they return to the client.

Please refer to PEPs 56 and 58 for details about the globalization design in Pegasus 2.3.
 

This document provides a HOWTO guide to be used by developers to globalize code that is being added to Pegasus.  The audience for this document are:
 

• Provider developers - both CMPI and C++
• Client developers
• Pegasus developers

The quickest way to approach this document is to read the General section, and then the developer section that relates to what you are doing.
 

2.0 General

2.1 Unicode Support

Pegasus 2.3 supports Unicode throughout the processing of requests.  External data to Pegasus is encoded in UTF-8.  Internal data is encoded in UTF-16.
 

UTF-8 support for external data includes the CIM-XML messages passed over the network, and the repository files.  Note:  UTF-8 support was NOT added to the MOF Compiler for MOF files in release 2.3.  For the CIM-XML messages, Pegasus follows section 4.8 of the  CIM-HTTP specification    Specifically, Pegasus supports the "utf-8" setting for the charset parameter of the Content-Type header and the XML encoding attribute.  If no charset is specified, the 7-bit ASCII is assumed.
 

The internal support of UTF-16 is encapsulated in the Pegasus String class.  This class has been updated to contain UTF-16 characters.  Specifically, the Char16 objects inside the String contain UTF-16 characters.  Note: a UTF-16 surrogate pair is contained in two consecutive Char16 objects.  To keep backwards compatibilty, the methods on the String class have not changed.  New methods have been added as needed.  The following describes this in more detail:

• The Pegasus 2.2 methods that take a char *, or return char *, are unchanged.  Code written to Pegasus 2.2 may have expected to store 8-bit ASCII (ISO-8859-1) characters into String.  These methods will convert the input to UTF-16 from 8-bit ASCII.  (This is simple because UTF-16 is a superset of 8-bit ASCII - simply need to prepend '\0' to each char).  The Pegasus 2.2 methods that return char data will attempt to convert from the UTF-16 internal representation to 8-bit ASCII.  Characters that cannot be converted will be replaced with a substitution character.

 
• All methods that take or return Char16 data are unchanged.  The String class now supports UTF-16 data in Char16, although surrogate pairs will require two consecutive Char16 objects.  The String class does NO checking for unmatched surrogate pairs.

 
• New methods have been added to take and return UTF-8 data.  The String class will convert between UTF-8 and the UTF-16 internal representation as needed.  These new methods will use char * parameters, but will be clearly labelled as UTF-8 methods.

 

2.2 Localization Support

2.2.1 Language Headers

Pegasus 2.3 supports clients and providers that wish to localize.  There are two areas to be localized:  ERROR  elements in the CIM-XML; and  Object Definition  elements in the CIM-XML.  Clients can request  the server to return error messages and CIM objects in a set of languages of their choosing.  Clients can also tag a language to the CIM objects they are sending to the server.  Providers and the server can return error messages and CIM objects that are tagged with one of  languages requested by the client.
 

The localization design is based on section 4.8 of the CIM-HTTP specification , which refers to RFC 2616.  The method used to tag a language to the CIM-XML is through the Accept-Language and Content-Language HTTP headers.  These headers are basically lists of language tags.  An HTTP request can contain an Accept-Language header, which indicates the list of preferred languages that the client wants in the response.  This list can be prioritized by using the quality numbers.  An HTTP request or response can contain a Content-Language header, which indicates the language(s) of the content in the message.  In the Pegasus case, this would be the CIM-XML.  Note that the Content-Language header is a list of language tags.  This allows the content of an HTTP message to contain more than one translation.  However, in the Pegasus case, there is only one CIM-XML document in the HTTP message, and thus one translation.
 

CIM clients may use the Accept-Language HTTP header to specify the languages they wish to be returned in the CIM response message.  CIM clients may also use the Content-Language header to tag the language of any CIM objects they are sending to the server in the CIM request message.  The server, and providers, should attempt to return error messages and CIM objects in one of the accept languages requested by the client.  The server and providers should set the Content-Language header in the CIM response message to indicate which of the requested languages they are returning.
 

NOTE:  Localization support was not added for the MOF files and repository in Pegasus 2.3.  The #pragma locale, #pragma instancelocale, and translatable qualifier flavor are not supported in the Pegasus 2.3 MOF compiler.  From the client perspective, classes, qualifiers, and instances stored in the repository are not tagged with a language.  The Accept-Language and Content-Language headers will be ignored for repository operations.  However, since the repository will support UTF-8,  characters for any language may be stored there.
 

NOTE:  Since the Content-Language header applies to the entire HTTP message, it applies to the entire CIM-XML document.  This includes all the objects in the document, including enumerated objects, and all the values in the objects.  This is a limitation that will remain until the CIM standard has been updated to support language tags tied to individual CIM values.  From the client perspective, it is possible for Pegasus to send a CIM response with NO Content-Language, even if the client had sent Accept-Language.   This can happen if Pegasus does not know the language of the response.  An example is a request that was sent to a Pegasus 2.2 provider.  Another example is an enumerated response where each provider returned a different language.  Please refer to PEP58 for details on these provider scenarios.
 

The Accept-Language and Content-Language headers are encapsulated in AcceptLanguageList and ContentLanguageList classes, respectively. These classes contain LanguageTag objects. The AcceptLanguageList class keeps its LanguageTags prioritized based on quality, according to RFC 2616.
 

AcceptLanguageList and ContentLanguageList are the objects used by code throughout the request/response processing, from the client to the server to the providers and back.  The server handles the creation of these objects from the HTTP headers.  Code at each point in the process will have access to these objects.
 

Please refer to the following files for details on the Pegasus language interfaces.
 

• pegasus/src/Pegasus/Common/AcceptLanguageList.h
• pegasus/src/Pegasus/Common/ContentLanguageList.h
• pegasus/src/Pegasus/Common/LanguageTag.h

See the sections below for details on how to write clients and providers to use these classes.
 
 

2.2.2 Message Bundles

One of the goals of globalization for Pegasus 2.3 is the extraction of hardcoded messages  into translated message files, loading translated messages from those files, and returning those messages to the client.  The topics to be discussed here are:  how to create message files, how to compile message files, and how to load messages into Pegasus.
 

At the time of writing, the message loading function in Pegasus 2.3 used the International Components for Unicode (ICU) libraries.  This is expected to be the future direction for Pegasus. ICU uses a resource bundle format for their message files.   In order to load the messages, ICU requires that the resource bundles are compiled into a binary form (.res file) using their genrb tool.
 

Platform Maintainers Note:  Please refer to PEP 58 for information about how to build Pegasus to use the ICU libraries.
 

The documentation for ICU resource bundles is in the Resource Management  section of the ICU User Guide .  This section will tell you how to create and organize your resource bundles for different languages.  Note:  your resource bundles should be organized in a tree structure similiar to the one shown in the Resource Management section, including the empty bundles in the tree. 

It is recommended that you ship a root resource bundle to be used as the fallback in case the client requests a language that you are not supporting.  The Pegasus make files are set up to automatically create and compile a root resource bundle for you.  For Pegasus 2.3, the make will use your "en" bundle, upper case all the messages, and then put the uppercased messages into the root bundle.  The uppercasing of the messages is necessary to create a "fallback" root bundle that contains invariant characters across all EBCDIC and ASCII codepages.
 

NOTE:  When creating your resource bundles, the name of the table resource should not contain the package name.    For example, if you
have a bundle with a package name of "xyz", then the "en" bundle should start like this:

en:table {
..... messages here
               }

not like this:

xyz_en:table {
..... messages here
               }
 

This is needed because the package name (-p) option is used by the Pegasus make files on the call to genrb.
 

NOTE:  Pegasus 2.3 only supports simple string resources in the ICU resource bundles.  String resources may only be loaded by key.  Tables, arrays, and other complex resource types, are not supported.
 

In order to compile your resource bundles, support has been added to the Pegasus make files to run genrb.  A new make target, "messages", has been added that will call genrb and put the compiled bundles (.res) in a directory of your choosing.  An example of ICU resource bundles and the make files to compile them are located in:
 

• pegasus/src/Providers/sample/LocalizedProvider/Makefile (just causes the make to recurse to the msg sub-directory)
• pegasus/src/Providers/sample/LocalizedProvider/msg/Makefile (compiles the bundles in the msg/ directory)
• pegasus/src/Providers/sample/LocalizedProvider/msg/*.txt (the resource bundles to compile, using the recommended ICU language tree structure)

NOTE:  At the time of writing, only the Linux make files have been updated to compile ICU resource bundles.
 

It is important to place the compiled resource bundles in a directory where your code can find them .  The make files above compile the resource bundles into $PEGASUS_HOME/msg/provider/localizedProvider.  The code that loads these messages uses the MessageLoader class (next section) to load messages from this directory.
 
 

2.2.3 Message Loading

Code that needs to load a message in Pegasus does not call ICU directly.  Two message loading classes were added for Pegasus 2.3:  MessageLoader and MessageLoaderParms.  These classes are abstractions designed to hide of the actual loader used (but note that at the time of writing, only ICU is supported).   The MessageLoader is used to load a message using a list of preferrred languages.  The parameters to MessageLoader are encapsulated in a MessageLoaderParms object.
 

The MessageLoader is the place where the Accept-Language header, Content-Language header, and the ICU resource bundles, join up.  The MessageLoader class is designed to receive an AcceptLanguageList object, and a set of parameters indicating the bundle base-name and message ID to use.  The AcceptLanguageList object contains the list of requested languages sent by the client.  The MessageLoader searches for the message in the set of bundles named with the base-name, using the AcceptLanguageList for the list of specific translated bundles to search.  The MessageLoader returns the message that it found, along with a ContentLanguageList object indicating the language of the message.  The ContentLanguageList object should be used to indicate the language of the response sent back to the client.
 

The MessageLoaderParms object contains the parameters to load the message.  There are many parameters, but many can be allowed to default.  Here is a description of the parameters:
 
 

String msg_id;	Input.  Required.	Message ID  of the message to load from the resource bundle.  This is the key that ICU will use to load the message.
String default_msg;	Input.  Required	Message to return if the no message can be loaded for msg_id from any resource bundle.  Note:  The args parameters below are substituted into this string.  Note:  For the args into this  string, use the Pegasus '$' form, as described in pegasus/src/Pegasus/Common/Formatter.h.  Don't use the ICU substitution format for the default message string.
String msg_src_path;	Input.  Optional Default: $PEGASUS_HOME/msg/pegasus/pegasusServer	Path to the resource bundle file which contains the msg_id.  Note: Only specify the path down to the bundle base-name.  Do not append a language tag, such as "_root" or "_en".  Do not append a file extension. Note: relative paths start at $PEGASUS_HOME/msg.  Note: defaults to the bundle containing the Pegasus server messages.
AcceptLanguageList acceptlanguages;	Input.  Optional Default: AcceptLanguageList()	Contains the list of preferred languages, in priority order.  This is combined with msg_src_path to determine which resource bundles to search for for the msg_id.   If not empty, overrides useThreadLocale and useProcessLocale.
ContentLanguageList contentlanguages;	Output	Contains the language that MessageLoader found for the msg_id.
Boolean useProcessLocale;	Input Optional Default = false	If true, MessageLoader will use the default locale of the process.  If true, overrides useThreadLocale.
Boolean useThreadLocale;	Input Optional Default = true	If true, MessageLoader will use the AcceptLanguageList set by Pegasus into the caller's Thread.   See the Note below for details.
Formatter::Arg arg0;  Formatter::Arg arg1;  Formatter::Arg arg2;  Formatter::Arg arg3;  Formatter::Arg arg4;  Formatter::Arg arg5;  Formatter::Arg arg6;  Formatter::Arg arg7;  Formatter::Arg arg8;  Formatter::Arg arg9;	Input Optional Default: Formatter::Arg( ) // empty arg	These are the substitution variables, using the Pegasus Formatter::Arg class.

Notes:
 

The "useThreadLocale" parameter defaults to true.  This flag indicates to use the AcceptLanguageList object set by Pegasus into the Pegasus Thread in which the caller's code is running.  This AcceptLanguageList object reflects the languages requested by the client.  This is useful for code that may not have access to the AcceptLanguageList from the client.  Pegasus sets this AcceptLanguageList object into the Thread of providers and internal Pegasus code.  For this reason, it is recommended that provider and internal Pegasus code use the "useThreadLocale" flag instead of explicity passing in an AcceptLanguageList object.  See the Provider Developer and Pegasus Developer sections for details.
 

The "useProcessLocale" flag can be used to tell MessageLoader to use the default locale of the process, as determined by ICU.  This is useful for situations where the caller is not localizing for a client request.  The caller may itself be a client (eg. cimconfig), or may need to log messages to the system log in the locale of the Pegasus server process.  See the CLI Messages and Logger Messages sections below.
 

"Master switch"
The MessageLoader class has a public static Boolean variable called _useProcessLocale that may be used to override all the AcceptLanguageList and useThreadLocale settings in the MessageLoaderParms objects passed in.  This is useful for CLI code (eg cimconfig) that needs to localize its messages based on the locale of its process, which refects the locale set by the user running the CLI (eg. $LANG on Unix).  The CLI code may call Pegasus APIs that are coded to use the Thread's AcceptLanguageList, which will not be set in this case.  The _useProcessLocale static variable tells the MessageLoader to ignore the AcceptLanguageList, useThreadLocale, and useProcessLocale settings in MessageLoaderParms that it gets.  The MessageLoader will use the default process locale, as determined by ICU, in this case.
 

Important Note:  The MessageLoader does not use the "fallback" mechanism described in the ICU Resource Management section.  This is because the Accept-Language header itself describes the fallback that the client wants.  If the MessageLoader cannot find a message file for any of the languages in the AcceptLanguageList, it will try the default process locale. If this fails, the ICU root resource bundle will be tried.
 

Please refer to the following files for details on the new Pegasus classes.
 

• pegasus/src/Pegasus/Common/MessageLoader.h

2.2.4 Message Loading Example

The following example shows how a message may be loaded using the classes described above.  Note: this a generic example.  Each of the developer sections below have 'real-life' examples that are better suited to each type of code.

// Build an AcceptLanguageList with some language elements
AcceptLanguageList acceptLangs;
acceptLangs.insert(LanguageTag("fr"), 0.5);
acceptLangs.insert(LanguageTag("de"), 0.8);
acceptLangs.insert(LanguageTag("es"), 0.4);

// Construct a MessageLoaderParms
MessageLoaderParms parms("msgID", "default message");
parms. msg_src_path = "/my_msg_dir/my_bundle";
parms.acceptlanguages = acceptLangs;

// Note: If you have args, set them into MessageLoaderParms

// Load the localized String
String localizedMsg = MessageLoader::getMessage(parms);
 
 

2.2.5 Message Writing Guidelines

Here are some basic rules for writing messages:
 

• If you want to claim that you are globalized, no hardcoded messages!
• Avoid creating a message in the code by combining other messages.  When you do this you are assuming that you know the grammar for every language.
• String substitutions into messages are generally untranslated, ie. not loaded from the resource bundle.   Example: a file name.
• Avoid jargon, humour, and cultural idioms.  Use full sentences.  Have your messages reviewed by your globalization team.  Your messages need to make sense to the translators, and ultimately the customer.
• TODO - find a good message writing guide to link to

When do I create a new message ?

A new message should be created if a message is needed with a content not described by any existing message.

A new message should be created if the number or placement of substitution parameters of an existing message would require an update.

It is not necessary to create a new message if just the text of the message is changed, while the meaning is kept. For instance if the event(error,warning,whatever) is described more precisely by the new message text, it is not necessary to create a new message, but the existing one should be updated.

Are there any special considerations while creating a new message ?

• If a message definition contains text within a single quote it is not interpreted in any way.

  Example:

  Server.CIMOperationRequestAuthorizer.NOT_IN_AUTHORIZED_GRP: string {"PGS05202: User '{0}' is not authorized to access CIM data."}

  Processed message:

  User {0} is not authorized to access CIM data.

• For a single quote to appear in a processed message, it needs to be preceded by another single quote.

  Example:

  Server.CIMOperationRequestAuthorizer.NOT_IN_AUTHORIZED_GRP: string {"PGS05202: User ''{0}'' is not authorized to access CIM data."}

  Processed message:

  User 'wbemuser' is not authorized to access CIM data.

• For a double quote to appear in a processed message, it needs to be preceded by a back slash.

  Example:

  ControlProviders.ProviderRegistrationProvider.ProviderRegistrationProvider. UNSUPPORTED_USERCONTEXT_VALUE:string {"PGS03029: Unsupported UserContext value: \"{0}\"."}

  Processed message:

  Unsupported UserContext value: "10".

How do I write a platform specific message ?

Platform specific messages generate in a non-platform specific source file should be formatted with a .<platform> or .STANDARD suffix.

Example:

Compiler.cmdline.cimmof.cmdline.MENU.PEGASUS_OS_HPUX

Compiler.cmdline.cimmof.cmdline.MENU.PEGASUS_OS_OS40

Compiler.cmdline.cimmof.cmdline.MENU.STANDARD

Where should I place platform specific messages ?

As described in the message bundle file pegasusServer_en.txt messages belong into the section corresponding the file they are created in. This does account the same to platform specific messages.

If a message is generated inside a source file not specific to a single platform, the message should be part of the message bundle section of that source file.

If a new platform specific message is generated inside a platform specific source file, the message belongs to the platform specific section of the message bundle file.

Examples:

ProviderManager.ProviderAgent.ProviderAgent.UNINITIALIZED_SECURITY_SETUP.PEGASUS_OS_ZOS - this message is and should be part of the section for the ProviderAgent as it is generated inside the provider agent and not a z/OS platform specific file

Common.safCheckzOS_inline.BAD_WBEM_SECURITY_SETUP - this message does and should reside inside the platform specific section as the message is generated in a z/OS platform only file

2.2.5 Localized Exceptions

The base Exception class, and derived classes, have been updated to support localization.  Constructors have been added that take a MessageLoaderParms object.  These constructors will use the MessageLoaderParms object to call the MessageLoader to load the localized exception message.  The localized message is saved in the Exception.  The ContentLanguageList object returned by MessageLoader is also saved in the Exception.  This indicates the language of the message.  The ContentLanguageList object is used later to set the Content-Language header in the HTTP message to the client.
 

The old Exception constructors that take a String will remain.  These should be used in cases where the code throwing the exception is not localized, or the String is not localized (for example, a file name).  Also, there are several exceptions in Pegasus where the String parameter is meant to be a non-localized substitution in a localized message owned by the Exception (see InternalException.h, ClassNotResolved for an example).  The old constructors for these have been kept.
 
 

3.0 Provider Developers

3.1 Design Issues

Providers that wish to globalize should consider the following in their design:
 

• Are there localized string properties that need to be supported?  If so, then the client will use Accept-Language to request specific languages for these properties.  If the properties are read-only, use MessageLoader to load the localized strings for the properties.
• If you have a localized read/write string property, then the client will use Content-Language to set the property with an associated language.  The client will expect to be able to retrieve the property in that same language later (using Accept-Language).
• Note: only the string property types in CIM are candidates for localization.  The other types, including datetime, are locale-neutral.
• Are there error messages that need to returned to the client in different languages?  The client will use Accept-Language to request specific languages for the error messages.
• What resource bundle translations, if any, will be shipped with the provider?
• Do any codepage conversions need to be done between the UTF-16 characters in the String objects and the codepage of data stored on the system?  This is a concern for EBCDIC platforms.  All EBCDIC data needs to be converted to at least 7-bit ASCII before it is passed into the String object.

To help providers handle the situations described above, Pegasus 2.3 will pass the Accept-Language received from the client to the provider.  The provider should load strings from its resource bundle based on the client's Accept-Language.  The client's Accept-Language is passed to the provider in two ways:
 

• Pegasus will set the Accept-Language from the client into the thread in which the provider is running.  By using the useThreadLocale setting in MessageLoaderParms, providers can easily load strings using the client's requested Accept-Language.  The provider does not need to know what the Accept-Language is.  This is the recommended method to load messages based on the client's request.

 
• The OperationContext will contain an AcceptLanguageList object that has the Accept-Language requested by the client.  The provider can use this AcceptLanguageList object to load strings with MessageLoader.

The OperationContext will also contain a ContentLanguageList object that is set from the Content-Language in the client request.  This is the language of the CIM objects being passed to the provider on that request.  A localized provider should store the content language along with the data from the CIM objects.  This will allow the client to use Accept-Language later to retreive the data in that language.
 

The provider should indicate the language of CIM objects it is returning by calling setContext( ) on the ResponseHandler.  This will be used to set the Content-Language in the CIM response message sent back to the client.  If setContext( ) is not called, then no Content-Language will be returned to the client.  The setContext( ) function should only be called once per response.
 

3.2 Sample Code

The following sample code shows a localized getInstance( ) where the instance returned is localized based on the Accept-Language of the client request.  Note that this example also throws a localized exception.
 

void LocalizedProvider::getInstance(
    const OperationContext & context,
    const CIMObjectPath & instanceReference,
    const Boolean includeQualifiers,
    const Boolean includeClassOrigin,
    const CIMPropertyList & propertyList,
    InstanceResponseHandler & handler)
{
     // convert a potential fully qualified reference into a local reference
     // (class name and keys only).
     CIMObjectPath localReference = CIMObjectPath(
          String(),
          String(),
          instanceReference.getClassName(),
          instanceReference.getKeyBindings());

     // begin processing the request
     handler.processing();

      // Find the instance to be returned.
     Uint32 i;
     Uint32 n = _instances.size();
     for (i = 0;  i < n;  i++)
     {
          if(localReference == _instanceNames[i])
          {
                // We found the instance to return

                // Build the parameters for loading the localized string property.
                // We are going to let the message loader parameters default to use the
                // AcceptLanguageList that Pegasus set into our thread.
                // (this equals the AcceptLanguageList requested by the client)
                // Note: This parms object could be constructed once and
                // reused.
                MessageLoaderParms parms("myMsgID", "myDefaultString");
                parms.msg_src_path = "/myprovider/msg/myResourceBundle";

                // Load the string for the localized property from the resource bundle
                String localizedString = MessageLoader::getMessage(parms);

                // Remove the old property from the instance to be returned
                 Uint32 index = instances[i].findProperty("myProperty");
                 if (index != PEG_NOT_FOUND)
                 {
                      _instances[i].removeProperty(index);
                  }

                 // Add the localized string property to the instance
                 instances[i].addProperty(CIMProperty("myProperty", localizedString));

                 // The MessageLoader set the contentlanguages member
                 // of parms to the language that it found for the message.
                 ContentLanguageList rtnLangs = parms.contentlanguages;

                 // We need to tag the instance we are returning with the
                 // the content language.
                 OperationContext context;
                 context.insert(ContentLanguageListContainer(rtnLangs));
                 handler.setContext(context);

               break;
            }        //  end if
        }            // end for

        // throw an exception if the instance wasn't found
        if (i == n)
        {
                // Build the parameters for loading the localized error message.
                // We are going to let the message loader parameters default to use the
                // AcceptLanguageList that Pegasus set into our thread.
                // (this equals the AcceptLanguageList requested by the client)
                // Note: This parms object could be constructed once and
                // reused.
                MessageLoaderParms errParms("myErrorMsgID", "myErrorDefaultString");
                errParms.msg_src_path = "/myprovider/msg/myResourceBundle";

                // Note: the exception calls MessageLoader::getMessage( )
                // Note: no need to call handler.setContext( ) in this case
                throw CIMObjectNotFoundException(errParms);
        }
 

        // complete processing the request
        handler.complete();
}
 

NOTE: A sample provider has been written that fully demonstates the design issues described above.  This provider is located at:
 

• pegasus/src/Providers/sample/LocalizedProvider/

This sample provider also demonstrates how some of the special issues can be handled.  The special issues are caused by having a read/only localized property and a read/write localized property.  What happens if the client sets the read/write property with a Content-Language that is not one of the supported languages for the read/only property?  This provider allows the client to set any language into the read/write property, and get that property back in the same language.  This becomes an issue when the client does a getInstance( ) later, because the Content-Language on the returned instance applies to all the properties.  A related issue is what to return for Content-Language when the client does enumerateInstances, but the instances have different languages.  Recall that Content-Language applies to the entire response (a limitation in the CIM specification).
 

NOTE:  Indication Providers have other special considerations for language support.  Please refer to  PEP58.
 

NOTE:  The CMPI interface has been updated for language support.  Please refer to the CMPI documentation for details.
 

NOTE: SPECIAL ISSUES FOR OS/400 PROVIDERS:

• Convert between UTF-16 in the String objects and EBCDIC system data as needed.  The converters in Pegasus/Common/OS400ConvertChar.h may be used to convert between EBCDIC CCSID 37 and ASCII CCSID 819 (a subset of UTF-16).
• The Pegasus program, and all bound service programs,  will run in a UTF-8 locale even though the job CCSID is 37.  The C-runtime library (printf, fopen, isalpha, strcmp, etc) will expect UTF-8, or at least 7-bit ASCII, characters.
• Consideration should be given to the codepage for the compiled string literals.  Use #pragma convert as needed.  But, remember that the C-runtime will expect UTF-8.
• For more details, refer to "Unicode support" in chapter 3 of the ILE C/C++ for iSeries Run-Time Functions, Version 5 publication for V5R3 (SC41-5607-02).  The Pegasus string literals will be compiled with the UTF-8 compile switch described in this section.  OS/400 provider developers should strongly consider using the same compile switch for their string literals.  This would allow the literals to match the UTF-8 encoding expected by the C-runtime.

4. 0 Client Developers

Methods have been added to CIMClient to set the Accept-Language and Content-Language on the request, and retrieve Content-Language on the response.  The language tags in the Accept-Language header must meet the ISO-639 and ISO-3166 standards.
 

Please refer to
 

• pegasus/src/Pegasus/Client/CIMClient.h

 

Here is a code fragment that uses the new methods on CIMClient

    //
    // Get a localized instance in French
    //

   // Language priority is martian, pig-latin, and french.  We should
   // get french back, even though its the lowest priority
  AcceptLanguageList acceptLangs;
  acceptLangs.insert(LanguageTag("x-martian"), 1.0);
  acceptLangs.insert(LanguageTag("fr"), 0.1);
  acceptLangs.insert(LanguageTag("x-pig-latin"), 0.4);

    // Set the requested languages into the CIMClient
  client.setRequestAcceptLanguages(acceptLangs);

   // Get the instance
  CIMInstance instance = client.getInstance(
    NAMESPACE,
    cimNInstances[0].buildPath(sampleClass),
    localOnly,
    includeQualifiers,
    includeClassOrigin);

  // Get the string property that should be french
  String returnedString;
  instance.getProperty (
    instance.findProperty("myProp")).
             getValue().
             get(returnedString);

  // Check that we got back french
  ContentLanguageList CL_FR();
  CL_FR.append(LanguageTag("fr"));
  String expectedFRString = "oui";
  PEGASUS_ASSERT(CL_FR == client.getResponseContentLanguages());
  PEGASUS_ASSERT(expectedFRString == returnedString);

    //
    // Create an instance in French
    //

   String oui = "Oui";
   CIMInstance frInstance(CLASSNAME);
   frInstance.addProperty(CIMProperty(
            CIMName("myProp"),
            oui));

   CIMObjectPath frInstanceName = frInstance.buildPath(sampleClass);

   client.setRequestContentLanguages(CL_FR);

   client.createInstance(NAMESPACE, frInstance);
 
 
 

Also, refer to

• pegasus/src/Clients/g11ntest/

NOTE:  Consideration should be given for converting the UTF-16 characters in the String objects passed over the CIMClient interface to a platform codepage.  This is especially needed for EBCDIC platforms.  See the Provider developer section for details of the EBCDIC considerations.
 
 

4.1 Default Process Locale

A method has been added to CIMClient to set the Accept-Language for the requests based on the default locale of the process, as determined by ICU.  If ICU is installed on the client system then CIMClient will set the Accept-Language from the default ICU process locale.  If ICU is not installed then the caller is required to set an AcceptLanguageList into CIMClient that meets the ISO-639 and IS0-3166 standards.  Note:  this is useful for local clients, such as the Pegasus CLIs, where ICU would be installed on both the client and server sides.
 
 

5. 0 Pegasus Developers

The design for Pegasus releases beyond 2.3 is to avoid using hardcoded messages.  All new messages should be loaded from a Pegasus resource bundle.  This section describes the process to follow if you are creating a new message.  The process depends on where you are in the code.
 
 

5.1 Pegasus Resource Bundles

Place any new Pegasus messages into one of the following resource bundles:
 

• pegasus/src/Pegasus/msg/Server/pegasusServer_*.txt  for server and MOF compiler (cimmof, cimmofl) messages
• pegasus/src/Pegasus/msg/CLI/pegasusCLI_*.txt for all CLI messages (except the MOF compiler)

The make messages target will compile these resource bundles.

Note:  As described above, the resource bundle path in MessageLoaderParms defaults to the server resource bundle.  For CLI messages, you will need to specify the bundle for your CLI.
 

5.2 Server Messages

For messages returned from one of the services in the Pegasus server (eg. CIMOperationRequestDispatcher, or ProviderManagerService), the goal is to make it easy for any code in the call chain to throw an exception with a localized error string.  The code throwing the exception will not need to know the Accept-Language that the client requested.  To understand how this works, some design points need to described:
 

Server Design Points:
 

The CIMMessage object has been expanded to include an AcceptLanguageList object and a ContentLanguageList object in its OperationContext member.  For CIMRequestMessage, these objects contain the Accept-Language and Content-Language headers that were built from the client request.  For CIMResponseMessage, the ContentLanguageList object is used to build the Content-Language header associated with the CIM objects in the response message.  The AcceptLanguageList object in the CIMResponseMessage is ignored.
 

The localization of the cimException object in the CIMResponseMessage is handled separately from the CIM objects.  The message string in the cimException object is assumed to have been localized by the time it is built into the XML.  For this reason, the localization of the exception is the responsibility of the code throwing the exception.  (The goal of the design is to make that easy - see below).  The ContentLanguageList object in the CIMResponseMessage has NO relation to this exception.  The cimException object keeps its own localization information once it is created.
 

To enable exceptions to be localized, the ability was added to set a global language for all the code running from a Pegasus Thread object.  The top level code for a Thread can set a global AcceptLanguageList object that can be accessed by all the low-level functions that it calls.  This will allow an exception thrown by the low-level function to be localized based on this global AcceptLanguageList object.  Note:  This applies only to Threads that are managed by a ThreadPool.
 

Each service in the request path of the Pegasus server sets the AcceptLanguageList into its Thread from the AcceptLanguageList in the CIMRequestMessage object that it dequeues.  This sets the global langauge for all the functions in the same thread that are called below handleEnqueue.  If you are writing a new service that processes requests, or discover a request service that was missed, please do this.  The CIMOperationRequestDispatcher service is an example.
 

How to Throw a Localized Exception from Server code:
 

With all that background, here is how code running in a Pegasus service can throw a localized exception:
This example assumes that the top-level code in the service had set the global thread AcceptLanguageList beforehand.  As described above, every service in Pegasus should do that.  The code here may be buried several layers deep in the call chain, but does not need to know the AcceptLanguagList of the current client request.

// First, construct a MessageLoaderParms
//
// Notes:
//  1) The errorMessageID must be in the Pegasus server resource bundle.
//  2) The default message is the old "hardcoded" message.
//  3) The MessageLoaderParms will default to use the Pegasus server resource bundle
//  4) The MessageLoaderParms will default to use the AcceptLanguageList set into the current Thread.  Don't change this!
//  5) You might need to set the arguments for the message into the MessageLoaderParms
MessageLoaderParms parms("errorMessageID", "default message");

// Second, throw the Exception
// Note: this applies to all the derived classes from Exception, including the CIMException's
throw new Exception(parms);
 

NOTE:  If you are throwing an Exception with un-localized data, use the constructor that takes a String.  An example of this would be an Exception where you are passing in a file name.  Most of the "non-CIM" exceptions defined in Exception.h and InternalException.h take un-localized data.
 

The Exception Macros
 

There are many spots in the server code that use the PEGASUS_CIM_EXCEPTION macro to throw a TraceableCIMException.  The use of this macro in the code like the following example presented a design problem:

....
} catch (Exception & e)
{
    throw PEGASUS_CIM_EXCEPTION(CIM_ERR_FAILED, e.getMessage());
}
 

This type of code would have lost the ContentLanguageList saved in "e", so that the Content-Language would not be set in HTTP response to the client.
 

For Pegasus 2.3, these types of macro calls can stay.  The TraceableCIMException constructed by the macro will "re-localize".  That is, the "CIM" part of the message (the part based on the error code) will be localized at throw time, and the ContentLanguageList re-established.  A key is to avoid a "language mismatch" problem between the CIM part of the message and the extra part of the message.  The design point here is that all internal exceptions thrown by Pegasus code are localized using the global AcceptLanguageList of the Thread...see above.
 

In the future, it will be safer and more maintainable to use of the  new "localized" flavors of the macro.  For example:
 

When the message from a caught  Exception needs to be become the extra message in a thrown CIMException:

....
} catch (Exception & e)
{
        throw PEGASUS_CIM_EXCEPTION_LANG(e.getContentLanguages( ),
                                                                                       CIM_ERR_FAILED,
                                                                                        e.getMessage( ));
}
 

This guarantees that the ContentLanguageList in "e" is copied to the newly created TraceableCIMException.
 

In the case where the extra message for the CIMException is determined by the throwing code:
 

throw PEGASUS_CIM_EXCEPTION_L(CIM_ERR_FAILED,
                                                                        MessageLoaderParms("Repository.CIMRepository.COMPACT_FAILED",  "compact failed"));
 

(example from CIMRepository.cpp)
This uses a MessageLoaderParms object to localize the extra message in the newly created TraceableCIMException.
 

5.2 Logger Messages

New methods have been added to Logger to take a message ID of a message to be loaded from the Pegasus server resource bundle.  The caller is only required to pass in the message ID, the old "hardcoded" message, and the args.  The Logger will use MessageLoader to load the message in the locale of the Pegasus server process, using the hardcoded message as the default string.  Please refer to pegasus/src/Pegasus/Logger.h.

Note:  Messages sent to the "logs", whether the system logs or the Pegasus log file, are converted to UTF-8 before being sent.
 

5.3 CLI Messages

The goal for messages returned by the Pegasus CLIs is to localize in the locale of the user running the CLI.  This should be automatic -- the user should not be required to tell the CLI what the locale is.   For the CLIs that are CIM clients (cimconfing, cimprovider) there are two sets of messages to localize  -- messages generated in the CLI process itself, and messages returned from the Pegasus server .  For CLIs that are directly linked into Pegasus (cimmofl), all the messages are generated in the CLI's process, but the CLI may call Pegasus APIs that are coded to localize based on a client's requested languages.
 

Code in the client side of the client/server CLIs (eg. cimconfig, cimmof), or in directly linked CLIs (cimmofl), should use the _useProcessLocale "master switch" described in the Message Loading section.  This will cause all messages, including exceptions thrown by Pegasus APIs,  to be loaded in the locale based on the environment in which the program is running.  This locale can be set by the user before running the program.
 

Code in the client side of the client/server CLIs need to send an Accept-Language to the Pegasus server that reflects the default locale of the CLI's process.  See the Client Developer section for details.
 

An example of these considerations can be seen in the source code for cimconfig.
 

Copyright (c) 2000, 2001, 2002 BMC Software; Hewlett-Packard Development Company, L.P.; IBM Corp.; The Open Group; Tivoli Systems. Copyright (c) 2003 BMC Software; Hewlett-Packard Development Company, L.P.; IBM Corp.; EMC Corporation, The Open Group. Copyright (c) 2004 BMC Software; Hewlett-Packard Development Company, L.P.; IBM Corp.; EMC Corporation; VERITAS Software Corporation; The Open Group. Copyright (c) 2005 Hewlett-Packard Development Company, L.P.; IBM Corp.; EMC Corporation; VERITAS Software Corporation; The Open Group. Copyright (c) 2006 Hewlett-Packard Development Company, L.P.; IBM Corp.; EMC Corporation; Symantec Corporation; The Open Group.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE SHALL BE INCLUDED IN ALL COPIES OR SUBSTANTIAL PORTIONS OF THE SOFTWARE. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.