version 1.1.2.7, 2013/06/16 13:04:23
|
version 1.3, 2014/11/12 13:34:09
|
|
|
Using the CIM/XML Pull Operations |
Using the CIM/XML Pull Operations and FQL query Language |
| |
STATUS | STATUS |
| |
<<< The TODO section is being maintained during the review and checkin process |
The Pull operations and FQL query language are incorporated into |
to keep track of problems, errors, notes, etc. Must be deleted before |
OpenPegasus 2.14. This readme defines the characteristics, limitations, |
checkin to head of tree. Please feel free to add notes, etc in this |
etc. for that implementation. |
section as you review/test.>>>>>> |
|
|
|
NOTES On working with task branch. |
|
|
|
Merge out Process |
|
|
|
To keep our TASK branch in sync with the current head of tree we need |
|
to do a regular merge out. the TaskMakefile contains the makefile |
|
procedures to do this efficiently. NOTE: Following these procedures is |
|
important in that you are merging out new material each time you do |
|
the merge out. If you were just to repeatedly merge out, you would be |
|
merging previously merged changes a second time causing a real mess. |
|
|
|
Start with new directory and put TaskMakefile above pegasus (needed so you |
|
have this file for the initial operations. |
|
|
|
make -f TaskMakefile branch_merge_out BNAME=PEP317-pullop ## takes a long time |
|
|
|
This checks out current head, merges it into task branch and sets tags |
|
for the mergeout. Note that at the end of this step this work is |
|
part of the TASK... branch. |
|
|
|
NOW check for conflicts, errors, etc. that resulted from the merge. |
|
Look for conflict flags, compare the results (I use linux merge as a |
|
good graphic compare tool) and build and test. When you are satisfied |
|
that the merge out is clean, you can commit the results to the TASK... |
|
branch |
|
|
|
To commit the work to this into Task branch |
|
|
|
make -f mak/TaskMakefile branch_merge_out_commit BNAME=PEP317-pullop |
|
|
|
or manually commit and finish as follows |
|
|
|
cvs commit |
|
make -f mak/TaskMakefile branch_merge_out_finish BNAME=PEP317-pullop |
|
|
|
## This last step is important since it cleans up temporary tags to prepare |
|
you for the next checkout |
|
|
|
COMPARE TASKBRANCH WITH HEAD |
|
|
|
In a new pegasus work space do same as above for merge out. |
|
|
|
make -f TaskMakefile BNAME=PEP317-pullop |
|
|
|
This produces a result which is all of the head merged into the branch. |
|
A diff of this is all the new changes to the head of tree that you will |
|
include into the merge. |
|
|
|
|
|
TODO list: |
|
1. Binary operation from OOP. Need to add counter to binary |
|
protocol to be able to count objects in response. Generates |
|
warnings in things like messageserializer and does not work with |
|
OOP right now. |
|
2. OpenExecQuery - Code is incorrect in that it used InstancesWithPath |
|
where the spec is instances with no path. Need new function to wrap |
|
getInstanceElement(withoutPathElement) in XmlReader. Note that |
|
Alternate is to put flag on InstancesWith Path to say no path |
|
3. Code for Pull part of OpenQueryInstancesRequest a) should be part of |
|
the common CIMOperationRequestDispatcher execCommon code. |
|
4. The changes to WQLCIMOperationRequestDispatcher and CQL... for handling |
|
pull not completed so we feed the responses back to the EnmerationContext |
|
queues |
|
3. Lots of minor TODOs, diagnostics, etc. |
|
4. External runtime variables. Decide this as part of PEP. The variables |
|
exist in CIMOperationRequestDispatcher but not in CIMConfig. The primary |
|
ones to consider are: |
|
a. System maxObjectCount. Setting some maximum size on what a pull |
|
client can request (i.e. the maximum size of the maxObjectCount on |
|
Open... and pull operations. |
|
b. Pull interoperationTimeout (max times between operations). This is |
|
the maximum number of seconds on the operationTimeout parameter of the |
|
Open operations |
|
c. Maximum size of the responseCache before it starts backing up |
|
responses to the providers. |
|
5. Decision on EnumerationContext timeout (separate thread or just |
|
checks during other operations). Can we, in fact really keep the |
|
enumeration context table and queue under control without monitoring |
|
with a separate thread. We must monitor for: |
|
a. Client operation that stop requesting (i.e. inter operation time |
|
exceeds operationTimeout). Note that if it simply exceeds the time |
|
the next operation does the cleanup. The issue is those clients that |
|
simply stop and do not either close or go to completion. |
|
b. We should protect against providers that no not every finish delivering |
|
or take to long between deliveries. This does not exist in Pegasus |
|
today |
|
6. Clean up code in Dispatcher. The associators code is still real mess |
|
and the pull code is in a template. The Pull code is good now but |
|
must be duplicated. Look at creating new CIMMessage CIMPullResponseMessage |
|
so that we can have common code. Everything is the same except what |
|
goes into the CIMResponseData so it is logical to have completely |
|
common processing |
|
7. Extension to avoid double move of objects in CIMResponseData (one |
|
into enumerationContext queue and second to new cimResponseData for |
|
response. Want to avoid second move by extending Open/Pull response |
|
messages to include count and CIMResponse data to count objects out |
|
of queue when converting (avoids the second move). Big issue here |
|
with binary data since need to extend format to count it. |
|
8. Still using templates, etc. in code in the Dispatcher. This is for |
|
all of the open operations where there is a lot of duplicate code |
|
and the pull operations that are 99% duplicate code (in a single template) |
|
9. NEXT TASK: get the pull operations into a single function by |
|
creating a new CIMPullResponse message in CIMMessage.h that contains |
|
the pull data. Then we can use a single function to process all pull |
|
operations. |
|
|
|
Fixed for 16 June CVS Update |
|
1. Cleaned up the enumerationContext and Table release functions and tested |
|
to confirm that we do not lose memory in either normal sequences or |
|
sequences that close early. Cleaned up pullop and added more tests |
|
|
|
Fixed for 9 June CVS update |
|
1. Cleaned up code for OpenQueryInstances. Note that this is incomplete. |
|
No support in WQL or CQL Operations |
|
2. |
|
|
|
What was fixed for 5 June checkin. |
|
1. Extended ResponseTest MOF for for both CMPI and C++ subclasses |
|
2. Fixed issues with pullop. |
|
3. Fixed temp issue with CIMResponseData size by putting in mutex. That |
|
is not a permanent fix but it gets around issue probably in the control |
|
of the move logic that meant counts were off. |
|
4. Fixed issues in Dispatcher so that associator code works. Still messy |
|
code in the dispatcher. |
|
5. Changed name of Enumerationtable.h & cpp to EnumerationContextTable.* |
|
6 Changed name of ResponseStressTest module, classes, etc. |
|
|
|
TAG: TASK_PEP317_5JUNE_2013_2 |
|
|
|
2 June 2013 |
|
|
|
Issues - KS |
|
1. have not installed the binary move in CIMResponseData. Please run |
|
with OPP off. |
|
2. Some problem in the processing so we are getting server crashes. |
|
Right no I am guessing that this is in the binaryCodec and am going to |
|
expand the test tools to allow testing through the localhost. |
|
|
|
3. Still way to many TODO and KS comments and KS_TEMPS. Removing bit by bit. |
|
|
|
4. Env variable connection for the config parameters not installed. |
|
|
|
5. Issue with the threaded timer. For some reason during tests it |
|
eventually calls the timer thread with trash for the parm (which is |
|
pointer to the EnumerationTable object). Caught because we do a valid |
|
test at beginning of the function. |
|
|
|
6. Still using the templates in CIMOperationRequestDispatcher to simplify |
|
the handle... processing. |
|
|
|
7. I think I have a way around the double move of objects in the |
|
EnumerationContext so that the outputter will just take a defined number |
|
of objects directly from the gathering cache and save the second move. |
|
|
|
8. Not yet passing all tests but getting closer now. |
|
|
|
9. Created a tag before this commit TASK_PEP317_1JUNE_2013. |
|
|
|
10. Next Tag will be TASK_PEP317_2_JUNE_2013 in the task branch |
|
|
|
| |
=========================================== | =========================================== |
|
|
OVERVIEW: | OVERVIEW: |
| |
The operation extensions for pull operations defined in the DMTF specification | The operation extensions for pull operations defined in the DMTF specification |
DSP0200 V 1.4 were implemented in Pegasus effective Pegasus version 2.11 |
DSP0200 V 1.4 were implemented in Pegasus effective Pegasus version 2.14 |
including Client and Server. | including Client and Server. |
| |
These operations extend the CIM/XML individual operations to operation | These operations extend the CIM/XML individual operations to operation |
|
|
The following new CIM/XML operations as defined in DSP0200 are included; | The following new CIM/XML operations as defined in DSP0200 are included; |
| |
-OpenEnumerateInstances | -OpenEnumerateInstances |
-openEnumerateInstancePaths |
-OpenEnumerateInstancePaths |
-OpenReferenceInstances | -OpenReferenceInstances |
-OpenReferenceInstancePaths | -OpenReferenceInstancePaths |
-OpenAssociatiorInstances | -OpenAssociatiorInstances |
-OpenAssociatorInstancePaths | -OpenAssociatorInstancePaths |
|
-OpenQueryInstances |
-PullInstancesWithPath | -PullInstancesWithPath |
-PullInstancePaths | -PullInstancePaths |
|
-PullInstances |
-CloseEnumeration | -CloseEnumeration |
-EnumerationCount |
-EnumerationCount (deprecated by DMTF and incomplete) |
OpenExecQuery |
|
|
|
The following operations have not been implemented in this version of Pegasus: |
|
|
|
-OpenQueryInstances |
|
|
|
The following limitations on the implementation exist; |
|
|
|
1. The filterQueryLanguage and filterQuery parameters are processed by |
|
the Pegasus client but the server returns error if there is any data in |
|
either parameter. This work does not include the development of the |
|
query language. Note that a separate effort to extend Pegasus to use |
|
the DMTF FQL query language is in process. |
|
|
|
2. The input parameter continueOnError is processed correctly by the client |
|
but the Pegasus server only provides for false since the server does not |
|
include logic to continue processing responses after an error is |
|
encountered. |
|
This is consistent with the statement in the specification that use of |
|
this functionality is optional and the fact that the DMTF agrees that all |
|
of the issues of continuing after errors have not been clarified. |
|
|
|
3. The operation enumerationCount is not processed by the server today since |
|
a) really getting the count would be the same cost as the corresponding |
|
enumeration, b) the server does not include a history or estimating |
|
mechanism for this to date. |
|
NOTE: After a through review as part of the development of the next version |
|
of CMPI we have concluded that this operation is probably not worth the |
|
effort. Since it is optional, Pegasus will only return the unknown status |
|
at this point |
|
| |
Since the concept of sequences of operations linked together (open, pull, close) | Since the concept of sequences of operations linked together (open, pull, close) |
is a major extension to the original CIM/XML operation concept of completely | is a major extension to the original CIM/XML operation concept of completely |
independent operations several new pieces of functionality are implemented | independent operations several new pieces of functionality are implemented |
to control interOperationTimeouts, counts of objects to be returned, etc. |
to control interOperation timeouts, counts of objects to be returned, etc. |
|
|
|
NOTE: More detailed information on the pull operations is contained in: |
|
1. The OpenPegasus wiki (pull operations work group) |
|
2. The pull operations PEP (preliminary today) |
|
3. Presentations in the OpenPegasus wiki, in particular in the |
|
pull operations workgroup documentation at: |
|
|
|
https://wiki.opengroup.org/pegasus-wiki/doku.php?id=dev:workgroups:pulloperationsupportworkgroup:pull_operation_support_work_group |
| |
TBD - Review this |
|
| |
CLIENT |
OPENPEGASUS CLIENT |
| |
The new operations follow the same pattern as the APIs for existing operations | The new operations follow the same pattern as the APIs for existing operations |
in that: | in that: |
|
|
endOfSequence which is the marker the server sends in open and pull | endOfSequence which is the marker the server sends in open and pull |
responses when it has no more objects to send. | responses when it has no more objects to send. |
| |
The significant differences include: |
The significant differences between the open... and original enumerate, |
|
associator, and reference operations includes: |
| |
1. Processing of parameters on responses (i.e. the endOfSequence and |
1. The new pull client operations typically require multiple client operations |
enumerationContext parameters are returned for open and pull operations). |
to retrieve a complete set of data (ex. OpenEnumerateInstances and |
|
PullInstancesWithPath) are equivalent to the old EnumerateInstances |
|
client request. |
|
|
|
2. Processing of parameters on responses (i.e. the endOfSequence and |
|
enumerationContext parameters are returned for open and pull operations) |
|
These return arguments are used to control the loop retrieving data |
|
for an enumeration sequence (ex. the sequence of OpenEnumerateInstances |
|
and PullInstancesWithPath that represents a complete enumeration |
|
sequence). The sequence continues until the server responds with |
|
EndOfSequence is true, and the enumerationContext is the identifier |
|
for the sequence. |
| |
2. Numeric arguments (Uint32 and Uint64 include the option of NULL in some |
3. Numeric arguments (Uint32 and Uint64) include the option of NULL in some |
cases so they are packaged inside classes Uint32Arg and Uint64Arg in the | cases so they are packaged inside classes Uint32Arg and Uint64Arg in the |
client api. |
client API. |
| |
3. The association and reference operations ONLY process instances. They do |
4. The openAsociatorInstances and openReferenceInstances operations ONLY |
not include the capability to return classes like reference and associator |
process instances. They do not include the capability to return classes |
do and therefore return CIMInstance rather than CIMObject. |
as reference and associator requests do and therefore return CIMInstance |
|
rather than CIMObject. |
| |
4. Paths are returned in all cases (i.e OpenEnumerateInstances and |
5. Paths are returned for the instance operations as part of the returned |
PullInstancesWithPath where they were not with EnumeratInstances. |
instances (ex. OpenEnumerateInstances and PullInstancesWithPath) where |
|
they were not with EnumerateInstances. |
| |
5. The client must maintain state between operations in a sequence (using |
6. The client must maintain state between operations in an enumeration |
the enumerationContext parameter). |
sequence (using the enumerationContext parameter returned with open |
|
and pull responses). The client must always return the last |
|
enumerationContext received as the server may modify the |
|
enumerationContext for each response in an enumeration sequence. |
| |
TBD- Are there more differences. |
The client API is defined in the header file: |
| |
|
pegasus/src/Pegasus/Client/CIMCLient.h |
| |
SERVER |
|
| |
The Pegasus server attempts to always deliver the requested number of objects |
OPENPEGASUS SERVER |
for any open or pull request (the specification allows for the server to |
|
deliver less than the requested number of objects and specifically to return |
|
zero objects on open). We felt that it was worth any extra cost in processing |
|
to provide the client with exactly what it had requested. |
|
| |
The pegasus server always closes an enumeration sequence upon receipt of any |
The Pegasus server attempts to always deliver at least some objects |
|
in a response. It does not wait to deliver the exact number requested but |
|
waits to deliver at least some. The sever will wait about 15 seconds to |
|
deliver and if there is nothing delivered from the providers it will then |
|
return a response with zero objects in it so that the client does not |
|
timeout. This would only occur if the providers are extremely slow (i.e. |
|
greater than 15 seconds) in preparing response objects to be delivered. |
|
|
|
The Pegasus server always closes an enumeration sequence upon receipt of any |
error from the providers, repository, etc. Therefore the server will reject | error from the providers, repository, etc. Therefore the server will reject |
any request that has continueOnError = true; | any request that has continueOnError = true; |
| |
|
|
future to support the continueOnError other than possibly allowing the provider | future to support the continueOnError other than possibly allowing the provider |
to continue processing after it has received an error. | to continue processing after it has received an error. |
| |
PROVIDERS |
OPENPEGASUS PROVIDERS |
| |
This implementation requires NO changes to the existing providers. The |
This implementation (OpenPegasus version 2.14) requires NO changes to the |
provider APIs operate just as they do with the original operations. |
existing providers. The provider APIs operate just as they do with the |
|
original operations. |
| |
Because the server processing is different however, there may be some | Because the server processing is different however, there may be some |
behavior differences primarily because the client now controls the speed of | behavior differences primarily because the client now controls the speed of |
delivery of objects. | delivery of objects. |
| |
In previous versions of Pegasus, the server attempts to deliver objects as |
In previous versions of Pegasus, the server attempted to deliver objects as |
rapidly as then can be put on the network. In the case of HTTP chunked requests | rapidly as then can be put on the network. In the case of HTTP chunked requests |
they are delivered in chunks of about 100 objects. The primary delay for the | they are delivered in chunks of about 100 objects. The primary delay for the |
providers was the processing of each segment through the server. The server | providers was the processing of each segment through the server. The server |
|
|
and then delivered as one non-chunked response. There were no delays for the | and then delivered as one non-chunked response. There were no delays for the |
providers, just lots of possible memory use in the server. | providers, just lots of possible memory use in the server. |
| |
The responses from providers (delivered through the deliver(...) interface are |
The responses from providers (delivered through the deliver(...) interface) are |
gathered into segments of about 100 objects and this group of objects is moved | gathered into segments of about 100 objects and this group of objects is moved |
through the server to be delivered to the client. | through the server to be delivered to the client. |
| |
However with the inclusion of the pull operations, The segments of objects |
However with the inclusion of the pull operations, the segments (CIMResponseData |
from the providers are cached in the server response path until the |
objects containing the instance or path objects) from the providers are cached |
maxObjectCount for that request (open or pull) and that number returned in a |
in the server response cache until the next pull request and that number |
non-chunked response. Thus, if the client is slow to issue pull requests, |
is returned in the response to that pull. Thus, if the client is slow to issue |
the providers might be delayed at some point to reduce memory usage in the |
pull requests, the providers might be delayed at some point to reduce memory |
server (the delay appears as slow response tothe deliver operation). |
usage in the server (the delay appears as slow response to the deliver operation). |
| |
In other words, the time to process large sets of responses from the provider | In other words, the time to process large sets of responses from the provider |
now depends on the speed of handling the client. | now depends on the speed of handling the client. |
|
|
provider gathering very large arrays of objects and sending them to the | provider gathering very large arrays of objects and sending them to the |
server. | server. |
| |
NEXT GENERATION PROVIDERS |
FQL (FILTER QUERY LANGUAGE) |
KS_TODO |
|
|
The FQL implementation is complete in accord with the DMTF |
|
specification DSP 0212 except for a few issues including: |
|
|
|
1. The regex for the LIKE operation is the same as CQL basic. It provides |
|
only the following special characters "." and "*". |
|
2. The implementation does not include the comparison of |
|
embeddedInstances |
|
3. The implementation does not include the handling of Uint8[] as |
|
strings. |
|
|
|
For more details on the FQL implementation see the readme.txt in the |
|
directory pegasus/src/Pegasus/FQL. |
|
|
|
OpenPegasus will remove these limitations in a future version. |
|
|
|
The FQL implementation includes a large set of sample queries in the |
|
directory: |
|
|
|
pegasus/src/Pegasus/FQL/tests/Parser |
|
|
|
including both good and error generating queries. |
|
|
|
================================================================== |
|
LIMITATIONS IN OPENPEGASUS 2.14 |
|
|
|
1. The openQueryInstances does not allow requesting the class on response. |
|
|
|
2. The openEnumerateInstanceNames, openAssociatorNames, and openReferenceNames |
|
do not allow use of the query filter. This is because |
|
a. The intention is to deprecate these operations completely and remove |
|
them in a future version |
|
b. They require that the server call the providers with the corresponding |
|
enumerate, associators, references to get the full instances to |
|
filter |
|
|
|
3. The filterQuery filtering is done by the server, not the providers in this |
|
version of OpenPegasus. This will be modified in a future version of |
|
OpenPegasus when the provider API extensions for the pull operations |
|
have been resolved. Version 2.1 of the CMPI specification will resolve |
|
this issue. |
| |
CONFIGURATION PARAMETERS |
4. The input parameter continueOnError is processed correctly by the client |
|
but the Pegasus server only provides code for 'false' value, since the server |
|
does not include logic to continue processing responses after an error is |
|
encountered. |
|
This is consistent with the statement in the specification that use of |
|
this functionality is optional and the fact that the DMTF agrees that all |
|
of the issues of continuing after errors have not been clarified. |
|
|
|
5. The operation enumerationCount is not processed by the server today since |
|
a) Getting the count would be the same cost as the corresponding |
|
enumeration, b) the server does not include a history or estimating |
|
mechanism for this to date. |
|
NOTE: After a through review as part of the development of the next version |
|
of CMPI we have concluded that this operation is probably not worth the |
|
effort. Since it is optional, Pegasus will only return the unknown status |
|
at this point. Further it is the intention of the DMTF to deprecate this |
|
function. |
|
|
|
=================================================================== |
|
PULL OPERATION CONFIGURATION PARAMETERS |
| |
The server includes several configuration parameters to set limits on the | The server includes several configuration parameters to set limits on the |
processing of pull operations. All of these configuration parameters are |
processing of pull operations. |
compile time parameters rather than runtime. |
|
|
RUNTIME CONFIGURATION PARAMETERS |
| |
1. Maximum value of minimum interoperation time. This parameter defines the |
1. Maximum value of interoperation time (pullOperationsMaxTimeout) - |
maximum time allowed between the return of an open or pull response and |
This parameter defines the maximum time allowed between the return of an open |
the receipt of the next pull or a close operation before the server may |
or pull response and the receipt of the next pull or a close operation before |
close the enumeration. The specification allows the server to set a |
the server may close the enumeration. |
maximum interoperation time and refuse open requests that with requested |
The specification allows the server to set a maximum interoperation time and |
operationTimeout greater than that time. |
refuse open requests that with requested operationTimeout greater than that time. |
CIM_ERR_INVALID_OPERATION_TIMEOUT |
|
|
2. Maximum number of objects returned in a single open or pull operation |
This value is set with the Pegasus environment variable |
(pullOperationsMaxObjectCount) - The server can set a maximum limit on the |
PEGASUS_PULL.... |
number of objects that can be returned in a single open or pull operation |
|
with the maxObjectCount parameter. This parameter sets the maximum limit. |
2. Maximum objects returned in a single open or pull operation. The server |
The absolute maximum allowed without recompiling (and changing a value in |
can set a maximum limit on the number of objects that can be returned in |
Common/Constants.h) is 10000. |
a single open or pull oepration with the maxObjectCount parameter. |
|
|
3. Default operationTimeout (pullOperationsdefaultTimeout) - If the client |
3. Whether the server allows 0 as an interoperation timeout value. The value |
does not specify an operation timeout in the open request, the server uses |
zero is s special value for the interoperationTimeout in that it tells the |
the value defined by this runtime configuration variable. The default |
server to not timeout any enumeration sequence. |
is 30 seconds. |
|
|
|
COMPILE TIME CONFIGURATION PARAMETERS |
|
|
|
1. Whether the server allows 0 as an interoperation timeout value - The value |
|
zero is a special value for the interoperationTimeout in that it tells the |
|
server to not timeout the enumeration sequence. This should never be used |
|
as it allows a client to open enumeration sequences that will not be |
|
cleaned up if the client does not properly terminate them. There is a |
|
compile time variable in CIMOperationRequestDispatcher that would allow this |
|
to be set so 0 interoperation is allowed but we recommend that it never |
|
be used since it removes a significant component of the server management |
|
of enumerationContext information. |
| |
With this value for interoperationTimeout, the only way to close an | With this value for interoperationTimeout, the only way to close an |
enumeration sequence is to complete all of the pulls or issue the close. | enumeration sequence is to complete all of the pulls or issue the close. |
If for some reason the sequence is not completed, that enumeration context | If for some reason the sequence is not completed, that enumeration context |
would remain open indefinitly. Since in Pegasus any open enumeration |
would remain open indefinitely. Since in Pegasus any open enumeration |
context uses resources (the context object and any provider resposnes that |
context uses resources (the context object and any provider responses that |
have not yet been issued in a response) it would appear that most | have not yet been issued in a response) it would appear that most |
platforms would not want to allow the existence of enumeration contexts | platforms would not want to allow the existence of enumeration contexts |
that cannot be closed by the server. | that cannot be closed by the server. |
| |
4, maximum consecutive pull requests with 0 maxObjectCount. The use of the |
2. Maximum consecutive pull requests with 0 maxObjectCount - The use of the |
pull operation with maxObjectCount set to zero could be used to keep an | pull operation with maxObjectCount set to zero could be used to keep an |
enumeration context open indefinitly (this tells the server to restart the |
enumeration context open indefinitely (this tells the server to restart the |
interoperationTimeout but not send any objects in the response). Therefore the | interoperationTimeout but not send any objects in the response). Therefore the |
specification allows for the server setting maximum limits on this behavior | specification allows for the server setting maximum limits on this behavior |
and returning the error CIM_ERR_SERVER_LIMITS_EXCEEDED if this limit is | and returning the error CIM_ERR_SERVER_LIMITS_EXCEEDED if this limit is |
exceeded. | exceeded. |
|
|
Note that this is maximum CONSECUTIVE pulls so that issuing a pull with | Note that this is maximum CONSECUTIVE pulls so that issuing a pull with |
a non-zero count resets this counter. | a non-zero count resets this counter. |
| |
KS-TBD - Is this really logical since we can still block by just issuing |
Pegasus sets the value of this limit to 1000 and allows the implementer to |
lots of zero request and an occansional request for one object. |
modify it by compiling with the PEGASUS_MAXIMUM_ZERO_OBJECTCOUNT define in |
|
CIMOperationRequestDispatcher.cpp modified. |
|
|
|
3. Time to wait for next response from providers - In the case where providers |
|
are responding very slowly, the goal is to generate responses with |
|
zero instances at regular intervals to allow the client to keep the |
|
enumeration context alive. This wait time is set by a compile time define |
|
(PEGASUS_PULL_MAX_OPERATION_WAIT_SEC)in pegasus/src/Pegasus/Common/Constants.h |
|
and is set to 15 seconds in the current release. |
|
|
|
4. Time to wait before killing off an enumeration context that is blocked |
|
by missing provider responses. In the rare case where providers do not complete |
|
their responses to the server there is a compile-time counter that trys to |
|
clean up the providers and finally just kill the enumeration sequence after |
|
a defined number of consecutive pulls that return zero objects. |
|
The limit is defined in pegasus/src/Pegasus/Constants.h |
|
|
|
NOTE: The development team is trying to consolidate all such constants and |
|
#define definitions that control overall server characteristics but are not |
|
runtime parameters in Constants.h |
|
|
|
=================================================================== |
|
TESTING |
|
|
|
The pull operations are tested primarily with two client programs in the |
|
directory Pegasus/Client/tests |
|
|
|
1. pullop and it corresponding Makefile provide extensive tests of the |
|
pull operations and comparison of the results with the corresponding non |
|
pull operations. |
|
|
|
2. PullErrors tests a number of error scenarios with the pull operations. |
|
|
|
3. cimcli has been extended to allow execution of the pull operations with new |
|
operations that parallel the existing operations: |
|
enumerateinstances(ei) - corresponding is pullenumerateInstances(pei) |
|
ni -> pni enumerateInstanceNames |
|
r -> pr references |
|
rn -> prn referenceNames |
|
a -> pa associators |
|
ar -> par associatorNames |
| |
Pegaus sets the value of this limit to 1000 and allows the implementer to |
These execute complete pull sequences (open, pull) with cimcli options |
modify it with the PEGASUS_MAXIMUM_ZERO_OBJECTCOUNT environment variable. |
to control parameters like maxObjectCount, interoperation Timeout, etc. |
| |
5. Default operationTimeout - |
|
| |
The default of this parameter is to refuse operat |
============================================================= |
|
TODO LIST - Post 2.14 release |
|
1. Binary operation from OOP. Need to add counter to binary |
|
protocol to be able to count objects in response. Generates |
|
warnings in things like messageserializer and does not work with |
|
OOP right now. Fixed by converting to XML. Concluded that we do not |
|
need to do this. The binary response is not really used often |
|
in the current environment So double mapping it is not a major issue. |
|
Leave this as FUTURE |
|
2. Minor TODOs, diagnostics, etc. still in the code. Reduced almost to |
|
none now. We are leaving some in as PEGASUS_DEBUG |
|
3. Extension to avoid double move of objects in CIMResponseData (one |
|
into enumerationContext queue and second to new cimResponseData for |
|
response. Want to avoid second move by extending Open/Pull response |
|
messages to include count and CIMResponse data to count objects out |
|
of queue when converting (avoids the second move). This would mean |
|
extending the output writers to allow the count field to be supplied |
|
so they would only create output for up to the count supplied.(Make |
|
this future beyond bug 9676). This is fairly extensive because it |
|
extends beyond CIMResponseData to SCMO and XML writers where the |
|
XmlWriters used by encodeXmlResponse would have to have counters |
|
added. Then instead of copying on getCache we would simply pass the |
|
cache and count on and the writer would take and remove. |
|
4. Add more static tests (currently only OpenEnumerateInstances and |
|
OpenEnumerateInstanceNames covered). |
|
5. Correct issue between operations and HTTP where we are sending |
|
trailers with exceptions. Modify response mechanisms so that we |
|
set non-chunked for all responses where we send error responses to |
|
avoid the trailers. NOTE: There should be now a bug on this in general |
|
where we would want to send an initial error without the trailer. Should |
|
have always done that. |
|
6. It would be more efficient in dispatcher to always use exception for |
|
rejects and change the _reject functions so that they never return |
|
when they reject. This is VERY LOW PRIORITY and primarily saves |
|
a few lines of code in the reject functions and their calls. Means we |
|
would code. |
|
_rejectIfEnumerationToBroad(...); |
|
_rejectIfThisParameterIncorrect(...); |
|
|
|
instead of |
|
if (_rejectIfEnum...) |
|
{ |
|
return true |
|
} |
|
It would mean that the method trace for the handlers would not return |
|
an exit if we rejected. VERY LOW PRIORITY. Possibly FUTURE. No behavior |
|
change, just more compact source code but it messes with the method |
|
trace logic. |
|
7. There are still a couple of template functions around the task of |
|
distributing requests to the multiple providers. |
|
In fact there are two similar but different templates for |
|
the associators(i.e. assoc, ref, etc.) functions and the parallel |
|
openAssoc functions. It would be nice to consolidate that logic and |
|
further to try to create a non-template form for those functions. LOW |
|
PRIORITY |
|
8. Trace functions in CIMResponseData,h & cpp should be PEGASUS_DEBUG. |
|
Same for trace function in EnumerationContext and |
|
EnumerationContextTable |
|
9. Question. We added trace in CIMRequestOperationDispatcher.cpp if |
|
query enum is recevied with no object path. However, since this is |
|
a provider problem should we be doing something other than a trace |
|
(ex. log) since traces are often not really used and they do generate |
|
lots of data whereby something like this could get lost. Also, need |
|
to review what level of trace if we keep trace. |
|
10. Better statistics keeping for open, etc. through cimperf. |
|
11. Incorporate a simple test of pull operations into the TestClient |
|
12. Define a simpler iteration interface for the client so that |
|
the client does not have to directly control the open, pull, etc. |
|
operations. |
| |
In the current release of Pegasus these are all compile time parameters. |
|