version 1.1.2.19, 2014/04/08 00:20:07
|
version 1.1.2.51, 2014/08/27 15:09:22
|
|
|
| |
<<< The TODO section is being maintained during the review and checkin process | <<< The TODO section is being maintained during the review and checkin process |
to keep track of problems, errors, notes, etc. Must be deleted before | to keep track of problems, errors, notes, etc. Must be deleted before |
checkin to head of tree. Please feel free to add notes, etc in this |
release or reduced to those things that are considered FUTURE and not for |
|
this release. Please feel free to add notes, etc in this |
section as you review/test.>>>>>> | section as you review/test.>>>>>> |
| |
TODO list: | TODO list: |
1. Binary operation from OOP. Need to add counter to binary | 1. Binary operation from OOP. Need to add counter to binary |
protocol to be able to count objects in response. Generates | protocol to be able to count objects in response. Generates |
warnings in things like messageserializer and does not work with | warnings in things like messageserializer and does not work with |
OOP right now. Corrected by converting to XML. |
OOP right now. Fixed by converting to XML. Concluded that we do not |
2. OpenExecQuery - Code is incorrect in that it does not include the |
need to do this. The binary response is not really used often |
return from the exec query function to the aggregator yet. |
in the current environment So double mapping it is not a major issue. |
3. Code for Pull part of OpenQueryInstancesRequest a) should be part of |
Leave this as FUTURE |
the common CIMOperationRequestDispatcher execCommon code. |
2. DONE OpenExecQuery - Code is incomplete in that it does not include the |
4. The changes to WQLCIMOperationRequestDispatcher and CQL... for handling |
return from the exec query function to the aggregator yet. Code for Pull |
|
part of OpenQueryInstancesRequest a) should be part of the common |
|
CIMOperationRequestDispatcher execCommon code. DONE, not tested. 29 May |
|
30 May tested with simple cli calls. |
|
3. DONE The changes to WQLCIMOperationRequestDispatcher and CQL... for handling |
pull not completed so we feed the responses back to the EnmerationContext | pull not completed so we feed the responses back to the EnmerationContext |
queues |
queues. DONE Not tested 29 May 30 May DONE, some testing done |
3. Lots of minor TODOs, diagnostics, etc. still in the code |
4. Minor TODOs, diagnostics, etc. still in the code. Reduced almost to |
4. External runtime variables. Proposing that they be fixed for this release |
none now. We are leaving some in as PEGASUS_DEBUG |
rather than set by configuration. This should be discussed. Am making |
5. Extension to avoid double move of objects in CIMResponseData (one |
this a separate bug. See bug 9819 for the changes to cover this. |
|
5. Decision on EnumerationContext timeout (separate thread or just |
|
checks during other operations). Can we, in fact really keep the |
|
enumeration context table and queue under control without monitoring |
|
with a separate thread. We must monitor for: |
|
a. Client operation that stop requesting (i.e. inter operation time |
|
exceeds operationTimeout). Note that if it simply exceeds the time |
|
the next operation does the cleanup. The issue is those clients that |
|
simply stop and do not either close or go to completion. |
|
b. We should protect against providers that no not every finish delivering |
|
or take to long between deliveries. This does not exist in Pegasus |
|
today |
|
6. Consider moving some of the code in dispatcher from templates to common |
|
functions which would mean adding intermediate classes in CIMMessage but |
|
would reduce code size. |
|
7. Extension to avoid double move of objects in CIMResponseData (one |
|
into enumerationContext queue and second to new cimResponseData for | into enumerationContext queue and second to new cimResponseData for |
response. Want to avoid second move by extending Open/Pull response | response. Want to avoid second move by extending Open/Pull response |
messages to include count and CIMResponse data to count objects out | messages to include count and CIMResponse data to count objects out |
of queue when converting (avoids the second move). Big issue here |
of queue when converting (avoids the second move). This would mean |
with binary data since need to extend format to count it. |
extending the output writers to allow the count field to be supplied |
8. NEXT TASKS: |
so they would only create output for up to the count supplied.(Make |
a. test the enumeration timeout thread |
this future beyond bug 9676). This is fairly extensive because it |
b. finish and test the OpenQueryInstances |
extends beyond CIMResponseData to SCMO and XML writers where the |
c. Clean up TODOs |
XmlWriters used by encodeXmlResponse would have to have counters |
d. Find issue when we run makepoststarttests in pullop client with |
added. Then instead of copying on getCache we would simply pass the |
forceProviderProcesses = true. This causes an operation like |
cache and count on and the writer would take and remove. |
cimcli pei CIM_ManagedElement to not complete (client timeout) |
6. Add more static tests (currently only OpenEnumerateInstances and |
sometimes. |
OpenEnumerateInstanceNames covered). |
|
Low priority because most of this except exact XML covered in other |
|
tests. note that we can only add very limited tests since a) no |
|
way to test multiple operations in sequence(i.e. open/pull) and |
|
cannot be sure how many responses will come back in all cases even |
|
for open. The only reason for most of these tests is to assure that |
|
we do not change XML inadvertendtly in future with change that is |
|
compatible to both our server and client but breaks other clients or |
|
servers. In any case we can do only minimal tests because most real |
|
testing involves both open and pull and the static tests are by their |
|
nature single operation. |
|
8. Correct issue between operations and HTTP where we are sending |
|
trailers with exceptions. Modify response mechanisms so that we |
|
set non-chunked for all responses where we send error responses to |
|
avoid the trailers. NOTE: There should be now a bug on this in general |
|
where we would want to send an initial error without the trailer. Should |
|
have always done that. |
|
9. It would be more efficient in dispatcher to always use exception for |
|
rejects and change the _reject functions so that they never return |
|
when they reject. This is VERY LOW PRIORITY and primarily saves |
|
a few lines of code in the reject functions and their calls. Means we |
|
would code. |
|
_rejectIfEnumerationToBroad(...); |
|
_rejectIfThisParameterIncorrect(...); |
|
|
|
instead of |
|
if (_rejectIfEnum...) |
|
{ |
|
return true |
|
} |
|
It would mean that the method trace for the handlers would not return |
|
an exit if we rejected. VERY LOW PRIORITY. Possibly FUTURE. No behavior |
|
change, just more compact source code but it messes with the method |
|
trace logic. |
|
10. There are still a couple of template functions around the task of |
|
distributing requests to the multiple providers. |
|
In fact there are two similiar but different templates for |
|
the associators(i.e. assoc, ref, etc.) functions and the parallel |
|
openAssoc functions. It would be nice to consolidate that logic and |
|
further to try to create a non-template form for those functions. LOW |
|
PRIORITY and not for bug 9676 ACTION: Define in new bug |
|
11. External runtime variables. bug 9676 leaves these variables as |
|
internal to CIMOperationRequestDispatcher.cpp. |
|
See bug 9819 for the changes to externalize these variables, both |
|
the runtime and buildtime variables. The Work group discussion |
|
documents what the variables are and how they tie to statistics info. |
|
See the OpenPegasus Wiki, Pull operations work group. |
|
12. Trace functions in CIMResponseData,h & cpp should be PEGASUS_DEBUG. |
|
Same for trace function in EnumerationContext and |
|
EnumerationContextTable |
|
13.Question. We added trace in CIMRequestOperationDispatcher.cpp if |
|
query enum is recevied with no object path. However, since this is |
|
a provider problem should we be doing something other than a trace |
|
(ex. log) since traces are often not really used and they do generate |
|
lots of data whereby something like this could get lost. Also, need |
|
to review what level of trace if we keep trace. |
|
14. Note that the build-a-class argument on OpenQueryInstances always |
|
returns error since we do not build the class. We propose to deprecate |
|
this in DMTF and not implement in in OpenPegasus. TODO add a bug |
|
defining this limitation. This is not a TODO, just a warning for |
|
the future. TODO add bug about this and add to release notes |
|
15. Clarify trace of completion of each enumeration sequence. |
|
16. Statistics keeping for open, etc. through cimperf. |
|
|
|
NEXT TASKS: |
|
a. finish 8, 12 above. Test against cimperf since there was a bug |
|
in cimperf from before. |
|
|
|
26 August 2014 |
|
1. Add flag to pullop to hide warnings when response smaller than requested |
|
size. This is because the default operation is to return what exists rather |
|
than waiting for a full response to match the MAXOBJECTCNT. Note that |
|
there is a flag in CIMOperationRequestDispatcher that modifies this |
|
behavior. |
|
2. Modify displays in pullop to display less info during poststarttests. |
|
|
|
20 August 2014 |
|
1. Update to greatly reduce the number of KS_TODO comments. We have reduce |
|
the number of these comments to about 50 total of which most are comments |
|
to remove diagnostics. |
|
2. Internationlize a number of messages in XmlReader, etc. This completes |
|
internationalization of messages added for pull operations. |
|
3. Added some detailed messages to cimcli to show what is happening in |
|
the multiple operations. |
|
4. Some efficiency changes (ex. remove String::EMPTY usage) |
|
|
|
18 August 2014 |
|
1. Update to remove some TODOs. |
|
2. Mergeout to the task branch. Note that for some reason there had bee |
|
a descrepency in the Pegasus/Server/Tests/TLS last bug fix not getting |
|
into the previous mergeout |
|
|
|
14 August 2014 |
|
1. Issue with the pullop test in some systems causing an HTTP 413 error ( |
|
too much data). We confirmed that this is probably actually a limit in the |
|
client with enumerate instances (the response is simply to big in the case |
|
where we are asking it to deliver 100,000 responses instances on some systems) |
|
Modified the test for the stress test to be 30,000 instances and added a |
|
manual test testSizeLimit to the make file to try to deliver up to 600,000 |
|
instances to determine limits to the environment. |
|
|
|
Note that this is not actually a failure of the pull operations. In fact |
|
where cimcli ei ... fails with 600,000 instances in the response cimcli pei |
|
works correctly. |
|
|
|
Update the Client/tests/pullop/Makefile stresstests to hopefully pass the |
|
nightly test on more systems. |
|
|
|
2. Eliminated a number of KS_TODOs and code that we had documented out with |
|
//// . Note that there are still a number of the KS_TODOs but most of them |
|
are really isolated around |
|
|
|
19 August 2014 |
|
1. Mergeout to head of tree. We picked up one issue that for some reason |
|
the cvs mergeout was missing, a change to Server/tests/TLS... |
|
|
|
19 August 2014 |
|
1. Remove a number of KS_TODO notes either by cleaning them up or by |
|
change the keyword to FUTURE if they are things that we might consider |
|
for the future |
|
2. Some efficiency improvements in dealing with strings primarily get |
|
rid of use of String::EMPTY in the new code. |
|
|
|
3. Removed some diagnostic trace output that was used during development. |
|
|
|
13 August 2014 |
|
1. Mergeout to head of tree. |
|
|
|
11. August 2014 |
|
1. Correct error in compile in windows (bad reference to sleep(...) |
|
|
|
29 July 2014 |
|
1. Cleanup functionality in the unittests for EnumerationContext and |
|
its table. |
|
|
|
23 July 2014 |
|
1. Mergeout from head of tree to Task branch |
|
|
|
21 July 2014 |
|
1. Removed strings.h from pullop.cpp |
|
2. Minor cleanup per testing |
|
|
|
17 July 2014 |
|
1. Correct error in CIMOperationRequestDispatcher.cpp OpenQueryInstances |
|
when PEGASUS_DISABLE_QUERY set. |
|
2. Correct issue in diagnostic code in CIMResponseData.cpp that used |
|
snprintf function not available in windows. |
|
3. Clean up some miscellaneous code in pullop and Makefile. |
|
4. Minor change to PullErrosrs. |
|
|
|
2 July 2014 |
|
1. Corrected error in the new getoopt functions that caused it to fail |
|
in build and also in it unit test with the --help option. |
|
|
|
28 June 2014 |
|
1. Corrected issue where ZOs was getting errors from pullop because we |
|
used getopt to parse command line and the linux getopt is not behavior |
|
consistent with ZOs. Note that the ZOs getopt matches the posix expect and |
|
the linux one has extensions (notably it permutes non option arguments to |
|
the end of the arguments array. The easiest and most logical was to move |
|
the whole parser to use the internal getoopt which is what we should be |
|
using all the time in any case. |
|
18 June 2014 |
|
1. Correct errors in pullop and CMIMessage.cpp identified by Ji Lee as comments 58,59 |
|
2. Correct error in Makefile identified by Ji Lee as bug comment 60 |
|
|
|
14 June 2014 |
|
1. Mergeout from head of tree. |
|
|
|
12 June 2014 |
|
1. Implement the basic code for the CIMOperationDispatcher portion of bug 9819 |
|
Note that the CIMConfig calls are disabled with comments. |
|
2. Corrected a number of differences in const declaration and definition for |
|
continueOnError and maxObjectCount in CIMClient. This corrects issue |
|
documented in bug 9676 comment 53 |
|
3. Corrected some documentation for Pull operations in CIMClient. |
|
4. Corrected similar issue In EnumerationContext class. See comment 54 |
|
5. Removed ExecInfo.h header from Client/tests/PullErrors/PullErrors.cpp |
|
|
|
30 May 2014 |
|
1. Correct issue in UintArgs. Modified to use smart pointers, consistent |
|
with public interfaces. Tested |
|
2. Complete code for OpenQueryInstances and PullInstances so that the complete |
|
sequence works for both the server and client (i.e. 2, 3 above done |
|
except for more tests in pullop). |
|
3. Fix issue in cimcli where the OpenQueryInstances was outputting a path |
|
component where there is no path transmitted from the server. |
|
4. Moved large block of common code in CQLOperationRequestDispatcher.cpp |
|
and WQLOperationRequestDispatcher.cpp to common function |
|
handleQueryRequestCommon in CIMOperationRequestDispatcher.cpp. These |
|
handlers now reduced to just the parser processing and minimal error |
|
checks. |
|
5. Cleaned up some code in EnumerationContextTable.CreateContext so only |
|
single interface for all requests. |
|
6, Commented out a number of PEG_TRACE messages. Note that they are |
|
commented with 4 /s at line beginning to be easily identified. |
|
//// code |
|
There are still a number of KS_TODOs but they are mostly some questsions, |
|
some internationalization (note that almost none of this environment is |
|
completely internationlaized (see the decoders)) and a couple of questions |
|
about the existing code or exactly what message should be sent. I argue |
|
that the current TODOs should not stop checkin. Will remove after we |
|
have tested couple of days and prove that there is no real need. |
|
7. Added trace in Query response code if there is no objectPath since that |
|
is really an error in the provider |
|
|
|
|
|
29 May 2014 |
|
1. Correct issue in cimcli that was causing rejects of open operations when |
|
there were no filters requested. The standard default is WQL in cimcli |
|
so we overrode this for the pull operations. |
|
2. Clean up OpenQueryInstances in CIMMessage.h and CIMMessage.cpp. |
|
3. Modify CIMOperationRequestDispatcher and |
|
EnumerationContextTable.CreateContext to pass request rather than request |
|
parameters. |
|
4. Remove a number of diagnostic messages. Note that there are still some. |
|
They are marked KS_TODO with with note to remove before checkin. They are |
|
all traces and validations. |
|
5. Correct error in inclusion of execinfo.h include in CIMOperationRequestDis. |
|
found by Ji Lee. |
|
6. Cleaned up execQuery and extended it for OpenQueryInstances. ExecQuery |
|
tested but have not tested OpenQueryInstances yet. One more checkin and we |
|
should be able to close 2,3 above (OpenQueryInstances). |
|
|
|
26 May 2014 |
|
1. Remove a number of diagnostics. |
|
2. Make cache size in EnumerationContextTable dynamic and remove function |
|
to set default values. MaximumEnums set from Dispatcher on constructor for |
|
EnumerationContextTable and table size set to 1/2 of maximum number of |
|
open contexts. |
|
3. Clean up some statistics in EnumerationContextTable |
|
4. Correct error in query and queryLanguage parameters in cimcli in that |
|
the default for queryLanguage is WQL but must be "" for pull operations. |
|
If WQL found they set it to "". |
|
|
|
|
|
9 May 2104 |
|
1. Converted several template functions in CIMOperationRequestDispatcher |
|
to real functions |
|
2. Added intermediate CIMMessage for CIMPullOperationRequestMessage since |
|
all of the pulls have exactly the same data. This allowed us to |
|
convert the template function processPullRequest from template to |
|
real function |
|
3. Added intermediate CIMMessage for CIMOpenOperationRequestMessage betweeh\n |
|
all of the Opens and the CIMRequestData message. This will allow |
|
common code to test all of these variables in all 6 |
|
CIMOperationRequestDispatcher handleOpen... functions |
|
3. Added variable in CIMOperationRequestDispatcher handleEnqueue to make |
|
releasing the original request conditional on return from the handle... |
|
for open... and pull... handlers. |
|
This allowed us to drop one place where we were creating a new request msg |
|
for the delayed response and use the original request. This meant that |
|
we could convert the issueOpenOrPullResponse from template function to |
|
real function. |
|
4. Moved open and pull constructors in CIMMessage.h to CIMMessage.cpp. NOTE: |
|
openQueryInstances not moved. Will do when we get this funciton running. |
|
5. Added test for correct namespace in pull operations. Note that it is |
|
never really used but since supplied with request, we test to be sure it is |
|
same as open request namespace. added internationalized exception for this. |
|
6. We should probably add same test to close just as an input parameter |
|
validation. |
|
6. Modified processPullRequest to close the enumeration context if |
|
incorrect response type received. Before we just returned exception and |
|
left context open for next request. Closing is much more logical since the |
|
client really did send an erronous request. |
|
7. Modified PullErrors to correctly test above changed in behavior. |
|
8. Removed a significant number of diagnostic messages. |
|
9. Added query and querylanguage parameters to cimcli. |
|
|
|
|
|
4 May 2014 |
|
1. Clean interface with EnumerationContext timeout thread |
|
2. Remove more diagnostics |
|
3. Correct issue with enumerationContext Release that was causing |
|
failures during nightly tests. |
|
|
|
2 May 2014 |
|
1. Remove Diagnostics from ResponseData.cpp |
|
2. Correct setup and start of EnumerationContextTimerThread |
|
3. Change limit on max number of enum contexts to be set from Dispatcher. |
|
4. Increase size of hashtable for enum contexts. |
|
5. Corrected issue causing failures in removeContext. Now runs through nightly |
|
tests most of the time. |
|
1 May 2014 |
|
1. Cleanup of code and variable names. |
|
2. Removed namespace variable from EnumerationContext. It was never used. |
|
3. Added test for max number of open enumerations. Right now it is fixed |
|
and set about 100. Note that this is tied to the size of hash table |
|
so that there is a issue with more dynamic setting of max and the |
|
hash table size. |
|
4. Removed several unused functions. |
|
5. Found one possible issue causing thread conflict but not the core one |
|
yet. |
|
|
|
30 April |
|
1. Added exception response to dispatcher for Limits exceeded on |
|
CreateEnumerationContext. Returns Exceeded server limits CIMException. |
|
2. Fixed possible deadlock where we were not unlocking EnumerationContext |
|
under error condition |
|
|
|
|
|
27 April - mergeout to head of tree |
|
1. Mergeout of the code to the head of tree. Documented that mergeout in |
|
the bug |
|
|
|
27 April 2014 |
|
1. Clean up in a number of areas, mostly just code without changing |
|
functionality. |
|
2. Modified CIMMessage.h slightly to clarify names of the pull intermediate |
|
messages (the common open and pull response); |
|
3. Removed a number of diagnostics from CIMResponseData |
|
4. Added new function to ResponseStressTestProvider to be able to terminate |
|
output after a defined number of objects with a CIMException. |
|
5. Removed a number of other TODOs |
|
6. Corrected a couple of errors in the server message bundle. |
|
7. Modified ResponseEncoder to only output the error for pull errors. |
|
It was errounously outputting the EndOfSequence, EnumerationContext and |
|
the error. This way, only the error is sent which is spec. |
|
8. Removed number of TODOs and cleaned up a couple of others. |
|
9. Added static tests for OpenEnumerateInstanceNames. |
|
10. Modified a number of the temporary diagnostics to be permanent by |
|
changing them to PEGASUS_DEBUG_ASSERTs |
|
|
|
22 April 2014 |
|
1. Clean up some messages. |
|
2. Still trying to find issue that randomly keeps messages in timeout |
|
queue long after they have been closed and returned. |
|
3. Fixed memory loss error. |
|
4. Added zero reject for OperationTimeout because that effectively disables |
|
the whole timeout mechanism so that enumerations could be started, never |
|
completed by the client and would just hang around in memory. That |
|
would be a significant DDOS attack (start very large enum with no |
|
timeout) and never do pulls after open. |
|
|
|
21 April 2014 |
|
1. Reorganized code in CIMOperationRequestDecoder.cpp ...Iparam classes. |
|
Cleaned up the rejectCIMException code. |
|
2. Cleanup some issues in the EnumerationContextTimeout Thread. |
|
3. Fixed issue in pullop where it was using zero as default for default |
|
operation timeout where it should have been NULL, the DMTF specified |
|
default. |
|
4. Enabled the test to reject 0 as operation timeout on a request and but |
|
controled by a #define in CIMOperationRequestDispatcher.cpp |
|
5. Added test for excessive new OperationContext entries in table. |
|
|
|
18 April 2014 |
|
1. Internationalize remaining messages in CIMOperationRequestDecoder.cpp and |
|
clean up the decoders for some of the messages so they use the |
|
...Iparam classes. Note that we did not clean up all of the old message |
|
decoders, just the new ones and we did not clean up the Count message |
|
since it is deprecated. TODO clean these up so the decoder is really |
|
consistent. |
|
2. Enabled the EnumerationContext Timer and removed code for alternate |
|
non-thread solution. |
|
|
|
17 April 2014 |
|
1. Reorganized the lock/unlock for handling the EnumerationContext because |
|
there were thread windows (i.e. using the context after it was released |
|
by other thread). |
|
2. Modified code for the InteroperationTimerThread and deleted old code that |
|
allowed this whole thread to by bypassed. |
|
3. Internationalized a number of CIMOperationRequestDispatcher CIMException |
|
response messages. |
|
4. Fixed stress tests in Client/tests/pullop that were getting timeout. The |
|
issue was not the pull code but that fact that the idle client timeout |
|
appears to be measureing to the end of the operation and the operation |
|
was taking more than 40 seconds which is the default timeout. The idle |
|
timeout takes no account of the fact that chunks are being regularly |
|
delivered. |
|
5. Several minor code changes. |
|
NOTE: At this point the code passes extensive tests of the pull operations |
|
both in OOP and none OOP mode (but with lots of diagnostics that may |
|
be affecting timing). |
|
|
|
10 April 2014 |
|
1. Fix issues causing failure with repeated pull operations. |
|
2. Consolidate some of the CIMOperationRequestDispatcher.cpp template code |
| |
7 April 2104 | 7 April 2104 |
MergeOut of head of tree. | MergeOut of head of tree. |