(file) Return to readme.pulloperations CVS log (file) (dir) Up to [Pegasus] / pegasus

Diff for /pegasus/readme.pulloperations between version 1.1.2.28 and 1.2

version 1.1.2.28, 2014/05/01 18:58:23 version 1.2, 2014/08/27 23:10:06
Line 4 
Line 4 
  
 <<< The TODO section is being maintained during the review and checkin process <<< The TODO section is being maintained during the review and checkin process
 to keep track of problems, errors, notes, etc.  Must be deleted before to keep track of problems, errors, notes, etc.  Must be deleted before
 checkin to head of tree. Please feel free to add notes, etc in this  release or reduced to those things that are considered FUTURE and not for
   this release. Please feel free to add notes, etc in this
 section as you review/test.>>>>>> section as you review/test.>>>>>>
  
 TODO list: TODO list:
    1. Binary operation from OOP.  Need to add counter to binary    1. Binary operation from OOP.  Need to add counter to binary
       protocol to be able to count objects in response. Generates       protocol to be able to count objects in response. Generates
       warnings in things like messageserializer and does not work with       warnings in things like messageserializer and does not work with
       OOP right now.  Corrected by converting to XML.        OOP right now.  Fixed by converting to XML. Concluded that we do not
    2. OpenExecQuery - Code is incomplete in that it does not include the        need to do this. The binary response is not really used often
       return from the exec query function to the aggregator yet.        in the current environment So double mapping it is not a major issue.
    3. Code for Pull part of OpenQueryInstancesRequest a) should be part of        Leave this as FUTURE
       the common CIMOperationRequestDispatcher execCommon code.     2. DONE OpenExecQuery - Code is incomplete in that it does not include the
    4. The changes to WQLCIMOperationRequestDispatcher and CQL... for handling        return from the exec query function to the aggregator yet. Code for Pull
         part of OpenQueryInstancesRequest a) should be part of the common
         CIMOperationRequestDispatcher execCommon code. DONE, not tested. 29 May
          30 May tested with simple cli calls.
      3. DONE The changes to WQLCIMOperationRequestDispatcher and CQL... for handling
       pull not completed so we feed the responses back to the EnmerationContext       pull not completed so we feed the responses back to the EnmerationContext
       queues        queues. DONE Not tested 29 May 30 May DONE,  some testing done
    3. Minor TODOs, diagnostics, etc. still in the code     4. Minor TODOs, diagnostics, etc. still in the code. Reduced almost to
    4. External runtime variables. Proposing that they be fixed for this release        none now.  We are leaving some in as PEGASUS_DEBUG
       rather than set by configuration.  This should be discussed.  Am making     5. Extension to avoid double move of objects in CIMResponseData (one
       this a separate bug.  See bug 9819 for the changes to cover this.  
    5. Consider moving some more of the code in dispatcher from templates to  
       common functions which would mean adding intermediate classes in  
       CIMMessage but would reduce code size. Currently the change that  
       we should make for this is to create a new message CIMPullRequestData...  
       between CIMRequestDataMessage and the Pull requests since they all  
       have exactly the same input and output pararameters (The  
       CIMResponseData hides the response object differences)  
    6. Extension to avoid double move of objects in CIMResponseData (one  
       into enumerationContext queue and second to new cimResponseData for       into enumerationContext queue and second to new cimResponseData for
       response.  Want to avoid second move by extending Open/Pull response       response.  Want to avoid second move by extending Open/Pull response
       messages to include count and CIMResponse data to count objects out       messages to include count and CIMResponse data to count objects out
       of queue when converting (avoids the second move). This would mean       of queue when converting (avoids the second move). This would mean
       extending the output writers to allow the count field to be supplied       extending the output writers to allow the count field to be supplied
       so they would only create output for up to the count supplied.        so they would only create output for up to the count supplied.(Make
    9. Add more static tests (currently only OpenEnumerateInstances and        this future beyond bug 9676). This is fairly extensive because it
         extends beyond CIMResponseData to SCMO and XML writers where the
         XmlWriters used by encodeXmlResponse would have to have counters
         added. Then instead of copying on getCache we would simply pass the
         cache and count on and the writer would take and remove.
      6. Add more static tests (currently only OpenEnumerateInstances and
       OpenEnumerateInstanceNames covered).       OpenEnumerateInstanceNames covered).
       Low priority because most of this except exact XML covered in other       Low priority because most of this except exact XML covered in other
       tests. note that we can only add very limited tests since a) no       tests. note that we can only add very limited tests since a) no
       way to test multiple opertions in sequence(i.e. open/pull) and        way to test multiple operations in sequence(i.e. open/pull) and
       cannot be sure how many responses will come back in all cases even       cannot be sure how many responses will come back in all cases even
       for open.        for open.  The only reason for most of these tests is to assure that
    10. Could clean up the CIMOperationRequestDecoder.cpp rejectCIMException        we do not change XML inadvertendtly in future with change that is
       further since there is some overlap.  That is low priority.        compatible to both our server and client but breaks other clients or
    11. Correct issue between operations and HTTP where we are sending        servers. In any case we can do only minimal tests because most real
       trailers with exceptions.  Also correct issue where we are        testing involves both open and pull and the static tests are by their
       sending endOfSequence and EnumerationContext in response when        nature single operation.
       we send CIMException.  DONE Except that we really need to get     8. Correct issue between operations and HTTP where we are sending
       rid of the trailers completely for pull operations.  Need to        trailers with exceptions. Modify response mechanisms so that we
       have flag to send as non trailer (probably non_chunked flag)        set non-chunked for all responses where we send error responses to
         avoid the trailers. NOTE: There should be now a bug on this in general
         where we would want to send an initial error without the trailer. Should
         have always done that.
      9. It would be more efficient in dispatcher to always use exception for
          rejects and change the _reject functions so that they never return
          when they reject. This is VERY LOW PRIORITY and primarily saves
          a few lines of code in the reject functions and their calls.  Means we
          would code.
          _rejectIfEnumerationToBroad(...);
          _rejectIfThisParameterIncorrect(...);
   
          instead of
          if (_rejectIfEnum...)
          {
              return true
          }
          It would mean that the method trace for the handlers would not return
          an exit if we rejected. VERY LOW PRIORITY. Possibly FUTURE. No behavior
          change, just more compact source code but it messes with the method
          trace logic.
       10. There are still a couple of template functions around the task of
          distributing requests to the multiple providers.
          In fact there are two similiar but different templates for
          the associators(i.e. assoc, ref, etc.) functions and the parallel
          openAssoc functions.  It would be nice to consolidate that logic and
          further to try to create a non-template form for those functions. LOW
          PRIORITY and not for bug 9676 ACTION: Define in new bug
       11. External runtime variables. bug 9676 leaves these variables as
          internal to CIMOperationRequestDispatcher.cpp.
          See bug 9819 for the changes to externalize these variables, both
          the runtime and buildtime variables.  The Work group discussion
          documents what the variables are and how they tie to statistics info.
          See the OpenPegasus Wiki, Pull operations work group.
       12. Trace functions in CIMResponseData,h & cpp should be PEGASUS_DEBUG.
          Same for trace function in EnumerationContext and
          EnumerationContextTable
       13.Question. We added trace in CIMRequestOperationDispatcher.cpp if
          query enum is recevied with no object path.  However, since this is
          a provider problem should we be doing something other than a trace
          (ex. log) since traces are often not really used and they do generate
          lots of data whereby something like this could get lost. Also, need
          to review what level of trace if we keep trace.
      14. Note that the build-a-class argument on OpenQueryInstances always
          returns error since we do not build the class.  We propose to deprecate
          this in DMTF and not implement in in OpenPegasus. TODO add a bug
          defining this limitation. This is not a TODO, just a warning for
          the future. TODO add bug about this and add to release notes
      15. Clarify trace of completion of each enumeration sequence.
      16. Statistics keeping for open, etc. through cimperf.
  
    NEXT TASKS:    NEXT TASKS:
       b. Finish and test the OpenQueryInstances        a. finish 8, 12 above. Test against cimperf since there was a bug
       c. Clean up TODOs        in cimperf from before.
       d. Find issue that is causing occasional failures under load.  It  
          is probably a threading issue between the 3 possible threads  
          that use the EnumerationContext (request thread, provider thread,  
          timeout thread) and a place where not mutuxed, primarily the  
          timeout thread.  
  
   26 August 2014
   1. Add flag to pullop to hide warnings when response smaller than requested
      size. This is because the default operation is to return what exists rather
      than waiting for a full response to match the MAXOBJECTCNT.  Note that
      there is a flag in CIMOperationRequestDispatcher that modifies this
      behavior.
   2. Modify displays in pullop to display less info during poststarttests.
   
   20 August 2014
   1. Update to greatly reduce the number of KS_TODO comments.  We have reduce
      the number of these comments to about 50 total of which most are comments
      to remove diagnostics.
   2. Internationlize a number of messages in XmlReader, etc. This completes
      internationalization of messages added for pull operations.
   3. Added some detailed messages to cimcli to show what is happening in
      the multiple operations.
   4. Some efficiency changes (ex. remove String::EMPTY usage)
   
   18 August 2014
   1. Update to remove some TODOs.
   2. Mergeout to the task branch.  Note that for some reason there had bee
      a descrepency in the Pegasus/Server/Tests/TLS last bug fix not getting
      into the previous mergeout
   
   14 August 2014
   1. Issue with the pullop test in some systems causing an HTTP 413 error (
   too much data).  We confirmed that this is probably actually a limit in the
   client with enumerate instances (the response is simply to big in the case
   where we are asking it to deliver 100,000 responses instances on some systems)
   Modified the test for the stress test to be 30,000 instances and added a
   manual test testSizeLimit to the make file to try to deliver up to 600,000
   instances to determine limits to the environment.
   
   Note that this is not actually a failure of the pull operations. In fact
   where cimcli ei ... fails with 600,000 instances in the response cimcli pei
   works correctly.
   
   Update the Client/tests/pullop/Makefile stresstests to hopefully pass the
   nightly test on more systems.
   
   2. Eliminated a number of KS_TODOs and code that we had documented out with
      //// . Note that there are still a number of the KS_TODOs but most of them
      are really isolated around
   
   19 August 2014
   1. Mergeout to head of tree.  We picked up one issue that for some reason
      the cvs mergeout was missing, a change to Server/tests/TLS...
   
   19 August 2014
   1. Remove a number of KS_TODO notes either by cleaning them up or by
      change the keyword to FUTURE if they are things that we might consider
      for the future
   2. Some efficiency improvements in dealing with strings primarily get
      rid of use of String::EMPTY in the new code.
   
   3. Removed some diagnostic trace output that was used during development.
   
   13 August 2014
   1. Mergeout to head of tree.
   
   11. August 2014
   1. Correct error in compile in windows (bad reference to sleep(...)
   
   29 July 2014
   1. Cleanup functionality in the unittests for EnumerationContext and
      its table.
   
   23 July 2014
   1. Mergeout from head of tree to Task branch
   
   21 July 2014
   1. Removed strings.h from pullop.cpp
   2. Minor cleanup per testing
   
   17 July 2014
   1. Correct error in CIMOperationRequestDispatcher.cpp OpenQueryInstances
      when PEGASUS_DISABLE_QUERY set.
   2. Correct issue in diagnostic code in CIMResponseData.cpp that used
      snprintf function not available in windows.
   3. Clean up some miscellaneous code in pullop and Makefile.
   4. Minor change to PullErrosrs.
   
   2 July 2014
   1. Corrected error in the new getoopt functions that caused it to fail
      in build and also in it unit test with the --help option.
   
   28 June 2014
   1. Corrected issue where ZOs was getting errors from pullop because we
      used getopt to parse command line and the linux getopt is not behavior
      consistent with ZOs.  Note that the ZOs getopt matches the posix expect and
      the linux one has extensions (notably it permutes non option arguments to
      the end of the arguments array.  The easiest and most logical was to move
      the whole parser to use the internal getoopt which is what we should be
      using all the time in any case.
   18 June 2014
   1. Correct errors in pullop and CMIMessage.cpp identified by Ji Lee as comments 58,59
   2. Correct error in Makefile identified by Ji Lee as bug comment 60
   
   14 June 2014
   1. Mergeout from head of tree.
   
   12 June 2014
   1. Implement the basic code for the CIMOperationDispatcher portion of bug 9819
      Note that the CIMConfig calls are disabled with comments.
   2. Corrected a number of differences in const declaration and definition for
      continueOnError and maxObjectCount in CIMClient. This corrects issue
      documented in bug 9676 comment 53
   3. Corrected some documentation for Pull operations in CIMClient.
   4. Corrected similar issue In EnumerationContext class. See comment 54
   5. Removed ExecInfo.h header from Client/tests/PullErrors/PullErrors.cpp
   
   30 May 2014
   1. Correct issue in UintArgs.  Modified to use smart pointers, consistent
      with public interfaces. Tested
   2. Complete code for OpenQueryInstances and PullInstances so that the complete
      sequence works for both the server and client (i.e. 2, 3 above done
      except for more tests in pullop).
   3. Fix issue in cimcli where the OpenQueryInstances was outputting a path
      component where there is no path transmitted from the server.
   4. Moved large block of common code in CQLOperationRequestDispatcher.cpp
      and WQLOperationRequestDispatcher.cpp to common function
      handleQueryRequestCommon in CIMOperationRequestDispatcher.cpp. These
      handlers now reduced to just the parser processing and minimal error
      checks.
   5. Cleaned up some code in EnumerationContextTable.CreateContext so only
      single interface for all requests.
   6, Commented out a number of PEG_TRACE messages.  Note that they are
      commented with 4 /s at line beginning to be easily identified.
      //// code
      There are still a number of KS_TODOs but they are mostly some questsions,
      some internationalization (note that almost none of this environment is
      completely internationlaized (see the decoders)) and a couple of questions
      about the existing code or exactly what message should be sent.  I argue
      that the current TODOs should not stop checkin. Will remove after we
      have tested couple of days and prove that there is no real need.
   7. Added trace in Query response code if there is no objectPath since that
      is really an error in the provider
   
   
   29 May 2014
   1. Correct issue in cimcli that was causing rejects of open operations when
      there were no filters requested. The standard default is WQL in cimcli
      so we overrode this for the pull operations.
   2. Clean up OpenQueryInstances in CIMMessage.h and CIMMessage.cpp.
   3. Modify CIMOperationRequestDispatcher and
      EnumerationContextTable.CreateContext to pass request rather than request
      parameters.
   4. Remove a number of diagnostic messages.  Note that there are still some.
      They are marked KS_TODO with with note to remove before checkin. They are
      all traces and validations.
   5. Correct error in inclusion of execinfo.h include in CIMOperationRequestDis.
      found by Ji Lee.
   6. Cleaned up execQuery and extended it for OpenQueryInstances.  ExecQuery
      tested but have not tested OpenQueryInstances yet. One more checkin and we
      should be able to close 2,3 above (OpenQueryInstances).
   
   26 May 2014
   1. Remove a number of diagnostics.
   2. Make cache size in EnumerationContextTable dynamic and remove function
      to set default values. MaximumEnums set from Dispatcher on constructor for
      EnumerationContextTable and table size set to 1/2 of maximum number of
      open contexts.
   3. Clean up some statistics in EnumerationContextTable
   4. Correct error in query and queryLanguage parameters in cimcli in that
      the default for queryLanguage is WQL but must be "" for pull operations.
      If WQL found they set it to "".
   
   
   9 May 2104
   1. Converted several  template functions in CIMOperationRequestDispatcher
      to real functions
   2. Added intermediate CIMMessage for CIMPullOperationRequestMessage since
      all of the pulls have exactly the same data. This allowed us to
      convert the template function processPullRequest from template to
      real function
   3. Added intermediate CIMMessage for CIMOpenOperationRequestMessage betweeh\n
      all of the Opens and the  CIMRequestData message.  This will allow
      common code to test all of these variables in all 6
      CIMOperationRequestDispatcher handleOpen... functions
   3. Added variable in  CIMOperationRequestDispatcher handleEnqueue to make
      releasing the original request conditional on return from the handle...
      for open... and pull... handlers.
      This allowed us to drop one place where we were creating a new request msg
      for the delayed response and use the original request.  This meant that
      we could convert the issueOpenOrPullResponse from template function to
      real function.
   4. Moved open and pull constructors in CIMMessage.h to CIMMessage.cpp. NOTE:
      openQueryInstances not moved.  Will do when we get this funciton running.
   5. Added test for correct namespace in pull operations.  Note that it is
      never really used but since supplied with request, we test to be sure it is
      same as open request namespace.  added internationalized exception for this.
   6. We should probably add same test to close just as an input parameter
      validation.
   6. Modified processPullRequest to close the enumeration context if
      incorrect response type received.  Before we just returned exception and
      left context open for next request.  Closing is much more logical since the
      client really did send an erronous request.
   7. Modified PullErrors to correctly test above changed in behavior.
   8. Removed a significant number of diagnostic messages.
   9. Added query and querylanguage parameters to cimcli.
   
   
   4 May 2014
   1. Clean interface with EnumerationContext timeout thread
   2. Remove more diagnostics
   3. Correct issue with enumerationContext Release that was causing
      failures during nightly tests.
   
   2 May 2014
   1. Remove Diagnostics from ResponseData.cpp
   2. Correct setup and start of EnumerationContextTimerThread
   3. Change limit on max number of enum contexts to be set from Dispatcher.
   4. Increase size of hashtable for enum contexts.
   5. Corrected issue causing failures in removeContext. Now runs through nightly
      tests most of the time.
 1 May 2014 1 May 2014
 1. Cleanup of code and variable names. 1. Cleanup of code and variable names.
 2. Removed namespace variable from EnumerationContext.  It was never used. 2. Removed namespace variable from EnumerationContext.  It was never used.

Legend:
Removed from v.1.1.2.28  
changed lines
  Added in v.1.2
No CVS admin address has been configured
 Powered by
ViewCVS 0.9.2