SPUpdateMultipleListItems Item limit

Apr 15, 2013 at 7:59 PM
Hello,

I need to update multiple list items to kick the workflow on all the items in a list. I am using SPUpdateMultipleListItems and it works fine as long as list has less than 500 items. When i try to update more than 500 items it fails (without any error message).

Is there a limit on number of items ? any workaround to update all the list items. I have around 3k list items. Any help will be much appreciated. Thanks !!
Coordinator
Apr 15, 2013 at 8:05 PM
I haven't ever tried to test with that many items, frankly. I didn't build the function to be efficient at that level.

You could take a look at how the function works and probably come up with something more workable for your special case. It's really just a GetListItems followed by an UpdateListItems on that set of items. My guess is that the batch is simply too big at your number of items.

If you come up with anytrhing that you think would be a general improvement, please let me know.

M.
Apr 15, 2013 at 8:24 PM
Hi Marc,

Thank you very much for your response. I will have a look if I can find some way around otherwise I will have to break my query to retrive less than 500 items at a time.
Apr 18, 2013 at 4:00 AM
I have had to update large amounts of data in a 'batch' way and did so by writing my own function that uses the UpdateListItem operation. I'm my case, I found that doing 100 at a time was a good number (kept the message size across the wire reasonable).
Here is the design I used:

- create a function that accepts an array of all the updates you want done (number of updates don't matter). Set that function to return a jQuery promise which will be resolved once all updates are done. This allows for your program to "attach" logic that you may want to execute later when all updates are done.
- in the function, set up your code to loop through the array an build your update contract up until your set limit (ex 100 items). Once that limit is hit, make the update (with async: true) and when that update is done, get back to the loop and continue on to do the same thing until the entire array is done.
- once all updates are done, resolve the Deferred, so that any functionally you may have waiting on this update can be executed.

Hope this helps.

(If I remember tomorrow, I'll see if I can find the function I wrote. I think I built it to be generic and specific to the application I used it on, so you may be able to reuse it. )

_____
Paul

Sent from mobile device.
Coordinator
Apr 18, 2013 at 5:24 AM
Paul:

It sounds like you approach might be an improvement over my brute force approach. I'd be interested in seeing your function if you can dig it up.

M.
Apr 19, 2013 at 7:06 PM
Hey Mark...
Here you go... Took a little while for me to find it (have not touched that project in over 1.5 years) and I had to make a few modification to remove specific dependencies on the original implementation, but should work...
function bulkUpdate (options) {
    
    // TODO: When resolveing Deferred or calling user's callback, need to find a way to provide the input params (xData) in the same format as the one used by SPServices.
    
    return $.Deferred(function(dfd){
        
        var opt = $.extend(true, {},
                    {
                        updates:    [],
                        batchSize:  100,
                        SPServicesOptions: {
                            operation: 'UpdateListItems',
                            async:      true
                        }
                    },
                    options
                );
        
        $.extend(
            opt,{
                _completeFunc:      null,
                _xData:             null,
                _xDataResultsEle:   null,
                _status:            'success',
                _lastIndex:         -1,
                _totalUpdates:      0
            });
        
        opt._totalUpdates = opt.updates.length;
        
        if (!opt._totalUpdates) {
            dfd.resolve();
            return;
        }
        
        // If caller defined a callback on SPServices, store it.
        if ($.isFunction(opt.SPServicesOptions.completefunc)) {
            opt._completeFunc = opt.SPServicesOptions.completefunc;
        }
        
        // Create the SPServices completefunc.
        opt.SPServicesOptions.completefunc = function(xData, status){
            
            // If there was a communications error, then no point in
            // continuing... error now and resolve Deferred.
            if (status === "error") {
                
                // If user had defined a callback, then call it now
                if ($.isFunction(opt._completeFunc)) {
                    opt._completeFunc.call($, xData, status);
                }
                
                dfd.rejectWith($, [xData, status]);
                return;
            }
            
            // If we did not yet create a response msg object, do it now
            if (opt._xData === null) {
                opt._xData = $(xData.responseXML);
                opt._xDataResultsEle = opt._xData.find("Results");
            }
            
            // Store the responses
            opt._status = status;
            
            $(xData.responseXML)
                .find("Results Result")
                    .clone()
                    .appendTo(opt._xDataResultsEle);
            
            // If the last item processed did not reach the end of the updates,
            // then call makeUpdates() again.
            if (opt._lastIndex < opt._totalUpdates) {
                
                makeUpdates();
            
            // ELse, we reached the end... resolve Deferred
            } else {
            
                // If user had defined a callback, then call it now
                if ($.isFunction(opt._completeFunc)) {
                    opt._completeFunc.call($, opt._xData, opt._status);
                }
                
                // Resolve deferred.
                dfd.resolveWith($, [opt._xData, opt._status]);
                
            }
            
            return;
            
        }
        
        // Loops through list of updates
        function makeUpdates() {
            
            var counter = 0,
                updXml  = '';
            
            // build update contract
            while ( counter < opt.batchSize && opt._lastIndex < opt._totalUpdates ) {
                
                counter++;
                opt._lastIndex++;
                
                updXml += opt.updates[opt._lastIndex];
                
            }
            
            opt.SPServicesOptions.updates = "<Batch OnError='Continue'>" + updXml + "</Batch>";
            
            // make call to SP
            $().SPServices(opt.SPServicesOptions);
            
        }
        
        makeUpdates();
        
    })
    .promise();
    
}//end: function: bulkUpdate()

Apr 20, 2013 at 4:40 AM
Forgot to post the usage for the function above...

    /**
     * GIven an array of updates (<Method/>), this method will
     * make the updates in batches and resolve the Deferred when
     * it is complete.
     *  
     * @param {Object} options
     * 
     * @param {Array} options.updates
     *                  An array of strings representing the udpates
     *                  to be done. Each update is essencially a
     *                  '<Method></Method>' xml element containing
     *                  the udpate Sharepoint will make.
     * 
     * @param {Object} options.SPServicesOptions
     *                  An object with the options for SPServices
     *                  UpdateListItems operation.
     * 
     * @param {Integer} [options.batchSize=100]
     *                  How many updates to make on each call to the
     *                  server.
     * 
     * @return {jQuery.Promise}
     *                  A Deferred promise will be return which
     *                  will be resolved once all udpates are done.
     *                  Promise will be given the 2 parameters that
     *                  are normally available to SPServices completefunc
     *                  callback (xData, status). Only difference is that
     *                  xData is a jQuery object representing the
     *                  XMLDocument object.
     * 
     * @example
     * 
     * Agile.bulkUpdate({
     *      SPServicesOptions: {
     *          listName: "Anoucements"
     *      },
     *      updates: [
     *            "<Method ID='1' Cmd='Update'>" +
     *               "<Field Name='Title'>New Title Value</Field>" +
     *               "<Field Name='ID'>1234</Field>" +
     *            "</Method>",
     *            ...
     *      ]
     * })
     * .done(function(xData, status){
     *     alert("all updates done!")
     * })
     * .fail(function(xData, status){
     *     alert("Update failed.")
     * });
     * 
     */
(turns out that I found a good spot for using this function in the "Black Magic Solutions for White Hat SharePoint" project)

Paul.
Coordinator
Apr 20, 2013 at 5:11 PM
I always love looking at your code! It's always so clean and well-organized. You've also been ahead of me on things like $.Deferred for a long time.

Thanks for sharing.

M.
Apr 20, 2013 at 8:05 PM
Thanks Marc for the complement.
May 26, 2013 at 10:36 AM
Edited May 26, 2013 at 10:36 AM
Hello ptavares, many thanks, I'm very eager to use this function, it is exactly something I need to use. I just need to ask you, what is the object "Agile" you are using in that example?
Ie. You have "Agile.Bulkupdate()". So, please tell me, What is Agile? Is it the array of batch <Method>'s? Or?
May 26, 2013 at 3:35 PM
Bbd,

It took me a while to figure out what you were talking about. I could not see that reference in the code; but then found it in the jsDoc for the function.

Agile was the namespace where I originally had this method. You should ignore it. Sorry for the confusion.


_____
Paul

Sent from mobile device.
Jul 26, 2013 at 8:09 PM
ptavares...it appears the code you have provided in this thread may be what I am looking for to do my multiple list item updates in batches of 100...please forgive the simpleness of my question...how do I use this code with my SP list? I cant imagine it being as simple as a copy/paste...and I cannot see where I would enter my list information and the information I required to be updated. Any help and patience is appreciated!
Jul 26, 2013 at 8:45 PM
@williamnevarez

Actually, it should be :) You can just copy/paste the function above (bulkupdate) and use it like this:
bulkUpdate({
    SPServicesOptions: {
        listName: "Anoucements"
    },
    updates: [
        "<Method ID='1' Cmd='Update'>" +
        "<Field Name='Title'>New Title Value</Field>" +
        "<Field Name='ID'>1234</Field>" +
        "</Method>"
     ]
})
.done(function(xData, status){
    
    alert("all updates done!");
    
})
.fail(function(xData, status){
    
    alert("Update failed.");
    
});
The other input parameters are documented above in the post right after the function I posted... They are:
    updates
    SPServicesOptions
    batchSize
Paul.
Jul 26, 2013 at 9:43 PM
Paul,
Thank you for your quick response. So I tried what you advised and I am not having any success. I copy/pasted the bulkUpdate function exactly as it was indicated above with no revisions on my part. I also copy/pasted the code you provided with revisions specific to my SP list including the CAML to only update the desired items, and currentdate is a global var while CurrentDate is the actual field name:

bulkUpdate({
SPServicesOptions: {
    listName: "Process Design Engagement and Intake",
    CAMLQuery: "<Query><Where><Eq><FieldRef Name='stringFormStatus' /><Value Type='text'>Active</Value></Eq></Where></Query>"
},
updates: [
    "<Method ID='1' Cmd='Update'>" +
    "<Field Name='CurrentDate'>" + currentdate + "</Field>" +
    "</Method>"
 ]
})
.done(function(xData, status){
alert("all updates done!");
})
.fail(function(xData, status){
alert("Update failed.");
});
}
Jul 27, 2013 at 3:48 PM
@williamevarez,
I think that you're trying to do something like: UPDATE all items WHERE stringFromStatus=Active
This function is not going to help you with that... With Client side programing (using the SOAP interface) there is no direct way to do that. You should also review the UpdatelistItems operation documentation and the MS documentation to get a better understanding of how the service works.

There is a Solution
With client side apps, you need to specifically know all of your updates upfront. So in your case, using the snippet above, you will need to do a GetListItems for everthing in the list with a stringFromStatus=Active and a CurrentDate != currentdate ... then you need to build a list (Array) of updates for each one that GetListItems returns.. Each update "contract" looks like this:
<Method ID='1' Cmd='Update'>
    <Field Name='CurrentDate'>currentdate</Field>
    <Field Name='ID'>1234</Field>
</Method>
Note the <Field Name="ID"> element - each update must have one of these. It tells SP which element needs to receive the update.

Depending on how many records need to be updated, you should not send them all at once to the server... The Function that I posted above basically throttles the updates in batches of 100 (configurable with input param) and when all updates are done, resolves the Deferred, which essentially calls your next action (callback).

Again, the key is that you need to create your list of updates yourself... The SP service is not going to do that for you by using a "WHERE" type of clause.

BTW: You did not post any errors or issues, so I assume you did not see any? Did you see the alerts? if so, which one?
My assumption is that you saw the "all updates done!" although when you checked the list, nothing was updated... My function does not fail when SP returns an error message (which I think it might have in this case)... If you look at the raw response (using browser debuging tools) you will probably see that..

Paul.
Jul 29, 2013 at 3:47 PM
You are 100% correct on what I am trying to accomplish; and thank you for taking the time to clarify my next steps and most importantly why I must take them. I was confused on the Field ID portion of the script, and it now makes more sense to me. You were also correct in that I did not receive any error messages and continued to received the all updates complete alert. I will tackle the solution you suggested and see what results I have. I cant thank you and Marc enough for all of your help and support!!!
Jul 29, 2013 at 5:33 PM
Glad I could help... Good luck.


_________
Paul T


Coordinator
Aug 5, 2013 at 12:43 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.