Trying to get around ListView Threshold with no server access

Oct 11, 2014 at 8:06 PM

I'm trying to get a list of all checked out documents across a site collection. This works perfectly fine until I hit a list with over 5000 items. My query would not return 5000 items however, but the fact that it's over the threshold prevents me from getting any results back.

I'm not sure how to get around this. I've done some quick research and I know you can override the threshold using
But I don't see a CAML Query equivalent or option to replicate this behavior with SPServices

I know Mark and ptavares discussed this in this old post wondering if there's been any progress regarding this since then.

This is what I'm trying to do:
var cQueryOptions = "<QueryOptions> <IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns><ViewAttributes Scope='RecursiveAll' IncludeRootFolder='True' /></QueryOptions>",

cQueryAllCheckedOutDocuments = '<Query><Where><IsNotNull><FieldRef Name="CheckoutUser" /></IsNotNull></Where></Query>'

for (var i = 0; i < libraryCount; i++) {
            itemPromise[i] = $().SPServices({
                operation: "GetListItems",
                webURL: library[i].siteUrl,
                listName: library[i].listID,
                CAMLViewFields: cViewFieldsLimited,
                CAMLQuery: cQueryAllCheckedOutDocuments,
                CAMLQueryOptions: cQueryOptions,
Also, I can't index any columns now that the list of over the threshold.

I'm trying to find a solution that doesn't rely on me making changes to a list if possible.
Oct 12, 2014 at 1:10 AM
Yeah, I have not found anything else outside of what posted. Did try the one workaround I posted? Maybe that will work .

Marc also recently write a blog post about the thresholds that you may want to check out.

On thing I have not tried is to see if the EnableThrottling can be updated via UpdateList operation. That property is on the List and although the code MS provided us was to be run from PowerShell, I wonder if it can be updated from the client side.
Oct 12, 2014 at 4:27 PM
Hey Paul,

The work around was to wrap my filters in an AND operation right? I wrote what I did below but it didn't help.
            cQueryAllCheckedOutDocuments = "<Query>"
                                                    +"<FieldRef Name='ID' />"
                                                    +"<Value Type='Counter'>0</Value>"
                                                    +"<FieldRef Name='CheckoutUser' />"
                                              +"<FieldRef Name='Modified' Ascending='False' />"
I looked at the UpdateList properties and there's nothing that enables us to enable throttling through webservices.

Is this the post you were talking about? It's good info but the use case doesn't apply unfortunately.
Oct 12, 2014 at 5:16 PM
Yeah, that was the post and the workaround. I can't think of anything outside of involving your IT group and turning off throtlling.

Re: UpdateList
I would try to set it even if it's not documented. Just to see what happens. If you do a GetList you will see that property on it.
Oct 13, 2014 at 1:24 AM
So I tried updating the list doing this:
  operation: "UpdateList",
  listName: "Site Assets",
  listProperties: "<List NoThrottleListOperations='True' ThrottleListOperations='False' MaxItemsPerThrottledOperation='100'/>"
Not happening. Looks like it just can't be done through web services which is disappointing.

It's fine though, I'll just output a message stating lists above the threshold need to be checked manually.
Oct 13, 2014 at 4:52 AM
Such changes must be done before the throttling limit is surpassed. Later on IT involvement is necessary.

As a workaround, you can try to split the query in a few parts with an appropriate RowLimit and only indexed fields in the sorting and filtering, like Excel does.
Oct 13, 2014 at 4:54 AM
In the message you can provide a link to /_layouts/sitemanager.aspx for a similar report, if the user has permissions to access it.
Oct 13, 2014 at 2:26 PM
ACrush wrote:
Such changes must be done before the throttling limit is surpassed. Later on IT involvement is necessary.

As a workaround, you can try to split the query in a few parts with an appropriate RowLimit and only indexed fields in the sorting and filtering, like Excel does.
I'm not sure I follow your work around. Either way, what I'm trying to do is build an application for which an admin can go in and quickly get this report. Ideally, I myself would never have had control over their environment so unless I can bypass this with webservices either through a query or temporarily disabling list throttling, there's not much I can actually do, which is fine.

For now I've just moved on to trying to figure out how to still get this report and inform the user of the libraries that could not be scanned. If I can tell him at 90% of his site was scanned but they have to manually look at 1 library over the threshold, I'm satisfied with that result.
Oct 14, 2014 at 11:51 AM
Edited Oct 14, 2014 at 11:51 AM
Does this work client side? I've never tried it, but it should:

Paging with Web Services

The thing of interest is: ListItemCollectionPositionNext="Paged=TRUE&p_ID=100 for when that link goes dead. ^_^

Oct 14, 2014 at 1:02 PM
Are you saying that adding that to a list View URL gets around the throttling? If so, nice tip. Thanks.
The string looks like a paging token, and although I dont think the property in the SOAP operation is named exactly the same as above, there is a spot for it. Something to try.
Oct 14, 2014 at 1:23 PM
Edited Oct 14, 2014 at 1:24 PM
This page states the <Paging> node can be used within the <QueryOptions>.
  <Paging ListItemCollectionPositionNext=" 
   101" />
You'll get back the next page in the XML, so it's trivial to send back another request based on that. Not sure how this works with throttling though as I've not tried this. THIS does work with JSOM and large lists though, but the set up is a little different.

I have a post on this as well: