Should SPServices provide option to return JSON?

May 11, 2011 at 6:48 PM

Just starting to explore SPServices and noticed that it returns XML only.   Marc recently put out a blog post requesting opinions on whether SPServices should support this natively or not.  I wanted to start a discussion thread to support the inclusion on JSON as an output.  It is preferrable to have JSON since it's lighter on the wire.  Trend or not, it seems that is now the standard for light weight data transport over the wire.  If heavy jQuery, most plugins are written to support JSON, not XML.  Yes, I can add a plugin and do a conversion, but it would be nicer to have one less step.  Less code to maintain = less bugs.

http://sympmarc.com/2011/04/28/spservices-xml-json-and-restthoughts/

Luke Holder has written a nice little utility to convert from the XML return from SharePoint to JSON.  

https://github.com/lukeholder/jquery-sharepoint-json
jQuery tmpl is a great example of why JSON is needed.  This is an official plugin now maintained by the jQuery team.  
http://stephenwalther.com/blog/archive/2010/11/30/an-introduction-to-jquery-templates.aspx

So questions for discussion on having SPServices support json:

1.  Should SPServices add Luke's or another method to easily convert to JSON?

2.  Could there be a SP 2010 only version that used the client object model to only get JSON over the wire, so no conversion would be necessary?  I'm assuming SPServices is currently coded against .asmx only

PS:  Great work on this tool Marc.  And well documented which is a rarity in the open source world.

Coordinator
May 12, 2011 at 8:54 PM

Jordan:

Thanks for summarizing. A few questions/observations/snarky comments. ;+)

  • For which Web Service operations would you want to see JSON returned? People seem to be saying "SPServices need to return JSON", but there are 197 operations which I support in SPServices. My gut tells me that you may all be talking about GetListItems, and maybe one or two other operations? In other words, it seems to be a general wish without very specific requirements.
  • To date, there have been about 5 people who have asked about JSON. Does using Luke's plugin cover the needs?
  • SharePoint's Web Services return XML. Period. So to get JSON, I'd have to add some overhead. You don't get it for free.
  • If you use the Client OM, then you don't need SPServices. It wouldn't make any sense to write something into SPServices to pass along the RESTful call to the Client OM and process it; just do it directly.

And finally, thanks. Glad you like it!

M.

May 12, 2011 at 9:03 PM

So do you feel that now that SP 2010 has the Client OM, that that replaces what SPServices offers in certain operations such as GetListItems()?  Regardless of Json or XML return type.

Coordinator
May 12, 2011 at 9:10 PM
Edited May 12, 2011 at 9:10 PM

I'm not sure that it replaces SPServices, because SPServices does a lot more. In fact there are benefits to using SPServices:

  • If you are using 2007 and planning to migrate to 2010, using SPServices and the Web Services is how you need to go, and if you're careful, everything ought to still work when you upgrade.
  • If you are starting in 2010, then there still many Web Services operations which are not available in the Client Object Model that SPServices makes it easy to call. (No, I don't have a list but I wish I knew of one.) If you are going to end up mixing Client OM and XML Web Services calls in your solutions, you may be better off going with the XML Web Services altogether so that you have better code reusability and build skills in one approach. (No idea about wave 15, BTW. I would really hope that the XML Web Services survive, but I have no info on that, either.) Governance and your decided upon architecture totally come into play on this.
  • SPServices provides a set of "value-added" functions that do some very commonly needed things (SPCascadeDropdowns, etc.).

M.

May 13, 2011 at 12:33 AM

Marc and I talked about having both Web Services and REST in the same library, and there didn't seem to be much benefit to it. For now I have started it as a separate project:
http://blog.pathtosharepoint.com/2011/05/02/two-new-projects-sprest-and-spell/

My personal view is that it is going in two directions:
- REST will become the tool of choice for CRUD operations on lists (Microsoft says performance is better)
- a bunch of specific tasks (in particular design and admin) will remain with the traditional Web Services

I too would be interested to know if Luke's plugin addresses the current needs. Also, what I've been suggesting - but haven't tried yet - is to use the jQuery converters to easily connect SPServices with the plugin. fyi the link to the original thread:
http://spservices.codeplex.com/discussions/255230

May 13, 2011 at 12:16 PM

Okay, I'm convinced.  It doesn't make sense to refactor SPServices just to support JSON when a simple javascript call can convert it easily.   I do like the idea of SPrest.  Something that abstracts away some of the REST calls and adds some value adds to them, such as the cascading dropdowns, etc.  Thanks for the feedback.

Coordinator
May 13, 2011 at 12:29 PM

No one ever answers the "which Web Service operations?" question! If it's just GetListItems, then it wouldn't be so difficult to add an abstraction just for that operation. For instance, I already have an abstraction on UpdateListItems (the valuepairs thing) which seems to help people quite a bit.

M.

May 13, 2011 at 12:30 PM

Sorry, was thinking of mostly just CRUD list operations.

Coordinator
May 13, 2011 at 12:33 PM

So that would be just GetListItems (_R__) and UpdateListItems (C_UD).

M.

May 13, 2011 at 12:44 PM

Yes, those two operations.  I've been using the sample that Chris O'Brien has up here.  The compression of Client OM at the bottom the post is interesting as well.

http://www.sharepointnutsandbolts.com/2010/10/sp2010-ajaxpart-2-using-javascript.htm

Coordinator
May 14, 2011 at 4:49 AM
Edited May 14, 2011 at 1:40 PM

Jordan:

I'm not sure what compression Chris is talking about in his post, so I've left him a comment to find out. Modern browsers all GZIP most traffic, but I'm not sure if Chris is talking about something else.

IMO, the bottom line is that efficiency in your code is going to be far more important than the compression. Frugal requests will win out over expectations of some miraculous compression.

While JSON will generally be smaller bytewise than XML, the thing to worry about is what data you ask for and what you do with it on the client.

M.

May 14, 2011 at 5:48 AM

People will tell you that JSON is generally smaller than xml. But I don't think this applies to SharePoint, because SharePoint uses attributes, not nested elements.

The real benefit from JSON in SharePoint is easy connection with existing plugins, not lighter data transport.

Coordinator
May 14, 2011 at 1:44 PM

Christophe:

What you say about nested elements is basically true about GetListLItems' XML response. Since we are requesting items which are essentially flat database tables (virtually, not literally), we get flat XML in response.

Again, I think about the other 196 operations (at the moment) I support in SPServices and there's a HUGE variation on this. (Clearly there were zero standards for the XML output when the Web Services were written over at Microsoft.) Some of the XML responses are highly nested, and in some cases, it even makes logical sense!

M.

May 16, 2011 at 1:14 AM

Well, this takes us back to square one: which Web services are we actually talking about?

Again, we are looking at two kinds of benefits: 1/ faster transport and 2/ reusable pieces of code (typically jQuery plugins).

For most of these 197 services, 1/ only xml is available and 2/ they are so specific that you can't leverage reusable pieces of code.

What I believe is that people who start JSON discussions are mostly talking about retrieving list items.

Didn't you talk about a poll?

Coordinator
May 16, 2011 at 1:19 AM

One correction: for *ALL* 197, only XML is available. These are all XML Web Services.

Yes, I do need ot get to the poll idea!

M.

May 16, 2011 at 1:38 AM

wiseacre (yes, I learnt that word from you ;-) ). I was really talking about the services they provide (see my comment). Some of these services can be delivered via the legacy xml Web services or the SP 2010 RESTful Web services.

Jul 22, 2011 at 12:42 PM

Sorry to bring up this post again, but I wanted to find luke's code as I am in need of converting to JSON for something!

Jul 22, 2011 at 12:49 PM

looks like it has a new location

https://github.com/bobby/jquery-sharepoint-json

Jul 22, 2011 at 1:10 PM

Thanks!

Dec 28, 2011 at 3:13 PM
Edited Dec 28, 2011 at 3:20 PM

Anybody have a quick example of how to use the jquery-sharepoint-json plugin with SPServices?  There's a couple examples on the github page but I can't wrap my head around it.

Coordinator
Dec 29, 2011 at 2:42 PM
Edited Dec 29, 2011 at 2:44 PM

Clint:

I haven't used Bobby's plugin myself, but it looks fairly straightforward. I think something like this would work:

 $().SPServices({
    operation: "GetListItems",
    async: false,
    listName: "Announcements",
    completefunc: function (xData, Status) {
      var json = $(xData).sharePointJSON({
mapping: {foo: "ows_Foo", barBaz: "ows_Bar_x0020_Baz"}
}); // do something with the json } });

M.

Dec 29, 2011 at 5:05 PM
Edited Dec 29, 2011 at 5:38 PM

Thanks Marc!  That was REALLY close and it got me where I needed.

This is what worked, note the PINK for anyone else that might stumble along.

$().SPServices({
    operation: "GetListItems",
    async: false,
    listName: "Announcements",
    completefunc: function (xData, Status) {
      $(xData.responseXML).SPFilterNode("z:row").each(function() {
        var json = $(xData.responseXML).sharePointJSON({
            mapping: {foo: "ows_Foo", barBaz: "ows_Bar_x0020_Baz"}
}); // do something with json } });
Coordinator
Dec 30, 2011 at 6:28 PM

Please see my blog post about including an XML to JSON conversion capability in SPServices and let me know what you think.

M.