My post “Using SharePoint CSOM in HTML5 apps” describes the minimum set of js files you need to use the SharePoint JavaScript Client Side Object Model (CSOM) in the browser.
I was curious if you can also use SharePoint CSOM in Node.js. I found out, you can! In this post I’ll show you how.
Warning: the SharePoint CSOM files are designed to run in the browser, so consider everything that follows as highly experimental!
The CSOM files
To use CSOM in the browser, you need the following 5 files from SharePoint:
- /_layouts/1033/init.js
- /_layouts/MicrosoftAjax.js
- /_layouts/sp.core.js
- /_layouts/sp.runtime.js
- /_layouts/sp.js
If you want to learn more about the code in these files, you can use the DEBUG versions which you can find here:
- /_layouts/1033/init.DEBUG.js
- /_layouts/sp.core.DEBUG.js
- /_layouts/sp.runtime.DEBUG.js
- /_layouts/sp.DEBUG.js
A debug version of MicrosoftAjax.js is included in VisualStudio 2010.
Concatenated, the debug versions of these 5 files contain approximately 33.000 lines of JavaScript code, which I will refer to as the ‘CSOM code’ in the rest of this article. Our goal is to use the CSOM code on Node instead of the browser, without modifying the source files.
I started out by just concatenating the 5 source file in one big file and trying to run it with Node. Node will terminate when an error is thrown and point to the error. You will quickly realize that all the errors raised are related to the difference in execution context between a browser and Node. Node doesn’t have objects like document, window, XmlHttpRequest and navigator, which the CSOM code expects.
To summarize my findings, the CSOM code expects and requires the following:
- a window object as global context
- a DOM element named “__REQUESTDIGEST” which contains a digest value that will be included in HTTP request header X-RequestDigest.
- a navigator object, identifying the user agent
- a document object, with a URL property, used to identify the domain of the SP server
- XMLHttpRequest to make server requests
- authentication cookies
The trick is to emulate these objects, just enough to make the CSOM code run as intended and without errors. I ended up with the following code which mocks the browser environment expected by the CSOM code.
var window = global; var navigator = { userAgent: "Node" }; var formdigest = { value: 'DUMMY VALUE', tagName: 'INPUT', type: 'hidden' }; var document = { documentElement: {}, URL: 'http://yourdomain.sharepoint.com/', getElementsByName: function (name) { //console.log('getElementByName: ', name); if (name == '__REQUESTDIGEST') { return [formdigest] } }, getElementsByTagName: function (name) { //console.log('getElementByTagName: ', name); return [] } };
Let’s have a closer look on the digest. Remember that our minimal aspx page contained a form digest control? This control will include a hidden form input element named “__REQUESTDIGEST”. The CSOM code will retrieve the value from the DOM, using document.getElementsByName(“__REQUESTDIGEST”)[0].value. As you can see, the code above emulates the document object and input element just enough to make this code work in the Node environment. We only need to set a valid digest value in the formdigest object. We’ll come to that later.
Remote authentication
In order to access the SharePoint server our Node SharePoint client needs to authenticate. Please see my post on remote authentication to SharePoint Online for more details. We will reuse the authentication code described in Node.js, meet SharePoint.
Emulating XMLHttpRequest
The CSOM code makes server requests using XMLHttpRequest. Luckily, there is already an implementation of XMLHttpRequest in Node and it works great. There’s only one workaround required in order to include the authentication cookies in the request. We will assign the authentication cookies to the XMLHttpRequest constructor and include them as part of the default headers. I made the following modification to the source of XMLHttpRequest:
var defaultHeaders = { "User-Agent": "node.js", "Accept": "*/*", "Cookie": this.constructor.authcookies };
Now we can set the authentication cookies by using:
// make sure the XMLHtpRequest includes the authentication cookies XMLHttpRequest.authcookies = 'FedAuth=' + client.FedAuth + '; rtFa=' + client.rtFa;
Obtaining a digest value
After authentication, but before the CSOM is used we need to obtain a valid digest value. The CSOM code expects the value to be available in a hidden input element which we emulated through the formdigest object. To get a digest value, we could fetch an aspx page with an embedded formdigest control and parse out the value. Another option is to use the sites.asmx SharePoint Web Service to request a digest value.
We will use the web service in the following code :
function requestDigest(params, callback) { var payload = '<?xml version="1.0" encoding="utf-8"?>' + '<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"' + ' xmlns:xsd="http://www.w3.org/2001/XMLSchema"' + ' xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">' + ' <soap:Body>' + ' <GetUpdatedFormDigest xmlns="http://schemas.microsoft.com/sharepoint/soap/" />' + ' </soap:Body>' + '</soap:Envelope>'; var options = { method: 'POST', host: 'yourdomain.sharepoint.com', path: '/_vti_bin/sites.asmx', headers: { 'Content-type': 'text/xml', 'Cookie': 'FedAuth=' + params.FedAuth + '; rtFa=' + params.rtFa, 'Content-length': payload.length } } var req = http.request(options, function (res) { var responseText = ''; res.setEncoding('utf8'); res.on('data', function (chunk) { responseText += chunk; }) res.on('end', function () { //console.log('XML: ', responseText); if ((responseText == null) || (responseText.length <= 0)) { return; } var startTag = '<GetUpdatedFormDigestResult>'; var endTag = '</GetUpdatedFormDigestResult>'; var startTagIndex = responseText.indexOf(startTag); var endTagIndex = responseText.indexOf(endTag, startTagIndex + startTag.length); var newFormDigest = null; if ((startTagIndex >= 0) && (endTagIndex > startTagIndex)) { var newFormDigest = responseText.substring(startTagIndex + startTag.length, endTagIndex); } if ((newFormDigest == null) || (newFormDigest.length <= 0)) { return; } formdigest.value = newFormDigest; callback(); }) }) req.end(payload); }
To summarize, the client app on Node must:
- Request a security token from Security Token Service
- Submit the security token to SharePoint Online and receive authentication cookies
- Request a digest value and insert the value into formdigest object
Once these steps are successfully completed, you can start using CSOM code, just like you would in the browser.
Running example
Here’s the final part:
// constructor for SharePoint Online client SPO = function (url) { this.url = urlparse(url); this.host = this.url.host; this.path = this.url.path; // External Security Token Service for SPO this.sts = { host: 'login.microsoftonline.com', path: '/extSTS.srf' }; // Form to submit SAML token this.login = '/_forms/default.aspx?wa=wsignin1.0'; }; SPO.prototype = { signin: signin }; var client = new SPO('http://yourdomain.sharepoint.com/yoursite'); client.signin("yourusername", "yourpassword", function () { // make sure the XMLHtpRequest includes the authentication cookies XMLHttpRequest.authcookies = 'FedAuth=' + client.FedAuth + '; rtFa=' + client.rtFa; // as of this point, you can use the CSOM service as if you are in a browser...... // for example, we will request the properties of the site 'teamsite'. var ctx = new SP.ClientContext("/teamsite"), web = ctx.get_web(); ctx.load(web); ctx.executeQueryAsync(function () { var properties = web.get_objectData().get_properties(); console.log(properties); }); });
SPO represents a CSOM client (for SharePoint Online). The signin method takes care of authentication and fetches the digest value. Once signin is completed, the authentication cookies are set on XMLHttpRequest. At that point we create a SP.ClientContext and set up a query to get web properties. CSOM will fire a query to the SharePoint server and we write the properties to the console.
Full code for this example is available here: https://gist.github.com/2404924. You will need to insert the CSOM code yourself.
And there you have it: the SharePoint Client Side Object Model running on Node! If you want to try this yourself, I recommend using node-inspector for stepping through your code and get a better understanding of the inner workings of CSOM.
Possible scenarios
CSOM is very useful when your requirements go beyond manipulating List data. CSOM on Node may be used in scenarios like:
- data transfer/synchronization between SharePoint with other data sources including automated creation of list/columns
- long running processes like monitoring or content analysis
- automated permission maintenance
- automated creation of sites
- upgrade of list and site definitions
- need for CAML queries on SharePoint data
- support for additional protocols (e.g. Web sockets)
Let me know if you have suggestions/comments.
Thanks for reading!
Great article! Will it be included in your node-sharepoint module? Because I think REST protocol in SharePoint has some shortcomings especially in multi language environments (in my blog post I wrote about the weird need of translation of properties you pass in to listdata.svc). So CSOM would make it much easier.
Hi Anatoly,
I am not sure if it can be included. It’s highly experimental and the use of the CSOM libs outside browser is unsupported by Microsoft (AFAIK). On the other hand it would be nice to have a single module which offers access to the three SP APIs, having common authentication support.
I agree with you on the issue of field names (using display names depending on language chosen by the user instead of internal names) in the REST interface. I don’t understand why they made that design choice.
I understand that is experimental. It was great that you got it working outside a browser. I wish there were an javascript Client Object Model for SharePoint which would not so dependent from browser.
Pingback: nodeunit and SharePoint: unit tests in javascript « Share… What?