Node.js, meet SharePoint

In this post I would like to introduce you to Node-SharePoint, a SharePoint client for Node.js available on GitHub and NPM. This Node module allows you to access SharePoint 2010 lists and items from Node.js. It is based on ListData.svc, the OData based REST API for SharePoint 2010. 

Getting started

Download and install Node (if you haven’t already): http://nodejs.org/#download.

This will also install NPM (Node Package Manager). Use NPM to install the SharePoint client module:

C:> npm install sharepoint

A first example

Create a JavaScript file called main.js and use the code below as a starting point:

var SP = require('sharepoint'),
    site = 'http://yourdomain.sharepoint.com/teamsite',
    username = 'yourusername',
    password = 'yourpassword';

var client = new SP.RestService(site),
    contacts = client.list('Contacts');

var showResponse = function (err, data) {
    console.log(data);	
}

client.signin(username, password, function () {
	
    // At this point, authentication is complete,
    // so we can do requests.

    // Example request: Get list items
    // showResponse is used as callback function
    contacts.get(showResponse)

});

Let’s highlight a few key topics:

  • require(‘sharepoint’) returns an ‘namespace’ object, which contains a RestService class.
  • an object of the SP.RestService class represents a client for the ListData service within the designated site.
  • ListData operates on lists, so we use client.list(listName) to create a List object.
  • client.signin implements the claims based authentication process for SharePoint Online. You can find more details about remote authentication in my  previous post.
  • once the signin is completed, the callback function is called. In the callback function you can start sending authenticated requests to SharePoint Online.
  • Please note that the SharePoint client uses the following callback convention:
function callback(err, data) {
   // err: is used to pass an error (string), if any

   // data: contains the output of the previous response or processing step
}

Provide your own credentials and site in main.js and run the file with node:

C:> node main.js

You should see the items of Contacts list in your window:

SNAGHTMLb94bb01

Here are some further example how to work with list items.

// get items ordered by FirstName		
contacts.get({$orderby:'FirstName'}, showResponse);

....

// get fist 3 items and total count of items in list	
contacts.get({$top:3, $inlinecount:'allpages'}, showResponse);

....

// get item with Id 412 from the Contacts list;
contacts.get(412, showResponse);

.....

// add a new item to the list
contacts.add({LastName: 'Peel', FirstName: 'Emma'}, showResponse)

....

// update an item	
contacts.get(412, function (err, data) {

    var changes = {
        // include the changes that need to be made
        LastName: 'Tell',
        FirstName: 'William',

        // pass the metadata from the fetched item
        // this includes the etag
        __metadata: data.__metadata
    };

    contacts.update(412, changes, function () {
        // at this point, the change is completed
        // so let's check by getting the item again
	     contacts.get(412, showResponse)
    })        
})

.....

// delete an item
contacts.del(412);

Hopefully this helps you get going in connecting Node.js to your SharePoint site! Let me know if you have questions/suggestions.

Advertisements

Remote authentication in SharePoint Online

Suppose you want to programmatically access SharePoint Online from Node.js, Ruby, Java, [insert your favorite platform here],…. SharePoint offers data access APIs, but how do you authenticate? SharePoint Online uses claims based authentication. Documentation about programmatic authentication is fairly limited and restricted to .NET solutions.

Fortunately there is this great post from Wictor Wilén, providing a working .NET example that connects to SharePoint Online. It requires the SharePoint Client Object Model and Windows Identity Foundation SDK and Runtime (WIF) to run.

OK, but can we use a platform other than .NET? In the end it’s all just data and http under the hood…

Fiddler to the rescue! By running the sample application and using Fiddler to look at the http traffic under the hood, we can find out how the authentication sequence works.

It’s actually quite simple:

image

Step 1: Send SAML Request to STS

The application POSTs an SAML Request Security Token message to the Microsoft Online Security Token Service (STS), located at the following address:

https://login.microsoftonline.com/extSTS.srf

The Request Security Token message should contain username, password of a Microsoft Live account and the url of your SPO site. Here is a template that you can use to build the XML message.

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope"
      xmlns:a="http://www.w3.org/2005/08/addressing" 
      xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <s:Header>
    <a:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>
    <a:ReplyTo>
      <a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
    </a:ReplyTo>
    <a:To s:mustUnderstand="1">https://login.microsoftonline.com/extSTS.srf</a:To>
    <o:Security s:mustUnderstand="1" 
       xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <o:UsernameToken>
        <o:Username>[username]</o:Username>
        <o:Password>[password]</o:Password>
      </o:UsernameToken>
    </o:Security>
  </s:Header>
  <s:Body>
    <t:RequestSecurityToken xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">
      <wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
        <a:EndpointReference>
          <a:Address>[endpoint]</a:Address>
        </a:EndpointReference>
      </wsp:AppliesTo>
      <t:KeyType>http://schemas.xmlsoap.org/ws/2005/05/identity/NoProofKey</t:KeyType>
      <t:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>
      <t:TokenType>urn:oasis:names:tc:SAML:1.0:assertion</t:TokenType>
    </t:RequestSecurityToken>
  </s:Body>
</s:Envelope>

 

Step 2: Receive SAML Response

If the credentials are valid, the STS will respond with a Request Security Token Response message. Look for the BinarySecurityToken tag and you’ll find the Security Token:

.....
<wst:RequestedSecurityToken>
   <wsse:BinarySecurityToken Id="Compact0">t=EwBgAk6hB...
   </wsse:BinarySecurityToken>
</wst:RequestedSecurityToken>
.....

Extract the token value (including ‘t=’) for the next step.

Step 3: Send the Security Token to SharePoint Online

After you received the Security Token from STS, you need to POST the Security token to SPO:

http(s)://yourdomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0

E-accounts need to include a user-agent header. Your POST should look something like this:

POST http://yourdomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0 HTTP/1.1
Host: yourdomain.sharepoint.com
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0)
Content-Length: [calculate]

t=EwBgAk6hB....abbreviated

Step 4: Receive the authentication cookies

After SPO has validated the Security Token, it will return 2 authentication cookies in the HTTP header: FedAuth and rtFa.

.....
Set-Cookie: rtFa=0U1zw+TnLmLfDtzmppbu....abbreviated
Set-Cookie: FedAuth=77u/PD94bW.....abbreviated
.....

Extract the values of both cookies.

Step 5: Send requests including authentication cookies

For each subsequent request from your application to SPO, you must include the FedAuth and rtFa cookie in your request headers. Like this:

....
Cookie: FedAuth=77u/PD....LzwvU1A+; rtFA=0U1zw+TnL......AAAA==
....

This will satisfy SPO to process your request, whether it is a request for a page, a document, a web service or ListData service.

And that’s all there is to it!

I hope this is useful for others implementing SharePoint Online access. If you want more details, please check out my SharePoint client for Node.js which includes authentication code in JavaScript.

Thanks for reading.

Building SharePoint Web Apps using Sencha Touch

In this post we will explore how to build a SharePoint mobile web app using Sencha Touch, a great mobile JavaScript framework.

SharePoint 2010 is a very comprehensive platform, but mobile support is fairly limited. Although it is technically possible to customize the mobile user interface on the server side, this is not allowed in cloud based SharePoint (e.g. in Office365).

An alternative  approach is to build a web app where the UI is generated in the browser and connects to one of the SharePoint APIs to access data stored in SharePoint. SharePoint 2010 offers three data APIs which can be used in web apps:

  • SP Web Services (SOAP)
  • REST (ListData.svc)
  • CSOM (Client Side Object Model)

Although each of the APIs offer a distinct set of capabilities, the REST interface is simple and lightweight and I prefer this for mobile use. The SharePoint REST interface offers full CRUD access to items in SharePoint lists and libraries, which might be all that you need for your mobile web app.

Getting started

Because of the Same Origin Policy, your html file must be served from the same domain as the SharePoint site you want to access. You can place your html file containing your app on the server file system or in a SharePoint document library (e.g. site assets).

If you are using SharePoint Online (Office365), you will notice that when you select a .html file in a doc library, it is presented as a download, not opened in the browser. This is due to SharePoint settings which you are not allowed to change in SharePoint Online. As a workaround, simply use an .aspx file extension, instead of .html. This way, you can start your single page SharePoint application from a file in an asset library.

So, to get going, you need to create a app.aspx file to include all the css and js for your Sencha Touch app.

<!DOCTYPE html>
<html>
    <head>
    <title>SharePoint web app example</title>
       <link rel="stylesheet" href="sencha-touch.css" type="text/css" />
       <link rel="stylesheet" href="app.css" type="text/css" />
       <script src="sencha-touch.js"></script>
       <script src="app.js"></script>
    </head>
    <body>
    </body>
</html>

The app.js file contains the basic Sencha Touch start-up and shows an alert:

new Ext.Application({
  launch: function () {
    Ext.Msg.alert('Hello world', 'Ready for action!')
  }
});

You can put these files  in any SP doc library. Let’s assume you have put this file in the Site Assets library of a SP site called demo:

image

Now you can open up the following url on your iPhone or Android device:

http://<SPserver>/demo/siteassets/app.aspx

After logon, you will see the following result:

IMG_0724

Not quite what we expected… SharePoint has detected that we access the page with a mobile device and responds with the default mobile UI. You can suppress the default mobile UI by appending ?mobile=0 to the url.

So, let’s try: http://<SPserver>/demo/siteassets/app.aspx?mobile=0

IMG_0725

Yes! We now have a Sencha Touch web app running of a SharePoint server.

OData proxy

The next step is to connect Sencha Touch models and stores to SharePoint items and lists through the REST interface using the OData protocol. For this you will need an OData proxy. I developed an OData proxy as a Ext user extension. It is designed to access SharePoint data using the SharePoint ListData.svc REST service which is based on OData. You may use it for other OData sources.

The OData SharePoint proxy is on GitHub: https://github.com/lstak/SharePoint-proxy-for-Sencha-Touch

Ext.ux.ODataProxy features:

  • create, read, update and delete SharePoint items as Sencha Touch models
  • fetch multiple items from a SharePoint list in a Sencha Touch store
  • JSON payloads
  • partial updates: only changed attributes are sent to the server during an update
  • fixes issues in Sencha Touch data Model implementation (e.g. missing destroy() method)

Let’s look at some examples how you can use the SharePoint proxy. In these examples we will assume you have a subsite ‘/teamsite’ in which you have created a Contacts list based on the standard Contacts list template.

First we need to define the Model.

var Contact = Ext.regModel('Contact', {
  fields: [
      // please note CamelCase convention for SharePoint column names
      { name: 'Id', type: 'int' },
      'LastName',
      'FirstName'
    ],

  idProperty: 'Id',

  proxy: {
    // use the special odata proxy defined in odataproxy.js
    type: 'odata',

    // the proxy will connect to the List
    // named 'Contacts' in the /teamsite subsite
    url: '/teamsite/_vti_bin/ListData.svc/Contacts'
  }
});

We can now use the following CRUD operations on the Contact data model:

// Create an instance
var contact = new Contact({ LastName: 'Johnson', FirstName: 'Al' })
contact.save();
...

// Read an instance from the server by id
var id = 200;
Contact.load(id);
...

// Update an instance, loaded from the server
Contact.load(id, {
    success: function (contact) {
        contact.set("LastName", "Maxwell");
        contact.save();
    }
});
...

// Delete an instance
Contact.load(id, {
    success: function (contact) {
        contact.destroy()
    }
});
...

Using the Contact model you can now easily define a Store to fetch multiple items:

var store = new Ext.data.Store({
    model: 'Contact'
});

store.load()

Build your application

Using the odata proxy to configure your Models and Stores that connect to the SharePoint server, you can develop your app further just like any other Sencha Touch app. Check out the tutorials on the Sencha site for more info.

I will conclude with an example of a contact list through the normal SharePoint UI and through a Sencha Touch app:

image

IMG_0743[1]

Just as easily, you can expose the content of document libraries:

image

IMG_0744[1]

Live demo

A live demo is available at http://oxida.sharepoint.com/demo. You can open the mobile web app and also explore the content of the SharePoint site with a desktop browser.

image

Sencha Touch and SharePoint are a great combination and open up exciting new opportunities for companies using SharePoint. Give it a try!

Working with SharePoint data in Backbone.js

In this post we will explore how to work with SharePoint data in a browser based application using the popular Backbone.js library.

SharePoint and Backbone.js

SharePoint 2010 offers 3 APIs which can be used in JavaScript apps:

  • SP Web Services
  • SP Client Side Object Model (CSOM)
  • SP ListData service (ListData.svc)

While Web Services and CSOM provide an wider range of capabilities, ListData is a lightweight RESTful interface offering CRUD access to SharePoint lists and items. It is based on the OData specification.

Backbone.js provides models and collections to work with data on server. Out of the box, models and collections can connect with a RESTful JSON API following common REST conventions. However, to work with the SharePoint ListData interface, some customization is required.

Enter…

Backbone.SharePoint

Backbone.SharePoint is a Backbone.js plugin providing special Models and Collections which you can use to work with SharePoint Items and Lists. You can find Backbone.SharePoint on GitHub.

Backbone.SharePoint allows you to:

  • create, read, update and delete SharePoint items as Backbone models
  • fetch multiple SharePoint items from a list as a Backbone collection
  • support for OData query string options ($orderby, $filter, $select, etc.)
  • custom sync() to communicate with the ListData service
  • partial updates: only changed attributes are sent to the server during an update.

Getting started

Because of the Same Origin Policy, your html file must served from the same domain as the SharePoint site you want to access. You can place your html file containing your app on the server file system or in an asset library.

<!doctype html>
<html>
   ....
<script src="jquery.js"></script>
<!-- you can also use zepto.js -->

<script src="underscore.js"></script>
<script src="backbone.js"></script>
<script src="backbone-sharepoint.js"></script>
  ...

</html>

Examples

Now let’s look at some examples how you can use Backbone.SharePoint. Let’s assume you have a subsite ‘/teamsite’ in which you have created a Contacts list based on the standard contacts list.

// You define a Contact Model for items by extending Backbone.SP.Item
// Required parameters are the SharePoint site and the name of the list

var Contact = Backbone.SP.Item.extend({
    site: '/teamsite',
    list: 'Contacts'
})

// Create a new contact, the attributes refer to item column names.
// Please note capitals. We follow SharePoint columnnames
var contact = new Contact({ LastName: "Davis" });

// At this point we have a new contact model, but is not saved to the server,
// so let's save it to the server.
contact.save();

  ....

// Update the attributes of the Item:
contact.set({FirstName: "John"});
contact.save(); 

  ...

// Finally, to remove an item:
contact.destroy();

Working with SharePoint lists is similar to collections.

// you can define a SP List by refering to the model
var Contacts = Backbone.SP.List.extend({
    model: Contact
})

// create a list
var contacts = new Contacts;

// get contacts list from the server
contacts.fetch()

// the fetch options allow you to use query options
// for example, the request below will fetch
// only the LastName and FirstName columns
// for item 11..15 when ordered descending by LastName
contacts.fetch({
    select: 'LastName, FirstName',
    orderby: 'LastName desc',
    top: 5,
    skip:10
})

   ....

// This is how you can create a new contact,
// save it to the server and add it to the list (collection)
contacts.create({
    LastName: "Peel",
    FirstName: "Emma"
})

Hopefully this is sufficient to get you going!