BCS Part 1 – What can I build and what is the best way to do that?

If you already know SharePoint and probably experienced developing application in this platform you already know that when thinking about a new solution and all the requirements you’re going to meet in your project, picking the right tools and finding the best way to build the right solution is not always easy. When thinking about maintenance costs, upgradability and deeper integration with SharePoint, solutions that make better use of out the shelf tools win most of the time. Even with the introduction of Apps in SharePoint 2013, you can still benefit a lot from using Business Connectivity Services (aka BCS). Apart from the range of features and tools that are provided under the name of BCS (e.g. WebParts, integration with Secure Store Service, Office client applications) one of the main and sometimes overlooked areas is how BCS and Search can work together. If you enable search over the entities of your external system, you can surface data coming from it through search and you can even customise search pages to create totally different user experience (e.g. I remember a shopping cart solution that might be a subject for a future blog post).

Instead of going through all the possible solutions, in this post I’m going to provide you with some links I have gathered from around the web that I think are essential if one needs to make real word solutions with these technologies. In the future posts I will go into the details and on cover the missing documents that you will need to make solid solutions.

First of all the basics (please note that REST, ODATA and some other features are not available out-of-the-box in SP2010):

And then read carefully the followings to get to know Search interoperability with BCS:

Don’t worry if there are terms that you don’t understand completely, I provide some examples and behind the scene information so you can understand them well.

Everything you need to know about BCS – Intro

Business connectivity services is an integral part of Microsoft SharePoint Server when it comes to interacting with external systems (i.e. Line of Business Systems). There lots of tools and foundation from QCRUD operations to out-of-the-box web parts to let you surface the data and let users interact with it. For when you need to go further there is a capabale object model that can be utilized to make custom solutions most probably used to develop custom web parts. There is a visual designer in Visual Studio 2010+ that makes it easier to design and validate the interface between SharePoint and your external system. In SharePoint 2013, Microsoft has taken it to the next level by supporting OData, event listeners, apps, REST interface and a better external list (still could be better 😉 ).

With all these said, what makes working wih BCS complex is the shortage of learning resources and the way its object model and BDC files (interface definition) are design (specially the latter). You cannot even trust the documentations on MSDN sometimes (I will come back to this later). Although there are blog posts here and there to help you move forward, sometimes you are on your own to discover hidden gems and pitfalls.

In these series of posts my aim is to share my experience in building real world BCS solutions so you can save valuable development time by focusing on more productive tasks. I will try to show you the safe paths and pitfalls and be as complete as possible. There is not much difference between SharePoint 2010 and 2013, but when things are a little different I will show you how. At the end, comments are always welcome!

Comming up:

  • Part 1 – What can I build and what is the best way to do that?
  • Part 2 – Interacting with databases (SQL Server and Oracle as examples)
  • Part 3 – Interacting with other sources / .NET Connectivity Assemblies
  • Part 4 – Interacting with dynamic sources / Custom Connectors
  • Part 5 – Searchable BCS solutions
  • Part 6 – Enhancing BCS Performance
  • Part 7 – Surfacing BCS with search
  • Part 8 – Surfacing BCS with out-of-the-box web parts
  • Part 9 – Surfacing BCS with custom web parts
  • Part 10 – Surfacing BCS with Java Script and JSON (MVVM) in SharePoint 2010
  • Part 11 – Adding CSOM support to BCS in SharePoint 2010
  • Part 12 – Business Data Catalog file, the complete reference
  • Part 13 – Bullet-proof deployment of BCS solutions in Farm
  • Part 14 – BCS and Claims-Based-Authentication considerations

Do you have any suggestions for another part? then let me know in the comments.

What’s new in Business Connectivity Services in SharePoint 2013

Business Connectivity Service (previously Business Data Catalogue / BDC) was one of the most exciting features in Microsoft SharePoint Server since its first release. Although it had a lot of short comings in its first version, but the idea behind it made it an important tool in the package. In SharePoint Server 2010, we have the possibility of using external data almost like a native list and it is possible to work with BCS in client object model which opens new doors for its extensibilities. Microsoft has made some changes in the next version (2013) to bring it to the next level. The new interoperability between BCS and other services (e.g. Visio Services) makes the ground ready for new solutions that were not possible / easy before.

Here are the most important changes (In my opinion):

  • SharePoint now supports OData as a data source for BCS
  • It can receive events from external systems
  • External content types can be application scoped
  • REST support is enhanced
  • Richer object model for BCS
  • It’s now possible to use external data in Visio / Visio Services and data will be refreshed when it’s changed!

OData (Open Data Protocol) Support

OData is one of the most popular protocols used for exposing data over http. It is actualy based on HTTP, Atom, and JavaScript Object Notation (JSON) to let developers write applications that communicate with data sources (called Producer in OData). You can also find different live data sources that expose data over the internet. If you are new in this field and want to know what applications support OData I suggest to take a look at this page. You will find SharePoint, Dynamics CRM 2011, TFS and Tridio in the list of applications and Facebook, Netflix, Pluralsight, eBay and many others in the list of live services.

External System Events

The external event system works as a push model meaning that the external system (LOB application) sends a notification to let SharePoint know that an event is occurred, SharePoint then keeps record of the event in a notification list and notifies to it’s subscribers. The external system notifies SharePoint by sending a message to a REST endpoint in SharePoint that is configured as delivery address. [Read the following article in MSDN for more information: External events and alerts in SharePoint 2013]

Accessing external data in Visio

External list can now be used in visio diagrams. The procedure is really easy and from the Visio point of view you don’t feel any difference at all. After you create an external content type with BCS, and an external list. You just need to open up Visio 2013, go to “Data” tab and click “Link data to Shapes” (as you normally do with any data source) and this time select “Microsoft SharePoint Foundation”, Select the web site and the list that you’ve created previously. [More about new features in Visio 2013 on MSDN]