|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Announcements
Chapters
Services
Feature Zones
|
IntroductionPeople are more productive when they can work anywhere anytime without getting nagging messages like "unable to connect" or "server not found." Just ask Microsoft Outlook users how much they love the offline mode. Application developers agree that enabling their applications to work offline is a customer value-add and an attractive proposition. However, the challenges with building and managing a local cache on top of developing reliable synchronization algorithms are a major concern and a potential risk to product release plans. Truth is: developers' concerns are hardly overstated! Microsoft Synchronization Services for ADO.NET is a new data synchronization framework that enables developers to build disconnected data applications using familiar concepts. In this document, I will give a quick overview of what the new synchronization framework is all about and how to use it to build occasionally connected applications. BackgroundI've been working in Microsoft for over five years focusing on synchronization and replication technologies. I have to admit that I've never been more excited about new synchronization technology than I am with the Synchronization Services framework. The reason is simple: this technology is for developers and not an end-to-end solution that is tailored toward administrators. Anyhow, I thought that it would be a good idea to write an introductory document on this new framework and post it here on The Code Project. After all, you guys are the future users of this API and your feedback is the most important of all. DisclaimerThis document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise) or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. © 2007 Microsoft Corporation. All rights reserved. Microsoft, MS-DOS, Windows, Windows Server, Windows Vista, SQL Server, Visual Studio are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners. Download CTP and DocumentationThe Synchronization Services for ADO.NET Beta 2 are available here. Please note that the package contains the SQL CE version 3.5. The documentation is available in a separate package; you can find it here. Synchronization Services Overview
ADO.NET introduced a set of disconnected objects that allows you to examine data offline after fetching it from the server. The key to this model is the SyncAgentThe sync agent is the synchronization maestro. Typically, it is located at the client side and runs the show in response to client application requests. The agent accepts two sync providers, one for the client database and the other for the server database. SyncTableDefines a table that the client wishes to synchronize. This includes the desired direction for the flow of changes, i.e. ClientSyncProviderThe client sync provider abstracts away the underlying data store at the client side. Sync Services ships with ServerSyncProviderThe server sync provider is where you will write your commands. Sync Services ships with
SyncAdapterThe
Although there are more commands on the Deployment ScenariosSynchronization services can be deployed in different set-ups: Two-tier DeploymentIn two-tier set-up, the N-tier DeploymentIn n-tier set-ups, as in the diagram above, the Sample Application OverviewLet us put the sync services to work. For this sample, I am developing a line of business applications (LOB) for sales personnel in the field. I want to enable them to enter orders while on the go, using their laptops. At the end of the day, each sales agent connects to the corporate network to synchronize his or her local changes and also receive orders made by other sales agents. That said, I need bidirectional synchronization. For simplicity's sake, I choose to focus on two tables only: CREATE DATABASE pub
GO
CREATE TABLE pub..orders(
order_id int NOT NULL primary key,
order_date datetime NULL)
GO
CREATE TABLE pub..order_details(
order_id int NOT NULL primary key,
order_details_id int NOT NULL,
product nvarchar(100) NULL,
quantity int NULL)
GO
Enable Change TrackingTake a quick look again at the Tracking Inserts and UpdatesTo track inserts and updates, I need the following columns:
-- Add create timestamp column (use bigint since only one timestamp
-- column type is allowed per table)
ALTER TABLE pub..orders add create_timestamp bigint default @@DBTS+1
ALTER TABLE pub..order_details add create_timestamp bigint default @@DBTS+1
GO
-- Add last update timestamp column
ALTER TABLE pub..orders add update_timestamp timestamp
ALTER TABLE pub..order_details add update_timestamp timestamp
GO
-- Create update_originator_id column defaulted to 0 to indicate server change
ALTER TABLE pub..orders add update_originator_id int null default 0
ALTER TABLE pub..order_details add update_originator_id int null default 0
GO
Setting and updating the values of the tracking columns is another matter. The The -- Create Update Triggers
-- Since there will be changes on the server outside of sync application
-- we need triggers to fix up update_originator_id back to 0 which
-- designated for server change
USE pub
GO
CREATE TRIGGER orders_update_trigger on orders FOR UPDATE
AS
-- Note: sync is the only app making changes to update_originator_id,
-- thus we only overwrite the value if it was not set by sync
UPDATE o
SET o.update_originator_id = 0
FROM [orders] o JOIN [inserted] i on o.order_id = i.order_id
WHERE NOT UPDATE(update_originator_id)
GO
CREATE TRIGGER order_details_update_trigger ON order_details FOR UPDATE
AS
-- Note: sync is the only app making changes to update_originator_id,
-- thus we only overwrite the value if it was not set by sync
UPDATE o
SET o.update_originator_id = 0
FROM [order_details] o JOIN [inserted] i on o.order_id = i.order_id
WHERE NOT UPDATE(update_originator_id)
GO
Tracking DeletesWith tracking inserts and updates taken care of, what about deletes? For deletes, I will need a ' -- Create tombstone tables to store deletes
CREATE TABLE pub..orders_tombstone(
order_id int NOT NULL primary key,
order_date datetime NULL,
update_originator_id int default 0,
update_timestamp timestamp,
create_timestamp bigint)
CREATE TABLE pub..order_details_tombstone(
order_id int NOT NULL primary key,
order_details_id int NOT NULL,
product nvarchar(100) NULL,
quantity int NULL,
update_originator_id int default 0,
update_timestamp timestamp,
create_timestamp bigint)
GO
While I only need to store the PK to identify the row that was deleted, I prefer to copy the entire row. My reasoning is that later on, when I implement conflict detection and resolution logic (not shown in this sample; see my blog), I would want to represent the deleted row to the end user to resolve the conflict. Showing only the PK is not a great help. Notice that the same tracking columns added to the base tables are created in the -- Create Delete Triggers
CREATE TRIGGER orders_delete_trigger ON orders FOR DELETE
AS
INSERT INTO into pub..orders_tombstone (order_id, order_date, _
create_timestamp, update_originator_id)
SELECT order_id, order_date, create_timestamp, 0 FROM DELETED
GO
CREATE TRIGGER order_details_delete_trigger ON order_details FOR DELETE
AS
INSERT INTO pub..order_details_tombstone (order_id, order_details_id,_
product, quantity, create_timestamp, update_originator_id)
SELECT order_id, order_details_id, product, quantity, create_timestamp,_
0 FROM DELETED
GO
With that, change tracking is complete and we are set for bidirectional synchronization. Code WalkthroughIn this example, I am building a two-tier offline application. This means that the client will have a direct connection to the server database to execute queries. The changes made to the back-end tables to track changes conclude the server-side work. Now we switch gears to the client side. InitializationIn two-tier set up, the // 1. Create an instance of the SyncAgent
SyncAgent syncAgent = new SyncAgent();
// 2. Prepare server db connection and attach it to the sync agent
DbServerSyncProvider serverSyncProvider = new DbServerSyncProvider();
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder();
builder["Data Source"] = textServerMachine.Text;
builder["integrated Security"] = true;
builder["Initial Catalog"] = "pub";
SqlConnection serverConnection = new SqlConnection(builder.ConnectionString);
serverSyncProvider.Connection = serverConnection;
SyncAgent.ServerSyncProvider = serverSyncProvider;
// 3. Prepare client db connection and attach it to the sync provider
string connString = "Data Source=" + dbPathTextBox.Text;
if (false == File.Exists(dbPathTextBox.Text))
{
SqlCeEngine clientEngine = new SqlCeEngine(connString);
clientEngine.CreateDatabase();
clientEngine.Dispose();
}
SqlCeClientSyncProvider clientSyncProvider =
new SqlCeClientSyncProvider(connString);
syncAgent.ClientSyncProvider = (ClientSyncProvider)clientSyncProvider;
Configuring Sync TablesTo configure a table for synchronization, a
To synchronize changes for both tables in one go, I need to use a common // 4. Create SyncTables and SyncGroups
SyncTable tableOrders = new SyncTable("orders");
tableOrders.CreationOption =
TableCreationOption.DropExistingOrCreateNewTable;
tableOrders.SyncDirection = SyncDirection.Bidirectional;
SyncTable tableOrderDetails = new SyncTable("order_details");
tableOrderDetails.CreationOption =
TableCreationOption.DropExistingOrCreateNewTable;
tableOrderDetails.SyncDirection = SyncDirection.Bidirectional;
SyncGroup orderGroup = new SyncGroup("AllChanges");
tableOrders.SyncGroup = orderGroup;
tableOrderDetails.SyncGroup = orderGroup;
syncAgent.SyncTables.Add(tableOrders);
syncAgent.SyncTables.Add(tableOrderDetails);
Configuring Sync AdaptersIn this step, I need to build a While the // 5. Create sync adapter for each sync table and attach it to the server
// provider
SqlSyncAdapterBuilder ordersBuilder = new SqlSyncAdapterBuilder();
ordersBuilder.Connection = serverConnection;
ordersBuilder.SyncDirection = SyncDirection.Bidirectional;
// base table
ordersBuilder.TableName = "orders";
ordersBuilder.DataColumns.Add("order_id");
ordersBuilder.DataColumns.Add("order_date");
// tombstone table
ordersBuilder.TombstoneTableName = "orders_tombstone";
ordersBuilder.TombstoneDataColumns.Add("order_id");
ordersBuilder.TombstoneDataColumns.Add("order_date");
// tracking\sync columns
ordersBuilder.CreationTrackingColumn = "create_timestamp";
ordersBuilder.UpdateTrackingColumn = "update_timestamp";
ordersBuilder.DeletionTrackingColumn = "update_timestamp";
ordersBuilder.UpdateOriginatorIdColumn = "update_originator_id";
// fix up anchor data type generated by the builder to use Binary
// instead of int64
SyncAdapter ordersSyncAdapter = ordersBuilder.ToSyncAdapter();
((SqlParameter)ordersSyncAdapter.SelectIncrementalInsertsCommand.Parameters[
"@sync_last_received_anchor"]).DbType = DbType.Binary;
((SqlParameter)ordersSyncAdapter.SelectIncrementalInsertsCommand.Parameters[
"@sync_new_received_anchor"]).DbType = DbType.Binary;
serverSyncProvider.SyncAdapters.Add(ordersSyncAdapter);
SqlSyncAdapterBuilder orderDetailsBuilder = new SqlSyncAdapterBuilder();
orderDetailsBuilder.SyncDirection = SyncDirection.Bidirectional;
orderDetailsBuilder.Connection = serverConnection;
// base table
orderDetailsBuilder.TableName = "order_details";
orderDetailsBuilder.DataColumns.Add("order_id");
orderDetailsBuilder.DataColumns.Add("order_details_id");
orderDetailsBuilder.DataColumns.Add("product");
orderDetailsBuilder.DataColumns.Add("quantity");
// tombstone table
orderDetailsBuilder.TombstoneTableName = "order_details_tombstone";
orderDetailsBuilder.TombstoneDataColumns.Add("order_id");
orderDetailsBuilder.TombstoneDataColumns.Add("order_details_id");
orderDetailsBuilder.TombstoneDataColumns.Add("product");
orderDetailsBuilder.TombstoneDataColumns.Add("quantity");
// tracking\sync columns
orderDetailsBuilder.CreationTrackingColumn = "create_timestamp";
orderDetailsBuilder.UpdateTrackingColumn = "update_timestamp";
orderDetailsBuilder.DeletionTrackingColumn = "update_timestamp";
orderDetailsBuilder.UpdateOriginatorIdColumn = "update_originator_id";
// fix up anchor data type generated by the builder to use Binary
// instead of int64
SyncAdapter orderDetailsSyncAdapter = orderDetailsBuilder.ToSyncAdapter();
((SqlParameter)
orderDetailsSyncAdapter.SelectIncrementalInsertsCommand.Parameters[
"@sync_last_received_anchor"]).DbType = DbType.Binary;
((SqlParameter)
orderDetailsSyncAdapter.SelectIncrementalInsertsCommand.Parameters[
"@sync_new_received_anchor"]).DbType = DbType.Binary;
serverSyncProvider.SyncAdapters.Add(orderDetailsSyncAdapter);
Configuring Anchor and ClientID CommandsAs I mentioned before, the
// 6. set up provider wide commands
// select new anchor command
SqlCommand anchorCmd = new SqlCommand();
anchorCmd.CommandType = CommandType.Text;
anchorCmd.CommandText =
"Select @" + SyncSession.SyncNewReceivedAnchor + " = @@DBTS";
anchorCmd.Parameters.Add("@" + SyncSession.SyncNewReceivedAnchor,
SqlDbType.Timestamp).Direction = ParameterDirection.Output;
serverSyncProvider.SelectNewAnchorCommand = anchorCmd;
// client ID command (give the client id of 1)
// in remote server scenario (middle tear), this command will reference a
// local client table for the ID
SqlCommand clientIdCmd = new SqlCommand();
clientIdCmd.CommandType = CommandType.Text;
clientIdCmd.CommandText = "SELECT @" + SyncSession.SyncOriginatorId + " = 1";
clientIdCmd.Parameters.Add("@" + SyncSession.SyncOriginatorId,
SqlDbType.Int).Direction = ParameterDirection.Output;
serverSyncProvider.SelectClientIdCommand = clientIdCmd;
The client ID of
You get the idea. Luckily for us, the framework provides a simple way to avoid doing that while in experimental phase. This is done by passing the hash value of the GUID, which is of type integer. Of course, there is a possibility of a hash collision but again, this is for testing purposes only. The code would look like the following: SqlCommand clientIdCmd = new SqlCommand();
clientIdCmd.CommandType = CommandType.Text;
clientIdCmd.CommandText =
"SELECT @" + SyncSession.SyncOriginatorId + " = @sync_client_id_hash";
clientIdCmd.Parameters.Add("@sync_client_id_hash", SqlType.Int);
clientIdCmd.Parameters.Add("@" + SyncSession.SyncOriginatorId,
SqlDbType.Int).Direction = ParameterDirection.Output;
serverSyncProvider.SelectClientIdCommand = clientIdCmd;
Synchronize!Now we are set to kick off synchronization. Before that, I'd like to subscribe to // 7. Kickoff Sync
clientSyncProvider.SyncProgress += new
EventHandler<SyncProgressEventArgs>(ShowProgress);
serverSyncProvider.SyncProgress += new
EventHandler<SyncProgressEventArgs>(ShowProgress);
SyncStatistics syncStats = syncAgent.Synchronize();
ConclusionThe interest in offline experience is growing rapidly, fueled by a new breed of rich internet applications. The vision of software as a service where the application has a web component and a desktop component stresses the need for synchronization as the glue that ties these components together. However, synchronization technologies have not become mainstream yet. Few developers have the know-how to build sync-enabled applications. With Microsoft Synchronization Services for ADO.NET, developers have a starting point to get into sync algorithms and learn how to extend their data applications with offline components. What's NextIn this document, I shared with you the first demo application of a series of demos that I prepared to showcase the new API. The rest of the demos are posted on my blog and website. Each demo builds on the previous one and shows a new feature of the API. Here is a quick overview of the demo series:
FeedbackI'd appreciate if people can post issues, requests, questions, etc. in the public forum. The forum is monitored by many people in the sync team and is thus a more effective way of communication. I also hope that I was able to introduce the new framework to you through this article. The only way I will know if you like it is by rating it. Please vote when you can. You can also contact me through my blog. History
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||