Introduction
When designing my entry into the App Innovation contest (if you are interested, it is called Rev Warrior and can
be found here) one of the features
I wanted to have was the ability to store the data that the program logged into a user's Google Drive. The purpose was to make the data easily accessible
for later analysis in Excel or some other program.
What I discovered was a lack of any documentation or working examples for doing this in .NET beyond a very simple sample provided
by Google. Although their sample worked, it required explicit authorization each time it was run and it didn't really address how
or why the sample worked. So I started digging into the problem and when I found a solution, I felt that I should share it here.
The attached source code is a very simple WPF application that, when run, will open a window with a single button. Click the button
and an open-file dialog will prompt you to select a file to upload. Select a file and the application will attempt to authenticate with
Google Drive. If needed, it will open a browser so you can authorize the access and, provided you OK it, the program will upload the file you picked into your Google Drive.
Where my application differs from the other samples I found is that my application extends the Google sample code to store the authorization tokens and use them in subsequent
accesses so the user is not prompted to authorize the application every time the program is run. The result is a pretty clean flow which will prompt the user the first time but
then silently upload the chosen file from then on provided the application remains authorized. (Google users can revoke authorizations to their Google Services in their user control panel.)
The actual upload is very easy as Google provides all of the libraries for you to use. The complication was in the Authorization sequence and how to cache those
tokens and use them on subsequent requests. Google's documentation reads like stereo instructions and there are not many examples out there that can be used for guidance.
Google provides a full application, called Dr. Edit, as a .NET sample but it is an ASP MVC application and not real helpful for those of us working in WPF, WinForms,
or something else. To get to a working WPF solution, I had to reverse engineer several Google sample projects. The upside is that the authorization steps
are pretty generic and can be applied to just about all of Google's API services which includes Tasks, Calendar, Maps, etc so you can use this project
as a basis for accessing many of Google's API Services.
Background
Many Google Services require two types of authentication; the user must be authenticated (logged in) and the application must be explicitly authorized by the user.
(For Google Drive Authorization information see the Google Drive Authorization API Docs.)
The authorization mechanisms are based on OAuth 2.0. Generally speaking, Google's OAuth mechanism requires the application requesting access to present
a client ID (which identifies the application to Google) and a client secret to an authorization service endpoint along with the scope(s) the application wishes to access.
(Note: The authorization scopes, included in the SCOPES array below, are different for each service. Refer to the Authorization section of the Google API
documentation for the service(s) you wish to use.) The authorization service will check to see if the application is authorized to use the requested scopes and if so,
it will return an access token to be passed with all of the service calls. All of this is done in JSON and is well defined in both Google's API
documentation and other OAuth2.0 documents. The trick is to get the access token because that part of the process is not very well documented... at least not for .NET developers.
For installed applications, the two required keys are kept in the code (see the
ClientCredentials class below) and not really considered to be a secret because the code could be easily
decompiled or the JSON intercepted with a proxy tool like Fiddler. For this reason, when the application keys are generated, Google must be told it is for
an 'installed application' so that it treats communications with a higher level of suspicion.
internal static class ClientCredentials
{
public static readonly string CLIENT_ID = "<< APPLICATION CLIENT ID GOES HERE >>";
public static readonly string CLIENT_SECRET = "<< APPLICATION SECRET GOES HERE >>";
public static readonly string[] SCOPES = new String[]
{
"https://www.googleapis.com/auth/drive.file",
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/userinfo.profile"
};
public static readonly string REDIRECT_URI = "urn:ietf:wg:oauth:2.0:oob";
}
When an application which is registered with Google and attempts access for the first time (or otherwise does not have cached credentials),
Google will check to see if the application identifier is authorized to access the user's requested service(s). If the application has not been authorized,
Google will fail the call usually with a 401 Not Authorized exception. It is really up to the client to know if it has been authorized previously or not.
Applications must be granted access to the specific services via the API console. More on that later...
The key to all of this is the Authorization Code.
If the application has the code, it can assume that it has been previously authorized. The authorization code is like the number ticket you get when you are waiting
in line at the Motor Vehicle Office.... it doesn't get you served, it only gets you the right to be served. The authorization code indicates that the user has authorized
the application for service access and it is returned to the client via the
REDIRECT_URI specified in the ClientCredentials class as seen above. That cryptic string seen
there tells the authorization server to return the code in the title of the page sent back after the user authorizes the application.
When an application wants to access Google services, it should check to see if it has cached credentials (assuming you want to skip the authorization
step after the first time). If it does not, it should pop-up a browser window requesting authorization for the Google services. The
NativeClientApplication object
knows how to generate the authorization URL so we don't need to figure it out ourselves.
The sample project has everything needed to do this for you. There is a Forms folder in the project which includes the required forms.
They rely on the built-in browser so they should work just fine without any modification.
The page you see above in the browser control was generated by the Google authorization service. The permissions your app requires will be listed on the page
and can be expanded by the user to get a better idea of what each permission entails. It goes without saying that your application should only request the minimum number
of permissions as required to accomplish its task. And users should review these permissions before authorizing an application... be it your app or anyone elses.
Provided the user allows access, the authorization server will return the authorization code. It can be sent back either by the Google server calling
a web service endpoint opened by your application or in the title of the page sent back to the browser. Because of the challenges of spooling up a web server,
opening a connection through any possible firewall(s), etc... it is much easier for installed applications to simply scrape the authorization code from the title
of the resulting web page. That is the technique used in the sample project. If successful, the title will be set to Success code=xxxxxxxxx
where the xxxx's are replaced by a unique authorization code.
As previously mentioned, the authorization code only gets you invited to the party.
You can't do anything with that code as far as API access. The Authorization Code must be exchanged for a short-lived access code and a long-term refresh code.
In the Google.Apis.Authentication.OAuth2 library is a class called
NativeApplicationClient. This is the wrapper for the authorization
server and it has a method called 'ProcessUserAuthorization'. This method takes the authorization code we retrieved after the user authorized the application's access
and turns it into the access token and the refresh token. The access token is what we actually need for the task at hand and it is maintained in the
NativeApplicationClient.
It gets passed with all subsequent API calls. The nice thing about the NativeApplicationClient is that it knows how to verify the access token and how old the token is.
If the token has expired, the client will use the refresh token to get a new access token. That takes the burden off of us to manage token lifetimes.
Cached Credentials
The workflow described previously addresses the situation where the application needs to get authorization to access the API Service(s). It is possible to always prompt
for authorization and some of the guidance around these types of operations suggests that, as a good developer, we should do that. But the issue of user experience comes into play
and we don't want to annoy them with the authorization dialog every time our program runs and we need to access their Google Drive. It is possible that the user may not keep a cached
login in their browser and would not only be prompted to authorize the app, but could also be prompted for their login first thereby making them go through two screens
to authorize the access each time. This would be a real turn-off for the average user so we need a better way. Once authorization is agreed to, we can safely assume
the user knows that they have authorized our application to access their Google Drive so we can reasonably skip the authorization step. In practice, we do this with Cached Credentials.
In the sample project, there is a helper class called AuthorizationMgr that handles (among other things) the authorization caching. When we get our initial
Authorization Code and turn it into an access token and a refresh token, we call the
SetCachedRefreshToken method on the AuthorizationMgr class.
public static void SetCachedRefreshToken(string storageName,
string key,
IAuthorizationState state)
{
string scopes = state.Scope.Aggregate("", (left, append) => left + " " + append);
string content = scopes + "\r\n" + state.RefreshToken;
byte[] salt = Encoding.Unicode.GetBytes(Assembly.GetEntryAssembly().FullName + key);
byte[] encrypted = ProtectedData.Protect(
Encoding.Unicode.GetBytes(content), salt, DataProtectionScope.CurrentUser);
string file = storageName + ".auth";
AppData.WriteFile(file, encrypted);
}
This method will take a storage name and an encryption secret as well as the credentials we received back and store them in an ecrypted file in the application data folder.
In my sample code, the name of the file and the secret are embedded in the source code... which we all know isn't safe. You should figure out a better way to do this...
at the minimum the secret should not be in the source code.
This authorization information is sensitive because it contains the refresh token. This is what allows us to get a current access token without requiring
the explicit authorization of the user. When the initial access is approved, the refresh token is generated and represents that access request and that explicit
authorization. It can be used as a proxy for the user's authorization in the future. There is another method on the
AuthorizationMgr class
called GetCachedRefreshToken which will retrieve that token from the encrypted file. When the program attempts to get the authorization for an action
on the Google Drive, the code attempts to load the cached refresh token using this method. If successful, the
NativeApplicationClient class has a method called RefreshToken which takes the cached authorization information and requests a fresh access token from Google. If successful, the program
has been silently authorized and API access will be granted.
Using the code
The hard part is over. Getting the application authorized and getting a valid access token were the most difficult part.
The included WPF project also shows how to get all the files from the user's drive to check and see if a file exists. The API
calls to upload or update the file are also in the project.
Getting Started with Google API Access
The first thing you have to do to access any Google API is setup a project in the API console and generate the OAuth credentials. If you have a Google account you can login
at the Google API Console. Once logged in, you must create a new project. Once you have done that, you will see that
your project has been assigned an API key. This is the master key for your project. You may have up to seven applications granted access via your project. (And these may
be different application types; installed, web, mobile, etc.)
Before your application can access Google Drive or most any other service of interest, you must create an OAuth2.0 Client ID. This can be found under 'API Access'
on the left menu. If it isn't obvious, you click on the big blue button in the API Console. This will launch the process of generating an OAuth2 client credential.
The process is different for different application types. Things may be a bit different if you aren't doing an installed application like this one.
The process starts by giving a name to the application.
The Product Name is what will show up in the authorization page when the authorization request is displayed to the user. You can optionally specify
a URL where a product logo resides which will be displayed on the page as well. In the image earlier in the article, you can see the page generated
for my sample application to see what the final looks like.
When you click next, you need to give the Google servers information about your app.
In our case, we are going to specify it is an installed application and that we will be running it on 'other'.
Once we create the Client ID, we will be returned to the main console and we should see our new OAuth credentials:
In the screen shot you can see where the client ID and client secret are found. These values go into the
ClientCredentials class in the sample code.
These two values identify your application as the app you have just setup in your API console. You should do your best to keep these values secret knowing
that it isn't ideal they are embedded in the code.
Now that your application is registered with Google and you have the Client ID and Client Secret in the code, you have to tell Google which services
your application is going to access. If you don't do this step, you will get 401 Unauthorized errors when you try to access specific APIs. Granting access
is pretty easy and is done under the 'Services' option on the left side of the console. The service access you turn on and off here is project
wide (not application specific) and determines which the requested permissions list look like. For access to Google Drive, I have enabled the Drive Service and the Drive SDK.
There are certain restrictions and other terms related to API access. You should refer to Google's documents to make sure you understand those limitations.
Once you have completed these steps, your application should be able to access Google Drive so let's turn our attention to the code.
Using the sample code
Although this is a WPF app, the code would be the same for a WinForms app or something similar. The sample project simply creates a window
with a single button labeled 'GO'. The event handler in the main page's code-behind simply calls the
AuthorizeAndUpload method
in the static MainClass.
public static void AuthorizeAndUpload()
{
_service = new DriveService(CreateAuthenticator());
OpenFileDialog dialog = new OpenFileDialog();
dialog.AddExtension = true;
dialog.DefaultExt = ".txt";
dialog.Filter = "Text files (*.txt)|*.txt|All files (*.*)|*.*";
dialog.Multiselect = false;
dialog.ShowDialog();
File body = new File();
body.Title = System.IO.Path.GetFileName(dialog.FileName);
body.Description = "A test document";
body.MimeType = "text/plain";
System.IO.Stream fileStream = dialog.OpenFile();
byte[] byteArray = new byte[fileStream.Length];
fileStream.Read(byteArray, 0, (int)fileStream.Length);
System.IO.MemoryStream stream = new System.IO.MemoryStream(byteArray);
List<File> fileList = Utilities.RetrieveAllFiles(_service);
foreach (File item in fileList)
{
if (item.Title == body.Title)
{
MessageBoxResult result = System.Windows.MessageBox.Show("The file you picked already exists " +
"in your Google Drive. Do you wish to overwrite it?",
"Confirmation", MessageBoxButton.YesNoCancel);
if (result == MessageBoxResult.Yes)
{
Utilities.UpdateFile(_service, item.Id, item.Title,
item.Description, item.MimeType, dialog.FileName, true);
}
else if (result == MessageBoxResult.No)
{
Utilities.InsertFile(_service, System.IO.Path.GetFileName(dialog.FileName),
"An uploaded document", "", "text/plain", dialog.FileName);
}
else
{
return;
}
break;
}
}
System.Windows.MessageBox.Show("Upload Complete");
}We start out by creating a new DriveService object. This is a class provided by the Google libraries and is meant to encapsulate the Drive API.
If you were working with a different service, like Tasks for example, this would be a different object, like
TaskService. The parameter to the class constructor is an object
that implements the IAuthenticator interface. In this case, we are using the results of the
CreateAuthenticator function
to pass in an Authenticator object. CreateAuthenticator looks like this:
private static IAuthenticator CreateAuthenticator()
{
var provider = new NativeApplicationClient(GoogleAuthenticationServer.Description);
provider.ClientIdentifier = ClientCredentials.CLIENT_ID;
provider.ClientSecret = ClientCredentials.CLIENT_SECRET;
return new OAuth2Authenticator<NativeApplicationClient>(provider, GetAuthorization);
}
You can see that CreateAuthenticator first creates a NativeApplicationClient. As mentioned before, this wraps a lot of the authentication mechanisms
up for us. You can see that the next two lines set the ClientID and Secret from our static
ClientCredentials class. The last thing we do is use
the NativeApplicationClient to create a typed OAuth2Authenticator (which implements the
IAuthenticator interface) which we return back.
What you may initially overlook is the reference to GetAuthorization in the
OAuth2Authenticator constructor. That parameter requires an object which implements
the IAuthorizationState interface. That interface contains the current authorization information and is used by the
NativeApplicationClient to provide the access
token on subsequent API calls.
private static IAuthorizationState GetAuthorization(NativeApplicationClient client)
{
const string STORAGE = "gdrive_uploader";
const string KEY = "z},drdzf11x9;87";
string scope = DriveService.Scopes.Drive.GetStringValue();
IAuthorizationState state = AuthorizationMgr.GetCachedRefreshToken(STORAGE, KEY);
if (state != null)
{
try
{
client.RefreshToken(state);
return state; }
catch (DotNetOpenAuth.Messaging.ProtocolException ex)
{
Debug.WriteLine("Using existing refresh token failed: " + ex.Message);
}
}
state = AuthorizationMgr.RequestNativeAuthorization(client, scope);
AuthorizationMgr.SetCachedRefreshToken(STORAGE, KEY, state);
return state;
}
GetAuthorization will first attempt to pull cached credentials by using the
AuthorizationMgr helper class. If no cached credentials are available, the method
will use the AuthorizationMgr.RequestNativeAuthorization function to get a user authorization code then exchange it for a valid access and refresh token which are
saved via the SetCachedRefreshToken method on the AuthorizationMgr helper class. This is the part in the process where the web browser would be opened
and authorization requested if we don't have cached credentials.
While the authorization mechanism is setup and defined at the beginning, it isn't
actually executed until our first API access which happens later. Back in the
AuthorizeAndUpload method we have created our Drive Service and passed
in all of these objects and methods to handle the authentication. Next, we prompt the user to pick a file to upload. This is done with a standard file-open
dialog box. Once we have the file, we have to turn it into a Google File object. The next few lines setup that file and copy the contents via a memory stream.
File body = new File();
body.Title = System.IO.Path.GetFileName(dialog.FileName);
body.Description = "A test document";
body.MimeType = "text/plain";
System.IO.Stream fileStream = dialog.OpenFile();
byte[] byteArray = new byte[fileStream.Length];
fileStream.Read(byteArray, 0, (int)fileStream.Length);
System.IO.MemoryStream stream = new System.IO.MemoryStream(byteArray);
The next chunk of code calls an API to retrieve all of the files in the user's Google Drive. In my sample project, I'm calling the
synchronous version of this API
and doing it on the UI thread... both bad programming practice but this is an example... not production. Obviously, you would want to call this in an async manner
and properly marshal the results over to the UI.
List<File> fileList = Utilities.RetrieveAllFiles(_service);
This call will return a list of Google File objects and it will be all files in the user's drive. Including files in subfolders. Each file has
a parent ID property which can be used to recreate the hierarchy of the drive. My sample is simply looking to see if the file being uploaded already exists
in the drive. There is actually a flaw in this logic because the sample only uploads into the root of the drive. If the target file existed in a subfolder,
it would get reported to the user as already existing and asking the user to overwrite when, in actuality, it would not be an overwrite because they were at different levels.
A production application would need to take this into consideration.
foreach (File item in fileList)
{
if (item.Title == body.Title)
{
MessageBoxResult result = System.Windows.MessageBox.Show("The file you picked already exists " +
"in your Google Drive. Do you wish to overwrite it?",
"Confirmation", MessageBoxButton.YesNoCancel);
if (result == MessageBoxResult.Yes)
{
Utilities.UpdateFile(_service, item.Id, item.Title,
item.Description, item.MimeType, dialog.FileName, true);
}
else if (result == MessageBoxResult.No)
{
Utilities.InsertFile(_service, System.IO.Path.GetFileName(dialog.FileName),
"An uploaded document", "", "text/plain", dialog.FileName);
}
else
{
return;
}
break;
}
}
A simple foreach loop steps through the list looking to see if the file names match. If they do, we need to know that because the Drive API is a little different here.
If a file does not exist, you should Insert it into the Drive. You can see the code to call that function above. If the file exists, you need to decide if you are
going to update the current file or upload a side-by-side version. If you are going to overwrite the existing file with the new data, you must call the
UpdateFile method
passing in the new information. It is interesting here that you can 'update' the file with just new metadata without changing the actual contents. If you were renaming a file
or changing the description, this is what you would do. Of course, if you are just updating the contents, you would use all of the existing metadata.
Alternately, if a file exists you can still use the Insert method to upload the file again. The difference is that Google will create an additional
version of the file. So if foo.txt existed, the Drive service would upload foo.txt and save it as foo(1).txt in the drive.
Subsequent uploads would simply increase the number appended to the file name. For a production application, it is important to understand the ramifications
of this behavior. You wouldn't want to create 99 copies of the user's file!
In the sample code, the result of the user's decision to the message box determines
how we will upload the file they have selected. The upload or insert file methods handle the process of getting the file up to the Drive.
public static Google.Apis.Drive.v2.Data.File InsertFile(DriveService service,
String title, String description, String parentId, String mimeType, String filename)
{
Google.Apis.Drive.v2.Data.File body = new Google.Apis.Drive.v2.Data.File();
body.Title = title;
body.Description = description;
body.MimeType = mimeType;
if (!String.IsNullOrEmpty(parentId))
{
body.Parents = new List<ParentReference>() { new ParentReference() { Id = parentId } };
}
byte[] byteArray = System.IO.File.ReadAllBytes(filename);
MemoryStream stream = new MemoryStream(byteArray);
try
{
FilesResource.InsertMediaUpload request = service.Files.Insert(body, stream, mimeType);
request.Upload();
Google.Apis.Drive.v2.Data.File file = request.ResponseBody;
return file;
}
catch (Exception e)
{
Console.WriteLine("An error occurred: " + e.Message);
return null;
}
}
The point of interest in this function is FileResource.InsertMediaUpload request object. This object packages up the information needed to insert a new file in the drive.
Similarly, the UpdateFile method uses the UpdateMediaUpload object to help it. Both are instantiated with concrete objects from the
DriveService.Files object. Again, if you
were working with another API like Tasks, these would be InsertTaskUpload methods and
TaskService.Tasks objects. The Upload method executes the actual upload.
Because the request comes from the DriveService and because the drive service was setup with the Authentication methods, there is a bunch of stuff going on behind
the scenes of that Upload method. Suffice to say it is all in Google's libraries and we can simply trust that it will all work. Our
CreateAuthenticator
and GetAuthorization methods in the MainClass may finally get called the first time we execute an upload (or a file listing) command.
When successful, the method returns a Google File object that is a local proxy for the remote object. That object contains key bits of information such as
the file's ID number and it's parent. These could be used in follow-up operations to
manipulate the file without having to go the long way around an find it through the file listing method.
Although you would have to check for the existence of the file and handle situations where it no longer existed as it did when you saved the information,
you could persist the file information locally as a means to get back to it quickly the next time the application was run. The file ID should not change unless
the file is deleted and re-create by the user or some other application.
Once the upload method completes, that is it. The only thing left is to tell the
user that the file is uploaded. Now obviously, as a sample of how to do this, a lot of important things have been left out for the purposes of clarity.
A production application should make a lot of this async and should notify the user of the progress as things are going on. Error handling should also
be added as something as simple as the user not granting access can really throw a monkey wrench into the whole thing. The app should detect that
and fail gracefully. And finally, something other than a big button labeled "GO" should probably be used...
Conclusion
Once I grasped the steps for the authorization and figured out how to store and reuse the cached credentials, the Drive API became very easy to use.
Indeed, most any Google API is pretty easy to consume in .NET provided you get the authorization part straightened out. Installed applications pose
a challenge and I hope this article and the sample project save you a lot of time and headaches when it comes to adding Google API calls into your application. Good luck!
History
- Nov 2, 2012 - Initial revision.
- Dec 10, 2012 - Fixed error in code where file wasn't uploaded if it didn't exist. Cleaned up unused Google DLLs in the Google API project folder.
