Click here to Skip to main content
13,633,209 members
Click here to Skip to main content
Add your own
alternative version

Stats

10K views
437 downloads
5 bookmarked
Posted 28 Dec 2017
Licenced CPOL

Google Takeout Image Downloader

, 28 Mar 2018
Rate this:
Please Sign up or sign in to vote.
This utility downloads Google takeout images from the downloaded Json files.

Introduction

Google provides a way for users to download data from Google photos through Google TakeOut. User can download a zip file containing all (all really ?) photos. The problem is that Google selectively replaces a Json file for an image file inside the zip. This json file contains the link where the actual image is located.

The utility described in this article takes an extracted takeput folder as an input and downloads all the real images in the output folder.

Background

Some days ago, my wife's Google account happened to stop working. On investigating, I found out that she had used all the 15GB data. The main culprit was the image files. Her Android phone was syncing every image on the phone to Google Photos.

It left me with two options:

  1. Opt for additional paid storage from Google.
  2. Download the images and free up the Google storage to unlock the account.

I chose the second option.

I downloaded the Google photos zip through Google TakeOut under the impression that I would be done once I downloaded the zip. But on extracting the zip, what I found was, Google had zipped json files (which contains links to the actual image file) and not all image files.

To tackle this, I developed a utility which reads all these json files and downloads the corresponding images.

Using the Application

Prior to using this application, you will need to:

  1. Download and extract the takeout zip from Google takeout.
    More information can be found here.
  2. Once you extract the zip file, Open the demo application.
  3. Choose the extracted directory as an input directory.
  4. Choose and Output directory as where you need to download your images.
  5. Click on "Start Download".

Now sit back and relax. The application will download all the images to your output folder.

About the Source Code

There are only three classes in the demo application added:

  1. DownloadManager: Which does the actual downloading task.
  2. IDownloadProgressNotifier: An interface to provide download progress information back to the UI.
  3. Form class: for the UI to be shown. It implements IDownloadProgressNotifier interface.

Download Manager

Download manager does the actual work of:

  1. Reading the json file from the extracted folder for image link
  2. Download the image file from the link in the json file

The fields of our interest in the json file are:

  1. url: It is a link where the actual image is located.
  2. title: Title of the image. We will save the image locally with this name.

Fragment of the json looks like this:

{
  "title": "IMG_20110711_192757772.jpg",
  "description": "",
  "url": "https://lh3.googleusercontent.com/-JHYN2OSYh21s/Vaxpt7adEp2I/
         AAAAAAAABCu0/hIlcgO9TzwwkEJm2eQ9PcBu2rL1kPOqZWqwCLABGAYYCw/s0-d/IMG_20110711_192757772.jpg",
  "imageViews": "0",

Using Download Manager

To use Download manager, you will need to create the object of download manager by providing the following parameters to its constructor.

  1. Input directory
  2. Output directory
  3. Flag to tell whether continue on error or not
  4. Download progress notifier handler
DownloadManager downloader = new DownloadManager(srcPath, dstPath, true, false, this);

Now call StartDownload(). This will start the download progress. The download progress will be notified to the notifier handler which can be used to update the UI.

downloader.StartDownload();

Inside Download Manager

Download manager has only one public function.

public void StartDownload()

StartDownload() Function

This function will be a starting point for all processing. It does the following things:

  1. It calls the PreDownload function to notify the UI that download process is about to start.
    if (Notifier != null)
        Notifier.PreDownload(TotalFilesCount);
    
  2. Calls CreateDirectory to create output root directory.
    //
    //Create Folder structure in the output directory.
    //
    
    CreateDirectory(new DirectoryInfo(Path.GetDirectoryName(inputRootFolder)));
    
  3. Calls TraverseFolderTree to go through all files in the inputFolder and its subfolder.
    TraverseFolderTree(inputRootFolder);
    
  4. Function TraverseFolderTree returns when it completes downloading all files. On completion, the UI is being notified by calling the notifier function OnDownloadFinish().
    if (Notifier != null)
        Notifier.OnDownloadFinish();
    

TraverseFolderTree Function

This function goes through the input directory tree structure. It traverses the input directory structure recursively to check for all files with the extension "*.jpg.json". We are interested in these files only.

First, we need to create outputfolder path. The directory structure of the input folder and outputfolder will remain the same.

e.g. If user selects the following directories as an input and output directory.

  • Input directory: C:\Users\test\Desktop\TakeoutProject\Input
  • Output directory: C:\Users\test\Desktop\TakeoutProject\Output

If the current input processing folder is C:\Users\test\Desktop\TakeoutProject\Input\2015-07-17, then the output directory name is created by replacing the input root directory prefix with the output root directory.

The current Output directory will be C:\Users\test\Desktop\TakeoutProject\Output\2015-07-17.

String sCurrentOutputFolder = sSeedFolder.Replace(inputRootFolder, outputRootFolder) ;

The output directory is created where the image files will be downloaded.

Directory.CreateDirectory(sCurrentOutputFolder);

Now enumerate through all the files with "*.jpg.json" extension in the currentInputDirectory. These files are our candidates for download.

//
// Now go through all files. Search for jpg link files only.
// "*.jpg.json" pattern will exclude other files like metadata.json
//
foreach (String strFile in Directory.EnumerateFiles(sSeedFolder, JSON_JPEG_FILE_FILTER))
{

        ....

        ....

        ....

}

Each file path is passed to the RetrieveImage function to read the file and download the actual image file.

RetrieveImage(strFile, sCurrentOutputFolder);

The UI is notified by calling the notifier function OnCurrentFileProgress. In case of the error, the UI is notified by calling the notifier function OnError.

The final loop looks like this:

//
// Now go through all files. Search for jpg link files only.
// "*.jpg.json" pattern will exclude other files like metadata.json
//
foreach (String strFile in Directory.EnumerateFiles(sSeedFolder, JSON_JPEG_FILE_FILTER))
{
    try
    {
        if (Notifier != null)
            Notifier.OnCurrentFileProgress(strFile, ProcessedFilesCount, TotalFilesCount);

        RetrieveImage(strFile, sCurrentOutputFolder);
        ProcessedFilesCount++;
    }
    catch (Exception exc)
    {
        if (Notifier != null)
            Notifier.OnError(strFile, exc.Message);

        if (stopOnError)
        {
            //
            // We are done as we are supposed to stop on error.
            //

            throw exc;
        }
    }
}

Now traverse through all the remaining directories/subdirectories recursively.

//
// Traverse through all directories recursively.
//
foreach (String strDirectory in Directory.EnumerateDirectories(sSeedFolder))
{
    TraverseFolderTree(strDirectory);
}

RetrieveImage Function

This function downloads the .jpeg.json file and reads it for actual image file location.

private void RetrieveImage(String sJsonFilePath, String sImageFilePathOutputFolder)
        {
            JsonValue fullJson = JsonValue.Parse(System.IO.File.ReadAllText(sJsonFilePath));
            String Title = fullJson["title"];
            String url = fullJson["url"];

            using (WebClient client = new WebClient())
            {
                client.DownloadFile(url, sImageFilePathOutputFolder + 
                                    Path.DirectorySeparatorChar + Title);
            }
        }

As already stated, we are interested only in "title" and "url" field.

The "url" represents the actual web location of the file. WebClient class is used to download the image file from this URL. The output filename will be the same as that of the "title".

using (WebClient client = new WebClient())
{
    client.DownloadFile(url, sImageFilePathOutputFolder + Path.DirectorySeparatorChar + Title);
}

IDownloadProgressNotifier Interface

This interface declares the following functions. Downloader manager uses this interface to notify the downloading status.

Interface Functions

OnCurrentFileProgress(String sCurrentFileName, int processedFilesCount, int totalFileCount)

This function notifies the handler about the current file being processed along with the total progress.

PreDownload(int TotalFilesCount)

This function will be called just before start of the download.

OnDownloadFinish();

This function will be called after download finishes.

OnError(String sFileName, String sErrorInformation);

This function will be called in case any error occurs. It passes the name of the file for which error occurred as well as the error information.

Form1 Class

This class is a simple C# form class which owns the UI. Along with the simple UI elements shown in the picture, it implements the IDownloadProgressNotifier interface. The reference to this class is sent to the Download Manager as a handler. The implementation of the IDownloadProgressNotifier interface functions is used to update the UI with progress status.

IDownloadProgressNotifier Interface Implementation

OnCurrentFileProgress(String sCurrentFileName, int processedFilesCount, int totalFileCount)

This function notifies the handler about the current file being processed along with the total progress. This callback function increments the progressbar by one. This makes progressbar to move forward by one unit.

public void OnCurrentFileProgress(string sCurrentFileName, int processedFilesCount, int totalFileCount)
      {
          pbTotalProgress.Increment(1);
      }
PreDownload(int TotalFilesCount)

This callback function is used to initialize the progress bar on the UI.

public void PreDownload(int TotalFilesCount)
       {
           pbTotalProgress.Minimum = 0;
           pbTotalProgress.Maximum = TotalFilesCount;
           pbTotalProgress.Step = 1;
       }

This function will be called just before start of the download.

OnDownloadFinish();

This function will be called after download finishes. This function is used to show the process completion message box.

public void OnDownloadFinish()
       {
           pbTotalProgress.Value = pbTotalProgress.Maximum;
           MessageBox.Show("Process finished.", "Success");
       }
OnError(String sFileName, String sErrorInformation);

This callback function is used to show the error occurred.

public void OnError(string sFileName, string sErrorInformation)
        {
            MessageBox.Show("Process interrupted with an error while processing file \n File Name :" + 
                             sFileName +"\n ErrorInformation" + sErrorInformation, "Error" );
        }

History

  • First version(1)
  • Second version: Added code walk through

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

PrafullaVedante
Software Developer (Senior) nVidia Corporation
India India
Prafulla is a passionate software developer. He has mostly worked on C++ and .NET projects. Currently he is working for nVidia corporation as a Senior System Engineer.

He is an embedded technology enthusiast. He likes to spend his time on electronics projects.

You may also be interested in...

Pro
Pro

Comments and Discussions

 
GeneralMy vote of 5 Pin
raihansazal29-Mar-18 21:31
memberraihansazal29-Mar-18 21:31 
GeneralRe: My vote of 5 Pin
PrafullaVedante3-Apr-18 16:39
memberPrafullaVedante3-Apr-18 16:39 
GeneralMessage Closed Pin
28-Mar-18 2:41
memberShop4mania28-Mar-18 2:41 
Questiongoogle takeout image downloader Pin
Member 1373198819-Mar-18 1:23
memberMember 1373198819-Mar-18 1:23 
AnswerRe: google takeout image downloader Pin
PrafullaVedante28-Mar-18 0:13
memberPrafullaVedante28-Mar-18 0:13 
GeneralRe: google takeout image downloader Pin
Member 1373198830-Apr-18 0:47
memberMember 1373198830-Apr-18 0:47 
GeneralRe: google takeout image downloader Pin
PrafullaVedante16-May-18 6:22
memberPrafullaVedante16-May-18 6:22 
QuestionTakeout Pin
Member 1364181824-Jan-18 12:04
memberMember 1364181824-Jan-18 12:04 
AnswerRe: Takeout Pin
PrafullaVedante8-Feb-18 4:33
memberPrafullaVedante8-Feb-18 4:33 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web02 | 2.8.180712.1 | Last Updated 28 Mar 2018
Article Copyright 2017 by PrafullaVedante
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid