Click here to Skip to main content
16,016,759 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I need to upload multiple large files in chunks using the following code:

The first file of the foreach upload works correctly, but I encounter issues starting before the second file upload.
The return type of $service->files->listFiles is GuzzleHttp\Psr7\Request, most likely because the media upload is deferred with the line $client->setDefer(true);.

I would like to find a solution where the loop waits for each upload to complete before moving on to the next file, similar to a thread. Alternatively, is it possible to send multiple uploads simultaneously (async/parallel)?

What I have tried:

PHP
$service = new Google\Service\Drive($client);

foreach ($files as $fileDatas) {

    //Check if a file with this name has already been uploaded
    $is_file_exists = $service->files->listFiles($Params);
    if (count($is_file_exists->files) > 0) {
       return false;
    }
    
    $file = new Google\Service\Drive\DriveFile();
    $file->name = $fileName;
    $chunkSizeBytes = 1 * 1024 * 1024;

    // Call the API with the media upload, defer so it doesn't immediately return.
    $client->setDefer(true);
    $request = $service->files->create($file);

    // Create a media file upload to represent our upload process.
    $media = new Google\Http\MediaFileUpload($client, $request, $type, null, true, $chunkSizeBytes);
    $media->setFileSize($size);

    // Upload the various chunks. $status will be false until the process is complete.
    $status = false;
    $handle = fopen($filePath, "rb");
    while (!$status && !feof($handle)) {
        // read until you get $chunkSizeBytes from TESTFILE
        // fread will never return more than 8192 bytes if the stream is read
        // buffered and it does not represent a plain file
        // An example of a read buffered file is when reading from a URL
        $chunk = readVideoChunk($handle, $chunkSizeBytes);
        $status = $media->nextChunk($chunk);
    }

    // The final value of $status will be the data from the API for the object
    // that has been uploaded.
    $result = false;
    if ($status != false) {
        ...
    }

    fclose($handle);
}
Posted
Updated 24-Jun-23 6:38am
v3
Comments
Patrice T 23-Jun-23 12:11pm    
Did you tried 1 by 1 ?
Deltahun 24-Jun-23 11:33am    
Yes. If the $files has only one element, it works. I also tried uploading each file separately and it worked.

Quote:
is it possible to send multiple uploads simultaneously (async/parallel)
- yes, it is but in my opinion, using parallel uploads in PHP with Google Drive API can be quite complex. I know that you can use PHP libraries or frameworks that support asynchronous operations, such as ReactPHP or Swoole or maybe use other languages like Node.js or Golang that has libraries for this.

To upload your files and wait for each to complete before starting the next in your loop, I will drop the defer option and make synchronous requests instead of asynchronous ones -
foreach ($files as $fileData) {
    $file = new Google\Service\Drive\DriveFile();
    $file->name = $fileName;
    $chunkSizeBytes = 1 * 1024 * 1024;

    // Create a media file upload
    $media = new Google\Http\MediaFileUpload($client, $file, $type, null, true, $chunkSizeBytes);
    $media->setFileSize($size);

    // Upload the file in chunks
    $handle = fopen($filePath, "rb");
    while (!$media->isFinished()) {
        // Read the next chunk from the file
        $chunk = readVideoChunk($handle, $chunkSizeBytes);
        $media->nextChunk($chunk);
    }

    // Get the result of the upload
    $result = $media->getResult();
    fclose($handle);

    // Do something with the result
    if ($result != false) {
        // Process the result...
    }
}
 
Share this answer
 
v2
Comments
Deltahun 24-Jun-23 11:45am    
I got error when create an instance of a class DriveFileUpload: "Google\\Http\\MediaFileUpload::__construct(): Argument #2 ($request) must be of type Psr\\Http\\Message\\RequestInterface, Google\\Service\\Drive\\DriveFile".

In case, I use request as argument, I got another error: "$media->isFinished()" method doesn't exists for MediaFileUpload: "Call to undefined method Google\\Http\\MediaFileUpload::isFinished()"
I solved it by resetting "setDefer" to the client at the end of the loop:
PHP
fclose($handle);
$client->setDefer(false);
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900