How to store a huge text file as blob field in an sqlite table ?

Question

0.00/5 (No votes)

See more:

Hello Everyone,

Hope all of you are safe.

I wondering if somebody has already came across a way to store huge text file as blob field type in an sqlite table in c++ ?

If somebody has an idea or a hint, please let me know.
Thank you very much in advance.
Have a great day.

What I have tried:

I am currently reading the entire text file and store it via the xxx_bind_xxx method but I am wondering what will happen if I have to deal with a huge text file ?

I have not found a way to write to the blob buffer sequentially but maybe is there another way ?

Posted 5-Aug-20 11:59am

SuperMiQi

Updated 5-Aug-20 17:43pm

Add a Solution

Comments

KarstenK 6-Aug-20 2:24am

as Garth wrote the correct way is to store a file somewhere else and only store the file name with path components in the database.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Garth J Lancaster · Accepted Answer · 2020-08-05T17:43:00

huge ? text ?? you don't supply numbers so a fully qualified answer may be hard

1) subject to this Implementation Limits For SQLite[^] ie 'limits', you may be able to 'zip' compress your file then insert it into a blob

- I'd suggest you write two functions or methods, storeFileAsZip retrieveFileAsZip that wrap the required functions, and maybe give you a stream back in the retrieve case

2) If your compressed (zipped) files are larger than 'blob' max size, you could use a different storage for the files and store the path / access method in the database

You may be able to glean more info/use Namespace Poco::Data[^] to help you.

Btw ..if you're going to store a file in a blob, I'd also include a 'checksum' of the original file, so you can verify integrity etc