User-defined Resource or other data format

Question

0.00/5 (No votes)

See more:

VS2010

I have a lot of data that I was compiling into a *.lib file. It's basically the equivalent of an Excel spreadsheet with 4,000 rows and 75 columns. I wrote a quick routine that read from a .cdf (.xls saved as .cdf) and wrote in the proper format of an array of structs with 4,000 elements, each element having 75 members to a .cpp which was part of my library project. Everything works fine and I was happy with doing it that way - the user gets a single .exe file. In VC2008, it takes about 40 minutes for the .lib file to build which isn't a big problem for me because I only need to change the data about once a year. FYI, the .cdf file is proprietary so I don't want to distribute it to the users who may inadvertently (or purposefully) modify it, but the .exe file needs to have access to the data.

I have migrated from VS2008 to VS2010. The build time for the .lib file is a little longer (about 50 minutes) but the big problem is that the .exe files that use it now take just as long to build (in VS2008, they took about 30 seconds each). That's about 100 times longer in VS2010 than in VS2008, with the same project settings in both. I can't be productive with the .exe projects (which I'm constantly updating and adding to) if it's going to take 50 minutes every time I want to test a change.

Someone suggested that I look into a user-defined resource which wouldn't be compiled and would significantly speed up my build times. Does anyone have a sample of how to use an .xls, .cdf, .txt, etc. file as a resource (or a better way of doing it than an array of structs in a library file)?

Thanks in advance,
Ed

Here is some sample code to show you how I do it now - Thanks, Ed

C++

////////////////////////////////////////////////////////////////////////////////////////
// This is a sample of the kind data structure I use to exchange stored data between the
// .lib file and the .exe file, and load into a dialog
// Significantly shortened for simplicity - the real one has 75 members
//
struct EMDData // in a header file #included in both the .lib and the .exe
{
CString csKey; // The key
CString csName; // (e.g. "PART_0375221")

double dWeight; // Weight (lbs)
double dDepth; // Depth (in)

int nFlange; // Flange type
int nBase; // Base type
};

////////////////////////////////////////////////////////////////////////////////////////
// Here's the Function from the .lib project that has all the data
// Significantly shortened for simpliity - the real one has #define NUMNAMES 4125
// and remember the EMDData struct has 75 members
// The MY_DATA[] array was written by a short routine that read from an .sdf file and
// wrote it this way. The .sdf file is like:
//
// W2209613 WXPRQTBG 335.0 44.000 21 1001 .....
// W3389512 VYAPSAFH 322.0 43.875 21 1001 ......
//
/****** FUNCTION getDataFromStruct() ******/
BOOL getDataFromStruct(CString csKey, struct EMDData *EData) // in file My_Data.cpp
{

int i;
BOOL bFound = FALSE;

struct EMDData MY_DATA[NUMNAMES];

MY_DATA[0].csKey = _T("W2290613");
MY_DATA[0].csName = _T("WXPRQTBG");
MY_DATA[0].dWeight = 335.0000;
MY_DATA[0].dDepth = 44.0000;
MY_DATA[0].nFlange = 21;
MY_DATA[0].nBase = 1001;

MY_DATA[1].csKey = _T("W3389512");
MY_DATA[1].csName = _T("VYAPSAFH");
MY_DATA[1].dWeight = 322.0000;
MY_DATA[1].dDepth = 43.8750;
MY_DATA[1].nFlange = 21;
MY_DATA[1].nBase = 1001;

// -> continues through MY_DATA[4124]

for(i=0;i<numnames;i++)>
if(MY_DATA[i].csKey.Compare(csKey) == 0) {

EData->csKey = MY_DATA[i].csType;
EData->csName = MY_DATA[i].csName;
EData->dWeight = MY_DATA[i].dWeight;
EData->dDepth = MY_DATA[i].dDepth;
EData->nFlange = MY_DATA[i].nFlange;
EData->nBase = MY_DATA[i].nBase;

bFound = TRUE;
break;

}
}

return bFound;

} /* end of function getDataFromStruct() */

////////////////////////////////////////////////////////////////////////////////////////
// From the .exe file, from a dialog where the user selects the Part, I would have a
// something like the following:
//

if(getDataFromStruct(m_csPart, &EData) {
// -> load the dialog box
// etc.....
} else {
EMDAlert(_T("Error 469: Data not retrieved."));
// etc.....
}

C++

Posted 27-Apr-12 6:35am

EMD1954

Updated 29-Apr-12 1:12am

v2

Add a Solution

3 solutions

Solution 3

Thank you all for your comments. It's a little above my skill level but that doesn't mean I can't teach myself a little more. One last question, once I have a pointer to the data, how would I, for example, access the information in the 10th row, 12th column? FYI, the first column is a key so I would search that first, know what row I'm in, and then grab whatever column I needed.

I will also look into encrypting the data so I can do a simple search on it.

Thank you all again,
Ed

PS. My current system is XP-Pro, SP-3. I'm getting a new machine this week with Windows 7 and 12 MB of ram, and I'm hoping the compiler will be much faster. I don't know why I'm having such a difficult time in the transition between VC2008 and VC2010 - thanks again.

Posted 28-Apr-12 0:42am

EMD1954

Updated 28-Apr-12 0:48am

v2

Comments

nv3 28-Apr-12 7:39am

You are welcome, Ed.

Just as a reminder to the common etiquette here: Please don't paste your comments and follow-up questions as a solution, but use the reply button to one ore more of the existing solutions. That way, the friendly author receives an e-mail notification that you are waiting for an answer. And don't forget to accept those solutions that you have found valuable in resolving your problem.

To your question: "How do I access the information in the resource?":
Whatever you put in your file is accessible as an array of bytes. If you put text in there, you can scan it like a text string. If you put binary data in there, you can cast the pointer into whatever structure you have given the data.

EMD1954 28-Apr-12 9:07am

Got it. Sorry about that.

Thank you all again,
Ed

nv3 28-Apr-12 9:15am

No problem, Ed. If you need more help on the way, just add additional questions to your post by "Improve question". I would like to encourage you using a resource as it is not that hard to do. To give you coding example for accessing your structures, please add some more information on the contents of your .cdf file. How are you reading this file in your present code?

EMD1954 28-Apr-12 15:02pm

Is there a way that I can attach a small file to show you some samples that will show you how I currently do everything? Or should I just cut and paste it here?

nv3 29-Apr-12 3:41am

I don't know of any way to attach a file. So I would attach it with "Improve question" to your original question. There you can put it between <pre> marks to have it formatted as source text (use the "code" button above the edit window).

EMD1954 1-May-12 10:09am

nv3,
I posted some code in the original question as you suggested. Take a look and let me know what you think.
Thanks,
Ed

nv3 1-May-12 14:44pm

Hi Ed,

I don't see anything new on your original request. I have already pasted an answer to your code in the "AMENDED" section of solution 1. Did you see that?

EMD1954 1-May-12 15:10pm

Thanks. I'm still getting used to how this works - I didn't see your amended response - sorry.
Thank you. I will try exactly what you're suggesting.
Ed

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

nv3 · Accepted Answer · 2012-04-27T09:17:00

In the .rc2 file of your application you would place an include directive for your data file as resource:

MyResName   CDF	    DISCARDABLE		"MyFileName.cdf"

Your .cdf file can contain text or binary data, just as you like. This file will then be placed by the resource compiler / linker inside your .exe file.

To access that resource from your program, you use a sequence like this:

HRSRC hrc = FindResource (hInst, "MyResName", "CDF");
if (hrc == NULL)
    return false;
HGLOBAL hgb = LoadResource (hInst, hrc);
if (hgb == NULL)
    return false;
LPVOID prsrc = LockResource (hgb);
UINT size = SizeofResource (hInst, hrc);

where hInst is the instance handle of your .exe file, which is the m_hInstance member of your CWinApp object. If you place that code in your main program you can even use NULL as hInst, which then defaults to searching the exe/dll file that Windows used to start the process.

Now you have a pointer to your data and a size in bytes. Its as simple as that.

Good luck.

AMENDED: After seeing how your generation process works I would recommend the following steps:

1. Put the .sdf file as resource in your program as described above

2. Take that little function that reads your .sdf and generates the source of your .lib file and include it in a slightly modified way in your main program. It should now read from the included resource, which appears as a big string buffer to it. Cast that prsrc to a PCSTR and give that to your function. The main modification you will have to do is:

Make your function read from a string instead of a file (that should be simple) and let it populate the MY_DATA structure directly instead of generating code lines. Thus it's in fact a simplification of that function.

3. When your program starts locate the resource as shown above and call your function the prsrc and let it fill your MY_DATA array (which must now be placed in your main program instead of the library).

When all that works you can start thinking about encrypting the resource, which is also a relatively easy task.

Let me know how you get on with it by pasting a reply to this solution (so I get an e-mail notification) and call for help if you need any.

Aescleal · Accepted Answer · 2012-04-27T09:34:00

Solution 2

As well as looking at a resource you might get some milage out of compiling your data into a DLL and then exporting a pointer to it. As you won't have to link the static data into each EXE (all you need is a pointer to how ever many access points you want for the data) the build will probably be faster.

To be honest I haven't seen link times go through the roof with VC++2010. Does your system thrash a lot when it's linking? If it does then it might be worth grabbing some extra RAM and seeing if that has any effect.

Cheers,

Ash

Posted 27-Apr-12 9:34am

Aescleal

Comments

nv3 27-Apr-12 15:58pm

Good point. And just to add one more: It seems that hiding the data from the end-user is the main point. So why not encrypt them into a data file and distribute that data file together with the .exe file, which would open and decrypt it with a built-in key. Easy to do and that would avoid the hassle of compiling those large data tables in the first place. And it's even safer than relying on the end-user not being able to figure what's in the resources of the .exe file.

Aescleal 27-Apr-12 16:07pm

Good point about resources - any idiot can open them with a resource editor, as this idiot (that's me, not the original poster) here has done with idiotically designed games to get rid of the chunk of the manifest that demand to be run elevated. Anyway this means they'd have to be encrypted or something similar. I like the idea of encrypted data files. If you were feeling particularly mean you could even have it as part of a licensing system - you have to grab the key/keys off a website first although you might loose a few users.