C / C++ / MFC

Re: How to fast read excel file, with VC2005

18-Mar-15 4:02

Which library?

lichengbyd wrote:
In this example, it's using loop to read cells, the reading speed is not good.

It is using the loops to extract the values from the safe array. But the data are already in memory of your application. You missed the call that retrieves the data from Excel:

// Get the data.
COleSafeArray saRet(oRange.get_Value(covOptional));

lichengbyd18-Mar-15 4:30

lichengbyd

18-Mar-15 4:30

Indeed, thanks for you infomation!

Writing Vector object into a .txt file

Member 935023718-Mar-15 0:16

18-Mar-15 0:16

Hi all!,

I've been working on a code to develop a tracking game with a haptic device as controller. I use library Chai3D and OpenGL.

I need to save some information of the game during the program execution. And write into a txt file.

I'm attaching part of my code where mentioning vector objects need to be saved:

C++

void updateHaptics(void)
{
    // main haptic simulation loop
    while(simulationRunning)
    {
        // for each device
        
            // read position of haptic devices 1,2
            cVector3d  newPosition1,newPosition2;
			 
			//hapticDevices[i]->getPosition(newPosition);
            hapticDevices[0]->getPosition(newPosition1);
			hapticDevices[1]->getPosition(newPosition2);

            // read orientation of haptic devices 1, 2
            cMatrix3d newRotation1, newRotation2;
            hapticDevices[0]->getRotation(newRotation1);
			hapticDevices[1]->getRotation(newRotation2);

            // update position and orientation of cursors
            cursors[0]->setPos(newPosition1);
			cursors[1]->setPos(newPosition2);
            cursors[0]->setRot(newRotation1);
			cursors[1]->setRot(newRotation2);

            // read linear velocity from devices 1,2
            cVector3d linearVelocity1, linearVelocity2;
            hapticDevices[0]->getLinearVelocity(linearVelocity1);
			hapticDevices[1]->getLinearVelocity(linearVelocity2);

            // update arrow for device 1
            velocityVectors[0]->m_pointA = newPosition1;
            velocityVectors[0]->m_pointB = cAdd(newPosition1, linearVelocity1);
			
			// update arrow for device 2
			velocityVectors[1]->m_pointA = newPosition2;
            velocityVectors[1]->m_pointB = cAdd(newPosition2, linearVelocity2);
			
			
			// Apply a coupled force F1= Kp (X1-X2)
			
			double Kp = 60.0; // [N/m]
			double Kb = 3.0; // [N.s/m]
			
			cVector3d newForce (0,0,0);
            cVector3d coupledForce1 = cMul(Kp, (newPosition1-newPosition2))-(cMul(Kb, (linearVelocity2)));
			newForce.add(coupledForce1);
			hapticDevices[1]->setForce(newForce);// send computed force to haptic device 1

		    
			cVector3d newForce1 (0,0,0);
			cVector3d coupledForce2 = cMul(Kp, (newPosition2-newPosition1))-(cMul(Kb, (linearVelocity1)));
			newForce1.add(coupledForce2);
			hapticDevices[0]->setForce(newForce1);// send computed force to haptic device 0
		
    }
    
    // exit haptics thread
    simulationFinished = true;
}

On the above code I need to save these objects into a file

cVector3d newForce
cVector3d NewForce1

Please help me how to make a provision to save these file effectively into a txt file.

Basically I dont have much idea on saving , writing into files.

Thanks in advance!

Jochen Arndt18-Mar-15 0:46

18-Mar-15 0:46

There is an edit link below your post which should be used rather than posting a new question.

There is also a delete link. Please use that to delete your first question.

To write the elements to a text file, you may open the file using fopen() and then print the elements using fprintf():

C++

FILE *f = fopen(fileName, "wb");
fprintf(f, "%lf %lf %lf\r\n", newForce.x, newForce.y, newForce.z);
fprintf(f, "%lf %lf %lf\r\n", newForce1.x, newForce1.y, newForce1.z);
fclose(f);

Depending on the required accuray it may be necessary to specify the number of significant digits to be printed (see the printf() format specifications).

[EDIT]
The above is for Windows text files. For Unix text files remove the '\r' from the fprintf() format strings.

Member 935023718-Mar-15 1:38

18-Mar-15 1:38

Hey Thank you Jochen!.

Im just wondering how can i make use of a buffer where intitially I can dumb my data..and once the execution is completed writing into a file..do you have a suggestion?.

Jochen Arndt18-Mar-15 1:59

18-Mar-15 1:59

You can do it similar using sprintf() to print to a buffer and then write the buffer to file:

C++

char buffer[128];
char *p = buffer;
int n = sprintf(p, "%lf %lf %lf\r\n", newForce.x, newForce.y, newForce.z);
s += n;
sprintf(p, "%lf %lf %lf\r\n", newForce1.x, newForce1.y, newForce1.z);
FILE *f = fopen(fileName, "wb");
fwrite(buffer, 1, strlen(buffer), f);
fclose(f);

But as you can see this is much more code and it is insecure without checking for buffer overflows (which adds more code).

If you don't need the data to be in text format (e.g. only loaded by your application), it might be better to write the data to a binary file. That solves also the problem of inaccuracies when converting floating point values to text and back again later.
Example:

C++

FILE *f = fopen(fileName, "wb");
fwrite(&newForce.x, sizeof(newForce.x), 1, f);
// Write other items here
fclose(f);

Reading is then done in a similar way:

C++

FILE *f = fopen(fileName, "rb");
fread(&newForce.x, sizeof(newForce.x), 1, f);
// Read other items here
fclose(f);

Member 935023718-Mar-15 5:03

18-Mar-15 5:03

Many thanks Jochen!! Thumbs Up | :thumbsup:

.

I'm planning to use just one file containing information about two forces, something like this:

timestamp newforce(X Y Z) newForce1(X Y Z))

where the values of x,y,z will be in column.

C++

fprintf(pFile, "%lf %lf %lf\r\n %lf %lf %lf\r\n", newForce.x, newForce.y, newForce.z, newForce1.x, newForce1.y, newForce1.z);

do you think it's a right assignment?, I mean how can I write data effectively into the file with proper indentation?

Jochen Arndt18-Mar-15 5:19

18-Mar-15 5:19

You can use the printf format specifiers (see http://www.cplusplus.com/reference/cstdio/printf/[^]).

Using the width specifier, all numbers will have the same width padded with blanks. For proper alignment of the decimal point you should also use the precision specifier and the flag specifier (use a space or minus). The final values to be used depend on the range of your values and the required precision. Assuming a range up to (but excluding) 100 and a precision of 2 fractional digits:
Precision = 2
Width = 1 (sign) + 2 (digits) + 1 (decimal point) + 2 (digits) = 6

Then use the format "% 6.2f" for each value:

fprintf(pFile, "% 6.2f % 6.2f % 6.2f\r\n% 6.2f % 6.2f % 6.2f\r\n", 
    newForce.x, newForce.y, newForce.z, newForce1.x, newForce1.y, newForce1.z);

For the time stamp you can use the strftime()[^] function.

modified 18-Mar-15 13:08pm.

Member 935023718-Mar-15 6:33

18-Mar-15 6:33

I tried as u suggested , but now the data I'm getting is kind of overflowing, I mean it's always generating a data txt file of 1kb!!

Jochen Arndt18-Mar-15 6:59

18-Mar-15 6:59

You may give an example of your data. Inspecting the text file may also give a hint what is going wrong.

[EDIT]
I have replaced the usage of "%lf" by "%f" in my post. But this should make no difference with MS compilers.

modified 18-Mar-15 13:09pm.

Member 935023718-Mar-15 22:32

18-Mar-15 22:32

Hi Jochen,

I'm getting data like this

18832315686612578000000000000000000000000000000000000000000000000000.00 18832315686612578000000000000000000000000000000000000000000000000000.00 18832315686612578000000000000000000000000000000000000000000000000000.00
18832315686612578000000000000000000000000000000000000000000000000000.00 18832315686612578000000000000000000000000000000000000000000000000000.00 18832315686612578000000000000000000000000000000000000000000000000000.00

It's 2 raws of numbers..3,5 lines are continuation of 1st line..ans so 4,6 ..

It's basically limited to 1kb irrespective of the duration of experiment

Jochen Arndt18-Mar-15 22:44

18-Mar-15 22:44

That are quite large numbers. Are they the correct values? If your numbers are in that range, you should use the "%E" format.

Member 935023719-Mar-15 22:27

19-Mar-15 22:27

Hi Jochen,

Now it's working perfectly!. I did a a mistake in closing the file. Thank you so much for your prompt and precise feedback!.

Jochen Arndt19-Mar-15 22:29

19-Mar-15 22:29

Fine that all is working now and thank you for the feedback.

Member 935023723-Mar-15 5:12

23-Mar-15 5:12

Hi Jochen,

The data i'm getting is of very large sample. I use a inbuilt thread class calls at very high frequency. Basically I want to reduce the sampling rate so that I'll get significant data with less samples..Do you have any idea how to create periodic thread with a fixed frequency?

Jochen Arndt23-Mar-15 5:57

23-Mar-15 5:57

It is difficult to answer this without knowing how samples are triggered, new data are signaled, and what is done by your thread class.

When sampling is triggered by your app, you can control the frequency. If it is free running you may skip samples (e.g. processing only every second one).

When using a worker thread you usually call a wait function (e.g. WaitForSingleObject) that returns upon events or a time out. The time out value can be used for periodic execution.

Member 935023723-Mar-15 6:20

23-Mar-15 6:20

Basically two functions one is for updating graphics and other for haptics rendering. Haptic rendering sampling rate is quite large compared to graphics. In the library I'm using there is a thread class basically gives priority to haptic and graphics rendering. But I'm not able to see any section in the code where they access system timing and setting sampling rate etc.

I'm thinking about assigning some condition inside data will be written into the file at a rate of 100 samples per second. There is an inbuilt precision clock with the library, so if I've I can get cpu time with precision clock , how can formulate a condition to write data at a rate 100Hz??

Jochen Arndt23-Mar-15 6:44

23-Mar-15 6:44

As far as I understood you have a haptic device which is some kind of hardware which generates data at a frequency defined by the hardware. If the library does not provide functions to set the sample rate, it is probably fixed. The thread is then activated when new data are available.

But again, without knowing the hardware and the library (which acesses the hardware specific driver), it is rather impossible to answer.

Do you want to write the data to a file with 100 Hz. This is probably too fast (especially when the file must be opened for each write). It again depends on your requirements (who is reading the file). If the data should be read by another process, you may use some kind of IPC (Inter Process Communication).

Regarding timers:
Windows has no high resolution timers. While it is possible to measure times with high resolution (QueryPerformanceCounter), the system timers have a resolution of about 10 ms by default and can be tweaked down to 1 ms.

Richard Andrew x6423-Mar-15 8:39

Richard Andrew x64

23-Mar-15 8:39

Jochen Arndt wrote:
the system timers have a resolution of about 10 ms by default and can be tweaked down to 1 ms.

What about the waitable timers? SetWaitableTimer[^]

Quote:
pDueTime [in]
The time after which the state of the timer is to be set to signaled, in 100 nanosecond intervals.

The difficult we do right away...
...the impossible takes slightly longer.

Jochen Arndt23-Mar-15 21:53

23-Mar-15 21:53

Windows uses internally 100 ns based times (FILETIME structure). But this does not mean that the internal timer tick is running with a resolution of 100 ns.

There are undocumented functions to query the range of resolution and set the resolution (NtQueryTimerResolution, NtSetTimerResolution). On a typical system, the range is 0.5 to 15.6 ms.

Member 935023723-Mar-15 23:47

23-Mar-15 23:47

How can I configure a periodic thread of particular frequency?

Jochen Arndt24-Mar-15 0:17

24-Mar-15 0:17

Just call Sleep() with a time out value of 1000 / frequency[Hz].

A better implememntation would use WaitForSingleObject() with the same time out value and a handle to a terminate thread event so that you can terminate the thread:

C++

UINT worker_thread_func(LPVOID pParam)
{
    // pParam is usually a pointer to a C++ class to which this thread belongs
    //  that is passed when creating the thread.
    // Cast the pointer to get access to member vars.
    MyClass *pThis = (MyClass *)pParam;
    HANDLE hKillEvent = pThis->m_hKillEvent;
    while (1)
    {
        if (WAIT_OBJECT_0 == WaitForSingleObject(hKillEvent, TIME_OUT_VAL))
            break;
        // Perform periodic task here
    }
    return 0;
}

Member 935023725-Mar-15 4:25

25-Mar-15 4:25

Hi Jochen,

I still didn't get idea of using WaitForSingleObject() as a timer. Basically WaitForSingleObject is a predefined function under windows.h? right?. Can you bit explain how exactly this can be embedded with a project?

Jochen Arndt25-Mar-15 4:46

25-Mar-15 4:46

It is a Windows API function that is used with threads. It will suspend the thread until the event specified by the passed handle occurs or the time out time has elapsed.

Using threads is an advanced topic. So you should read about it first. A starting point may be Using Worker Threads[^].

Your requirement was to execute specific tasks in fixed intervals. This can be done by Windows timers from within your main (GUI) thread. But this will block the main thread for the task execution time. With short intervals, this will lead to delayed user input (mouse and keyboard actions are not performed immediately). To avoid this, a worker thread can be used.

Message Closed

25-Mar-15 6:47