|
Please refer my posting dated 29-Apr-16 3:52. I am still finding problem with that. Even If I try to send the data as small chunks as per the suggestions here, it gets struck or hanged.
Sever Side Code for Sending data:
void CMainFrame::OnSendactiveInsdata()
{
// TODO: Add your command handler code here
if(!InstClosedFlag)
{
TransData.Init();
long nSize = sizeof(struct TagModalDB);
memset(EsimData, 0, nSize);
memcpy(EsimData, Esim, nSize);
long nApproxSize = 900000;
int nEnd = 0;
long j = 0;
int nOffset = nApproxSize/100;
for (int i=0; i<100; i++)
{
TransData.Init();
memset(TestData, 0, 9000);
memcpy(TestData, EsimData + nEnd, nOffset);
for(j = 0; j < nOffset; j++)
{
TransData.m_sData.Insert(j, TestData[j]);
}
SendData(m_pInsSocket, &TransData);
nEnd += nOffset;
if ((nEnd + nOffset) > nSize) nOffset = nSize - nEnd;
}
Sleep(500);
}
}
Client Side Code for Receiving data:
void CMainFrame::ProcessPendingReadIns(void)
{
long nApproxSize = 900000;
int nEnd = 0;
long j = 0;
int nOffset = nApproxSize/100;
long nSize = sizeof(struct TagModalDB);
memset(EsimData, 0, nApproxSize);
for (int i=0; i<100; i++)
{
TransData.Init();
ReceiveData(&TransData);
memcpy(EsimData + nEnd, (const char*)TransData.m_sData, nOffset);
nEnd += nOffset;
if ((nEnd + nOffset) > nSize) nOffset = nSize - nEnd;
}
memcpy(Esim, EsimData, nSize);
Sleep(500);
}
Please provide me suggestions and step by step approach to send/receive huge data continously in cycles.
Previous Posting:
How to send huge data via sockets in continuous intervals
Our MFC Server application send huge real time data of size 900000bytes using CSocket (TCP/IP Sockets) via serialization continuously every 2 second to the MFC Client.
But After a few minutes, the application gets slowed down.
Is there any specific ways or methods of send/receive huge data continuosly via Csocket.
How to send the big data every interval in a better way.
|
|
|
|
|
Your code could be made more efficient just by a bit of tidying up. The following two lines will add considerably to processing time:
memset(EsimData, 0, nSize);
memcpy(EsimData, Esim, nSize);
Why clear the buffer to zeroes and then immediately overwrite it with data? The memset serves no purpose. Later you copy partial buffers into your large buffer a block at a time, again using memset for no good reason. Why not just receive the data direct into the large buffer?
|
|
|
|
|
I removed the memset statement. Server is Ok now. But the client is still hanging
|
|
|
|
|
manoharbalu wrote: But the client is still hanging Then you need to do some debugging. There is no way anyone here can guess why.
|
|
|
|
|
Of coarse it slows down you send the client and server to sleep for 1/2 a second everytime it sends or recieves. You do get that sleeps the entire MFC framework don't you?
It appears to be a some sort of half arsed timing mechanism that will inevitably fail ... there are things called TIMERS for timing functions, use them.
If you want it blunt get rid of these and things will work a whole lot quicker =====> Sleep(500);
If you want me to be super blunt there is no good reason to sleep an MFC or any Win32 program ever, forget that call and never use it in proper code it's a hack we sometimes use when trying to proto stuff. There are a multitude of proper ways to load balance and wait for things in nice Windows proper ways.
In vino veritas
modified 23-May-16 4:28am.
|
|
|
|
|
I removed the Sleep, but still the client is hanging.
|
|
|
|
|
This needs to be done properly but lets see if we can get this at least going, so let me guess at the next problem what is Transdata?
You loop in some sort of weird poll loop 100 times and call Transdata.Init ... if that is a constructor you just made 100 transdata and you never dispose of them. Check if it is your responsibility to dispose of TransData. However you can pull up windows resource manager and see if your client program memory growing and growing and growing because that will be the source and rather quickly. You don't dispose of transdata and if what I fear is true your program will leak memory like a sponge.
I actually can't get why you even need transdata, can't you just send the receive data to your buffer EsimData
Wouldn't something like
ReceiveData(&Esimdata[nEnd]);
put the read data directly where you need it without the the need for transdata. I don't know what EsimData is so the format may not be right but explain why you can't directly put the data in in Esimdata.
I sort of need to know what Transdata and Esimdata are to help any further.
Finally if you really want to do the receiving by polling, which is fine then do it properly setup a timer and check for received data from time to time don't sit in a loop and spin your wheels for 100 times.
In vino veritas
|
|
|
|
|
The loop is not a poll loop. Just segmenting the data 900000bytes/100 and sending it 100 times, the same way I am receiving it at the client. And the Init function just initialises the CString var. I send/receive the data through Serialization and CTrans is derived from CObject for that purpose. And thats why I am using the EsimData as intermediate. Its declared globally as...
char Esimdata[900000];
void CTrans::Init()
{
m_bClose = FALSE;
m_sData = _T("");
}
Please correct me if I am wrong.
|
|
|
|
|
I am still concerned you are bleeding memory did you check the program memory usage isn't continually growing?
Can you use m_sData.Empty(); rather that m_sData = _T(""); I rarely use MFC strings but that line is ringing alarm bells with me.
To smooth the recption down you really need to build a parser on the receive data rather than just sit looping around waiting for the full packet.
Can I get a look at the code for ReceiveData .. as in the call ... ReceiveData(&TransData);
Finally can I suggest you format the data into 3 sections, which was actually suggested to you back in April to make parsing easier and robust.
The packet transmission will be a (i) A fixed signature (ii) a data body size (iii) the body data of the size of (ii)
The parser would be 4 stage .... read signature, read data size, read data, data available (do what you want with it).
enum ReadParseStatus {
readSigState = 0,
readDataSizeState,
readDataBodyState,
readDataAvailState
};
The fix signature needs to be something unlikely to be in data along lines of
const char signature[] = "PACKETSIG%$#";
Your server needs to output matching data packets. Same signature, the size of the data body and then the data body.
If you provide the ReceiveData code I can help with the parser setup, which is a lot better than looping around waiting for data and it's pretty trivial.
In vino veritas
modified 24-May-16 10:49am.
|
|
|
|
|
void CMainFrame::ReceiveData(CTrans* pData)
{
/*if(CAsyncSocket::FromHandle(m_pSocket->m_hSocket) == NULL)
{
m_pSocket->Attach(m_pSocket->m_hSocket);
}*/
TRY
{
if( m_pArchiveIn )
pData->Serialize(*m_pArchiveIn);
else
{
pData->m_bClose = TRUE;
m_pArchiveOut->Abort();
}
}
CATCH(CFileException, e)
{
pData->m_bClose = TRUE;
m_pArchiveOut->Abort();
}
END_CATCH
if (pData->m_bClose)
{
delete m_pArchiveIn;
delete m_pArchiveOut;
delete m_pFile;
delete m_pSocket;
m_pArchiveIn = NULL;
m_pArchiveOut = NULL;
m_pFile = NULL;
m_pSocket = NULL;
}
}
|
|
|
|
|
Okay 1 polled 4 state parser code .. I commented it so you can follow what its doing. The parse buffer is 4K so you need to poll min rate of 20-30 times a second. You can increase or decrease the parse buffer size and change the poll rate requirements. Just remember your 90K data/ parse buffer size = min poll rate you need.
You can use a timer or hook into the MFC message system to fire of a poll request ... personally I would use a timer. If its just writing the data to a file or something you could just loop the poll call in a thread.
The data body is available in the dataReadyState and you haven't indicated what you do with it so its blank. The parser just cycles an maintains and cleans its own buffers.
enum ReadParseStatus {
readSigState = 0,
readDataSizeState,
readDataBodyState,
readDataAvailState
};
const char signature[] = "PACKETSIG%$#";
enum ReadParseStatus parseState = readSigState;
char parseBuffer[4096];
int parsePos = 0;
long bodyDataSize = 0;
unsigned char* bodyData = NULL;
long bodyDataPos = 0;
BOOL Poll_Receive_Data (CAsyncSocket sock){
int iResult = sock.Receive(&parseBuffer[parsePos], sizeof(parseBuffer) - parsePos);
if (iResult <= 0) return (FALSE);
parsePos += iResult;
switch (parseState){
case readSigState: {
char tempbuf[32];
int siglen = strlen(signature);
BOOL sigFound = FALSE;
while ((sigFound == FALSE) && (parsePos > siglen)) {
memcpy(tempbuf, &parseBuffer[0], siglen);
tempbuf[siglen] = 0;
if (strcmp(signature, &tempbuf[0])){
memmove(&parseBuffer[0], &parseBuffer[siglen], parsePos - siglen);
parsePos -= siglen;
sigFound = TRUE;
parseState = readDataSizeState;
} else {
memmove(&parseBuffer[0], &parseBuffer[1], parsePos - 1);
parsePos--;
}
}
}
break;
case readDataSizeState: {
char tempbuf[sizeof(long)+1];
if (parsePos > sizeof(long)) {
memcpy(tempbuf, &parseBuffer[0], sizeof(long));
tempbuf[sizeof(long)] = 0;
bodyDataSize = atol(&tempbuf[0]);
if (bodyData) free(bodyData);
bodyData = (unsigned char*) malloc(bodyDataSize);
bodyDataPos = 0;
memmove(&parseBuffer[0], &parseBuffer[sizeof(long)], parsePos - sizeof(long));
parsePos -= sizeof(long);
parseState = readDataBodyState;
}
}
break;
case readDataBodyState: {
if (parsePos > 0) {
long amountLeft = bodyDataSize - bodyDataPos;
int transferSize = parsePos;
if (transferSize > amountLeft) transferSize = amountLeft;
memcpy(&bodyData[bodyDataPos], &parseBuffer[0], transferSize);
bodyDataPos += transferSize;
memmove(&parseBuffer[0], &parseBuffer[transferSize], parsePos - transferSize);
parsePos -= transferSize;
}
if (bodyDataPos == bodyDataSize) {
parseState = readDataAvailState;
}
}
break;
case readDataAvailState: {
parseState = readSigState;
}
break;
}
return (TRUE);
}
In vino veritas
modified 26-May-16 5:41am.
|
|
|
|
|
manoharbalu wrote:
Is there any specific ways or methods of send/receive huge data continuosly via Csocket.
How to send the big data every interval in a better way. See here.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"You can easily judge the character of a man by how he treats those who can do nothing for him." - James D. Miles
|
|
|
|
|
I rarely suggest a completely different approach, but have you considered ASIO[^](the non-Boost version if you aren't using Boost) and making both the send and receive asynchronous?
|
|
|
|
|
i am working on a movie database and i have output from my file that i want to load into my linked list,no problem opening file but when i run this function it enters junk values in the list and only works if i enter all the strings without spaces or seperated by special characters.
i am reading this from the txt file
SHAWSHANK REDEMPTION 120 5 MYSTERY 2009 ENGLISH 3
THE MATRIX 120 5 MYSTERY 2009 ENGLISH 3
THE MATRIX PART1 120 5 MYSTERY 2009 ENGLISH 3
THE MATRIX PART2 120 5 MYSTERY 2009 ENGLISH 3
this is the code for load text from file into a linkedlist(doubly)
void double_llist::loadfromfile(){
string mname,mgenre,mlanguage;
int mrelease_year,mamount, mrating,mruntime;
ifstream fromfile;
fromfile.open("database.txt");
while (fromfile >> mname >> mgenre >> mlanguage >> mrelease_year>>mamount>>mrating>>mruntime)
{
cout << mname<<mgenre<<mlanguage<<mrelease_year<<mamount<<mrating<<mruntime<<endl;
create_list(mname,mgenre,mlanguage,mrelease_year,mamount,mrating,mruntime);
}
}
the code for doublylinkedlist
void double_llist::create_list(string mname,
string mgenre,
string mlanguage,
int mrelease_year,
int mamount,
int mrating,int mruntime)
{
struct node *s, *temp;
temp = new(struct node);
temp->info=id++;
temp->name = mname;
temp->genre=mgenre;
temp->language=mlanguage;
temp->release_year=mrelease_year;
temp->amount=mamount;
temp->rating=mrating;
temp->runtime=mruntime;
temp->next = NULL;
if (start == NULL)
{
temp->prev = NULL;
start = temp;
}
else
{
s = start;
while (s->next != NULL)
s = s->next;
s->next = temp;
temp->prev = s;
}
}
my class and node struct for linkedlist
#include<iostream>
#include<cstdio>
#include<cstdlib>
#include<string>
#include<fstream>
using namespace std;
struct node
{
int info;
string name;
string genre;
string language;
int release_year;
int amount;
int rating;
int runtime;
struct node *next;
struct node *prev;
static int counter;
}*start;
int id=1;
class double_llist
{
public:
void create_list(string mname,
string mgenre,
string mlanguage,
int mrelease_year,
int mamount,
int mrating,int mruntime);
void delete_element(int value);
void search_element(node *head, int value);
void display_dlist();
void sorting();
void update(node *q ,int value);
int count();
void reverse();
void savetofile();
void loadfromfile();
void revertsort();
double_llist()
{
start = NULL;
}
};
|
|
|
|
|
A couple of minutes with the debugger would have shown you that you cannot enter text like that and expect the system to realise that the film title comprises more than one word. Stream input breaks fields up at white space so in your first case you will have something like:
mname = SHAWSHANK
mgenre = REDEMPTION
mlanguage = 120
mrelease_year = 5
mamount = MYSTERY - which is not an integer
mrating = 2009
mruntime = ENGLISH - also not an integer
the final 3 will now go into the title next time round the loop.
|
|
|
|
|
yes i know that's why i am asking a solution for that.
|
|
|
|
|
You need to split the lines manually so that you can separate the title from the other fields. You could just read the entire line, tokenise it into an array, and then work backwards field by field. Whatever is left at the beginning of the array can be recombined into a single string which is the title.
|
|
|
|
|
The easy solution is to override the read and write methods in your own stream extension.
It is the "standard", "usual" and "prefered" way to fix behaviour of classes or objects with methods and functions that aren't exactly as you desire.
Can I go so far as to suggest MovieNameStream as a possible title for the extension.
In vino veritas
|
|
|
|
|
Farhan_Karim wrote: works if...all the strings...seperated by special characters. Is that not a possibility?
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"You can easily judge the character of a man by how he treats those who can do nothing for him." - James D. Miles
|
|
|
|
|
I have Dev C++ on my system to practice writing C++ code from my C++ Programming for the Absolute Beginner book. So far, the constant repetition and copying have improved my muscle memory. Then, recently, I just noticed that this application could autocomplete my line.
My biggest worry is that this autocomplete thing would make me fail the Computer Science placement exam. I really want to take the placement exam, so then I can get tested into the software development class and skip the introductory CS classes. On the school's website, it says that the placement exam takes place on paper and pencil, so that implies I should know how to write everything down to smallest detail and get it accurate enough so the algorithms would run correctly.
Should beginners use Autocomplete? Maybe it's good practice for a beginner to write code on paper and then transfer the code onto the computer?
|
|
|
|
|
Member KL wrote: Maybe it's good practice for a beginner to write code on paper and then transfer the code onto the computer? That's a good idea for anyone, not just beginners.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"You can easily judge the character of a man by how he treats those who can do nothing for him." - James D. Miles
|
|
|
|
|
That sounds completely impractical... perhaps good for a learner but you'd never get anything done if you were writing things on paper first before transposing the same exact thing onto real code.
|
|
|
|
|
I guess you never had the joy of writing your code in pencil, on fixed width coding sheets. These were then sent to the data prep department to be punched onto Hollerith (80 column) cards, before being submitted to the computer room for compilation. And then you had to wait a couple of hours for the results which showed you had missed a comma on the second line.
|
|
|
|
|
Luckily, no... that would be painful!
I have however worked on FPGAs, that's a similarly slow process of building/routing. You're not writing things on paper but you're definitely waiting for a really long time for synthesis and routing. I was working on these guys a few years ago now but our builds would take about a couple of hours too. You'd be really disappointed when things didn't quite work or you forgot some debug traces.
|
|
|
|
|
It depends how much work your auto-complete does for you. As a learner, you should avoid auto-complete that makes classes and sets of methods for you. If however, all it's doing is completing a word, well... you're learning to program, not to spell... so that little bit of help won't undo what you're learning.
At the end of the day what will make you a better programmer will ultimately be how many hours you spend programming. Practice, practice, practice....
Good luck!
|
|
|
|
|