Re: Validating a "Text" file - C# Discussion Boards

musefan20-Jan-09 4:23

musefan

20-Jan-09 4:23

What do you mean 'is binary'? everything is stored in binary remember.

Do you mean that its a file containing only the characters '1' and/or '0'?

If so do you just want to check if the file contains only those 2 characters?

EliottA20-Jan-09 4:24

20-Jan-09 4:24

I think he means an executable, but that was my assumption, I may be stupid.

musefan20-Jan-09 4:36

musefan

20-Jan-09 4:36

No time to wait for replies lol. If you want some code that works for my understanding of your question then read on. otherwise, if i misunderstood your needs ignore the rest. but it may help others who pull up your post.

System.IO.StreamReader sr = new System.IO.StreamReader("C:\\TargetFile.txt");

int b = sr.BaseStream.ReadByte();

while(b != -1)
{
char c = (char)b;

if(c != '0' && c != '1')
//DO ERROR HANDLING HERE

b = sr.BaseStream.ReadByte();
}

sr.Close();

Of course you could just test the byte against ASCII equivalent without char conversion

And remember you may need to handler other characters such as 'new lines' etc.

EliottA20-Jan-09 4:23

20-Jan-09 4:23

Loop through and check for non printable characters?

Simplest and easiest way; check file extension? If someone renames a .exe to a .txt, checking for non printing characters in the file would be your easiest way. Another simple safeguard is checking filesize and restricting large files.

dwolver20-Jan-09 4:32

dwolver

20-Jan-09 4:32

Thats basically what I need to do. Make sure that the file the user is uploading is not a renamed file, too big etc or especially not a script. I was hoping there would be some .Net functionality to determine a file type.

So I'll read it into a stream and check for ascii unless I hear there is another way.

EliottA20-Jan-09 4:35

20-Jan-09 4:35

FileInfo Class[^] can get you some generic info, but you'll probably have to go deeper then that.

musefan20-Jan-09 4:39

musefan

20-Jan-09 4:39

you would need to identify the file headers for that kind of thing, which are different for each filetype so not an easy task

Re: Validating a "Text" file [modified]

Guffa20-Jan-09 6:11

Guffa

20-Jan-09 6:11

Perhaps you should look for what you don't accept.

There are a lot of file types that contain plain text, so if someone renames one of those you could not determine that by just checking if the file contains text. This includes script files, so to detect one of those you would have to look what's in the text.

A lot of binary file formats have some identifying bytes at the beginning, you could check for some of the common ones.

Despite everything, the person most likely to be fooling you next is yourself.

Luc Pattyn20-Jan-09 4:38

Luc Pattyn

20-Jan-09 4:38

Hi,

This is what I do when I need to open a file that may or may not be text, and my app can show either text or a hex dump:
- read its first 1K bytes (less if smaller);
- check first 4 bytes for any of the Unicode/UTF8 markers; if found, it is assumed to be text; if not:
- assume it is code page 1252 and calculate the percentage of printable characters;
that would be anything in [0x20,0x7E] and then some. If percentage sufficiently high, I open
as text. Doing so allows for the odd strange character, transmission error, whatever.
All the above is binary operations, there is no string, no char and no Encoding class involved.

BTW: I call the above recognizing a text file, it does not validate anything.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Good riddance W

modified on Friday, June 10, 2011 12:09 PM

dwolver20-Jan-09 4:42

dwolver

20-Jan-09 4:42

Luc, thanks. That sounds promising. Would you be able to provide some code for and example
Dale

Luc Pattyn20-Jan-09 5:09

Luc Pattyn

20-Jan-09 5:09

Hi,

I don't do examples normally, I'll make an exception this once, without warranties, without
service, this is old code, to be taken as is; I use it on a daily basis though.
Class LP_File will not be provided, as far as this method is involved
it just encapsulates simple File methods.

//----------------------------------------------------------------------------------
// Count non-printable characters
// accepts printable characters, plus CR, LF and HTAB
// plus everything in accept string
// return false iff file contains too many non-printable characters
// UTF-8 files (starting with 0xEFBBBF) are always assumed printable.
// Unicode files (starting with 0xFFFE or 0xFEFF) are always assumed printable.
//----------------------------------------------------------------------------------
public static bool IsPrintableFile(string filename, byte[] accept, out long nonPrintables) {
	bool printable=true;
	bool firstRead=true;
	long chars=0;
	int logged=0;
	nonPrintables=0;
	if (accept==null) accept=new byte[0];
	using (LP_File f=new LP_File(filename)) {
		try {
			f.OpenBinaryReader();
			byte[] buffer=new byte[512];
			for ( ; ; ) {
				int len=f.ReadBinary(buffer);
				if (len<=0) break;
				if (firstRead) {
					firstRead=false;
					if (len>=2) {
						byte c0=buffer[0];
						byte c1=buffer[1];
						if (c0==0xFF && c1==0xFE) return true;
						if (c0==0xFE && c1==0xFF) return true;
						if (len>=3) {
							byte c2=buffer[2];
							if (c0==0xEF && c1==0xBB && c2==0xBF) return true;
						}
					}
				}
				for (int j=0 ; j< len ; j++) {
					byte c=buffer[j];
					chars++;
					if (c>=0x20 && c<=0x7F) continue;
					if (c=='\t' || c==0x0A || c==0x0D) continue;
					bool acceptable=false;
					for (int i=0 ; i< accept.Length ; i++) {
						if (c==accept[i]) acceptable=true;
					}
					if (!acceptable) {
						logged++;
						if (logged<20) {
							int i=(int)c;
							Console.WriteLine("Bad character ("+i.ToString("X2")+
								") in "+filename);
						}
						nonPrintables++;
					}
				}
			}
			f.CloseBinaryReader();
			if (logged!=0) Console.WriteLine(logged+" bad chars");
		} catch (Exception exc) {
			env.error("IO error on "+filename+": "+exc);
		}
	}
	if (nonPrintables>(chars/64)) printable=false;
	return printable;
}

I wish tabs were less than 8 positions on CP!
Nowadays I probably would add a few more printable chars such as copyright, euro, and
e with some accents.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Good riddance W

EliottA20-Jan-09 5:53

20-Jan-09 5:53

Nice Snippet, Going to insert it into my archives for referencing

PIEBALDconsult21-Jan-09 5:58

PIEBALDconsult

21-Jan-09 5:58

Try indenting with SPACEs. Big Grin | :-D

Luc Pattyn21-Jan-09 6:10

Luc Pattyn

21-Jan-09 6:10

PIEBALDconsult wrote:
Try indenting with SPACEs

Nooo way. All those spaces would take too much space on disk as you recently admitted yourself. Frown | :(

Luc Pattyn [Forum Guidelines] [My Articles]

Good riddance W

[newbie] 'System.Data.SqlDbType.BigInt' is a 'field' but is used like a 'type'

PIEBALDconsult21-Jan-09 6:51

PIEBALDconsult

21-Jan-09 6:51

But it's Chris' disk, not yours.

jon-8020-Jan-09 3:51

jon-80

20-Jan-09 3:51

I would like to update the field of a database with the maximum value + 1. The following code doesn't compile because I get an error, so I must be doing something wrong.

Confused | :confused:

//@Task_ref = MAX(Task_ref from database) + 1
string _getMaxTaskRef = @"SELECT MAX([task_ref])
FROM [teamwiki].[dbo].[task]";
SqlCommand cmdGetMaxTaskRef = new SqlCommand(_getMaxTaskRef, conAppDB);
cmdGetMaxTaskRef.ExecuteScalar();

if (_getMaxTaskRef != null)
{

SqlDbType.BigInt _nextRef = (SqlDbType.BigInt) _getMaxTaskRef.
+ 1;
Error:
Error 1 'System.Data.SqlDbType.BigInt' is a 'field' but is used like a 'type' c:\inetpub\wwwroot\TaskEntry\TaskEntry.aspx.cs 100 31 http://localhost/TaskEntry/

Jon

Re: [newbie] 'System.Data.SqlDbType.BigInt' is a 'field' but is used like a 'type'

Christian Graus20-Jan-09 4:09

Christian Graus

20-Jan-09 4:09

The SQL types are only for passing in to DBs. you'll want to cast this to int or long instead.

Christian Graus

Driven to the arms of OSX by Vista.

measure performance of function

George_George20-Jan-09 1:58

George_George

20-Jan-09 1:58

Hello everyone,

I have a managed console application which invokes both managed DLL and native COM (by using COM interop). I am using VSTS 2008 and Windows Server 2003 x64 edition.

My question is, I want to measure a specifc function call performance, e.g. from calling a function of managed console application, to managed DLL and to native COM. I also want to get the execution elapsed time for each function call stack from the top managed code level to bottom native level.

Are there any tools or solutions? It is great if I only get the specific function call performance for each stack, overall performance is not needed.

thanks in advance,
George

Dave Kreskowiak20-Jan-09 2:05

Dave Kreskowiak

20-Jan-09 2:05

You'll need a profiling tool to do this. Compuware has a good one, and Red-Gate has its ANTS profiler. There are others, but those two I can recommend.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

George_George20-Jan-09 2:46

George_George

20-Jan-09 2:46

Thanks Dave!

1.

"Compuware has a good one" -- what is the name? I need the name to do some search. Smile | :)

2.

No .Net or VSTS built-in tool?

regards,
George

Dave Kreskowiak20-Jan-09 5:39

Dave Kreskowiak

20-Jan-09 5:39

George_George wrote:
what is the name? I need the name to do some search

Oops. DevPartner Profiler.

George_George wrote:
No .Net or VSTS built-in tool?

Nope. Maybe in VS2010, though I haven't poked around in it very much.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

SeMartens20-Jan-09 2:10

SeMartens

20-Jan-09 2:10

Good one is also JetBrains dotTrace.

Regards
Sebastian

It's not a bug, it's a feature!

Check out my CodeProject article Permission-by-aspect.

Me in Softwareland.

George_George20-Jan-09 2:47

George_George

20-Jan-09 2:47

It is not free?

regards,
George