|
did you try the (free) log4net library, has a LOT of features built-in in it....
Jonathan de Halleux.
|
|
|
|
|
Hi Jonathan,
I know Log4j because I used to program in Java. And I know Log4Net. This is a great product, far more complicated and heavy than this little solution. I think it is really well suited for websites (something this log can't do) and professionnal projects where you can enjoy the SMTP and UDP features of Log4Net for example.
I use this little log solution for my little projects. In that case I don't need the Log4Net artillery.
For a very critical website or huge project I will may sign for Log4Net, but you should also look at the Tracelistener capabilities built-in the .NET framework. Something we could maybe argue against Log4Net is that it is originally a Java port and doesn't rely on all the available features of .NET. Creating another full featured logging system very .NET oriented will be interresting, even if Log4Net is already up for the job. But if you have some time to spare...
Also take a look on the messages at the bottom of this page. The last one, I believe, provides 2 links to other solutions.
Cheers,
R. LOPES
Just programmer.
|
|
|
|
|
How about sending this info to another machine via SOAP? So, the developer can get a instant first hand info for debugging.
|
|
|
|
|
Like the way the IE 6 does it.
|
|
|
|
|
Hello,
Yes why not ? But it would be a lot more complicated. In particular if you have to check if the internet or LAN connection is still alive when you want to send an instant error message.
It could be the purpose of a new article.
Regards,
R. LOPES
Just programmer.
|
|
|
|
|
One of the main problems I see with logging to XML is the "well-formedness" constraint.
Because XML requires that there be only one root element, it is difficult to simply append log entries to the XML document.
You get around this by wrapping everything inside an <error> element, but as you noted, there was a bug that prevented the ending tag from being written.
With your approach, I would have to create a new document each time the application started (instead of perhaps logging by day).
I think the whole single root element requirement in XML is arbitrary. I wish that it was not necessary. They could have simply used an implied root element named "/". This would reference the document node similar to XPath/XSLT. But I digress...
Although I like the idea of using XML (well structured, widely used format, etc.) it is difficult to write in a "streamed" sequential manner when there really is no "well-defined" end.
-- Kiliman
|
|
|
|
|
Hello,
Thank you for the feedback.
I agree with you about the appending problem but solutions may exist according to a friend. I will look into them.
Concerning the bug that prevented the ending tag from being written, I modified this class a little bit as described in the next message board topic of this article to solve it. I don't have any problem for the time being.
I use this class for a 24/7 application and I create a brand new error log file each day. My program is heavily multi-threaded, so the only thing I have to take care of (at midnight) is to pause all threads before closing the old error log and creating a new one. It works pretty nicely .
Like you I really believe in the benefits of using XML files. I use them for configurations, for error logging, application logging... These files can be read by a wide variety of tools and loaded into a database with ease. I also created specific tools and web pages to create XML configuration files or display/search the XML logs just by using the XMLWriter and XMLReader classes provided in the .NET API. The only drawback is the size of the files But I am looking into specific compression models for XML files (better compression because you know it is a XML file). Of course you can not compress a XML stream but you can do it when closing the file. I solve the size problem but I lose the benefit of making the file readable by a wide range of tools. So I have to think twice. For my tools I just need to add decompression before working with the file and add compression when writing an updated version of the file.
Regards,
R. LOPES
Just programmer.
|
|
|
|
|
Delimited text can be written by even more things. Good XML parsers don't grow on trees. XML isn't the answer for everything. The problems you presented regarding the root node and appending logs is a real problem; that's why every other continuous logging system uses text or delimited text files. If you need to display this in a weblog or something, it's easy to parse. Heck, PERL is a perfect languages for this and you could write-out XML either plainly or using a DOM.
XML is great, but it won't solve all your problems. In fact, in this case, it's created more problems for you. Maybe that should be your first clue this isn't such a good idea, although your first clue should've been that no one else does this (and for a good reason).
"Well, I wouldn't say I've been missing it, Bob." - Peter Gibbons
|
|
|
|
|
Hello,
I disagree with you.
Appending is not a big challenge. Solutions exist. Ok you will not do it with the class presented in this article but you can do it.
You make a big case of text and delimited text files. But what is XML, if not a structured text file ?
XML is easy to parse, because you have a HUGE range of classes and tools to do it. When using XML, you get also the opportunity to use things like XLS and XLST to produce readable reports or present data in many different ways. And everything is relying on standard, not on home-made solutions for text files. PERL is great and I used it to parse text stream, but C# is nice .
Of course XML is not the answer to everything. Again, if you need speed or tiny file size, you should look at something else.
In my case, XML is not creating more problems, actually it helped me a lot and opened a wide range of possibilities. You can also trust me if I say that others have done this kind of solution before me and are happy with it.
In particular, I know a big company which monitors people Internet usage (they agreed on) and stores locally crawled URL as a XML stream in realtime.
This solution is pretty close to logging error in XML, no ? And this solution also appends informations in XML files.
You don't want to use XML, so don't use it. But in my business, producing XML files is a big plus.
Regards,
R. LOPES
Just programmer.
|
|
|
|
|
Logging should be one of the fastest, most unobtrusive operations in your solution. Since you don't implement it on a separate thread as many solutions do, all other operations are blocked until your log function is finished.
Besides, delimited text files don't have to be a home-brewed solution. Many "standards" currently exist for dealing with named fields and what not.
Also, if you are logging to XML files and your logs tend to be big, you do *not* want to use XSLT. This is a completely inefficient process. For this purpose SAX was developed. It's a top-down, forward-only, event driven way of reading large XML documents.
The point is that performance is almost everything in business apps these days. The world is too hell-bent on XML and tries to solve all its problems with it (and most people like my boss who wanted us to replace an entire, very large database with XML - and lost the battle - really don't know squat about it). Knowing your technology and the tools to work with it are important. Logging should be fast and generating reports should be secondary to the primary function of your application, so an extra step to produce XML is appropriate in lieu of hampering your performance.
"Well, I wouldn't say I've been missing it, Bob." - Peter Gibbons
|
|
|
|
|
Actually, there is one corolarry to this: if you're using a logger to output XML as text, there's not much wrong with this, but using serialization or the DOM to do so is crazy. You do realize that when using XML classes and parsers that the *entire* document is read into memory in order for the tree to be created. Again, this is why SAX was developed, so that the entire tree didn't have to be read.
"Well, I wouldn't say I've been missing it, Bob." - Peter Gibbons
|
|
|
|
|
Hello again,
You are right about the memory requirements and DOM .
I often used SAX instead of the DOM in my project just because of these issues. Most of the time I rely on SAX to read and parse my XML configuration files and sometimes I admit that I write XML like text instead of using DOM functions. But still, you can use XML and find the best solution to read and produce XML files depending on your constraints.
Nevertheless, thank you for all your remarks. I think I'm more a XML advocate than you but I totally respect your point of view and arguments.
Regards,
R. LOPES
Just programmer.
|
|
|
|
|
Whoa whoa whoa, hold on there. I've been researching and developing with XML since it was still a wet-dream to most people. I truly believe in XML and have done a lot of work with it. But a good advocate knows when his or her subject isn't the best solution. XML is great for may things and does solve many problems, but it doesn't solve all of them. Platform-agnostic solutions are difficult at best without a standard capable of expansion like XML. Many applications like Office (currently, when you save as HTML it's actually saved as XHTML, but future versions will apparently use XML for native formats if I gather correctly), StarOffice, and many others use XML and would allow for any application to read-in their files on any platform. Only a couple years ago this was a very difficult thing to accomplish and always involved having to develop such documents or applications with little help from frameworks.
However, it still doesn't work for everything. Take, for example, the current solution I architected for work that relies on remoting on a system that must be extremely scalable since it's an Internet-based Windows Forms application with a remote DB. With remoting (even exposed as a Web Service for firewall'd clients) we can either use a SOAP formatter or binary formatter. With SOAP, a DOM must be created in memory to serialize data correctly, then transfered across the wire in a Unicode format (so 2 bytes per character, already twice the size to transfer), then reconstructed so the server can deserialize using the DOM. A lot of memory, CPU time, and throughput are wasted. If we use a binary formatter, the size is consistent and serialization is much faster since a DOM is not used on either side. There are many other examples where XML may not be the right choice. That's the only point I'm trying to make here.
If you are just writing XML as text, so be it. I was just warning about the implications of using the DOM because - as I said before - logging must be faster and having a potentially large tree in memory (most of which is probably in the pagefile) that must be parsed just to add a new "record".
"Well, I wouldn't say I've been missing it, Bob." - Peter Gibbons
|
|
|
|
|
Back,
If I had to develop the same kind of app over the Internet, I would do something similar I think. I totally agree with you and your binary approach is the best solution to handle the bandwith, size and speed issues of transfers over the Internet. Did you try to add some compression in the process, if it is possible ?
You made your point about the use of XML and I'm also convinced it is not the best solution for everything. I believe our views are just a little bit different for this particular logging case .
Bye,
R. LOPES
Just programmer.
|
|
|
|
|
We'll implement simple RLE compression for this first version and implement something closer to zlib later. Basically, we have to override the formatter sink and just run the bytes through a compressor. There are several compression libraries out there, but since one needs to sit on the client, I don't want throughput wasted by excessive functionality we don't need. We just need the compression algorithm, not file support, stream support, MIME encoded base64 support, etc. etc. By simply overriding the binary formatter, we can hypothetically call the base classes primary methods then just run the return through an algorithm. Almost like inserting another sink *after* the formatter.
"Well, I wouldn't say I've been missing it, Bob." - Peter Gibbons
|
|
|
|
|
Yes,
It sounds very good. Your project looks very interresting indeed.
If you want some inspiration, here is a nice link I crawled a lot about compression (maybe you already knows it):
http://datacompression.info/[^]
Good luck with your project,
R. LOPES
Just programmer.
|
|
|
|
|
No I didn't. It's a nice site and made its way into my bookmarks. It's nice to look through the documentation of the algorithms and not have to pour through tons of code. Thanks!
"Well, I wouldn't say I've been missing it, Bob." - Peter Gibbons
|
|
|
|
|
Hello,
Yes logging must be fast, but it also depends on the amount of informations you log. If you log not so often, the logging process will not be an issue. Yes this particular solution (very basic indeed) is not running on a separate thread and may raise some concerns, but multi-threading, just like XML is not always the best solution. Sometimes keeping the number of threads low helps helps saving some CPU cycles wasted by thread synchronisation and other mechanisms. When logging or whatsoever in a thread, it is the job of the programmer to avoid a long blocking operation.
Speaking about performance, it is everything in the business, but it depends of what kind of app you are developing.
Performance is a more bigger subject than just XML. Ok you can save CPU cycles with a efficient homemade logging system, but maybe you waste somewhere else because you believe you should use this specific utility class or library. And if we talk about performance, we could argue that C# gives worse results than C++, assembly code is faster than C++ and so on...
We have to be reasonable to find what the business is really looking for: a good balance between performance, reduced time to market and ease of maintenance/update.
I agree with you about the stupid idea of using XML instead of a very large database. It makes very much sense. It is one example of being reasonable.
Regards,
R. LOPES
Just programmer.
|
|
|
|
|
I re-wrote the class to append to the xml document instead of always creating a new one. Below is the code.
<br />
using System;<br />
using System.IO;<br />
using System.Xml;<br />
using System.Text;<br />
using System.Diagnostics;<br />
using System.Runtime.CompilerServices;<br />
<br />
namespace ErrorLogger.Utility<br />
{<br />
public class ErrorLogWriter:TextWriter<br />
{<br />
<br />
<br />
private bool Disposed;<br />
private XmlDocument doc;<br />
private string FileLocation;<br />
<br />
public ErrorLogWriter(string FilePath)<br />
{<br />
FileLocation = FilePath;<br />
<br />
Disposed=false;<br />
<br />
doc = new XmlDocument(); <br />
FileInfo fiMXF = new FileInfo(FilePath);<br />
if ( fiMXF.Exists )<br />
{<br />
<br />
doc.Load(FilePath);<br />
}<br />
else<br />
{<br />
XmlDeclaration xmldecl;<br />
xmldecl = doc.CreateXmlDeclaration("1.0",null,null);<br />
XmlElement oRoot = doc.CreateElement("error");<br />
doc.AppendChild(xmldecl);<br />
doc.AppendChild(oRoot);<br />
}<br />
}<br />
<br />
~ErrorLogWriter()<br />
{<br />
Dispose(false);<br />
}<br />
<br />
<br />
protected override void Dispose(bool Disposing)<br />
{<br />
if(!Disposed)<br />
{<br />
if(Disposing)<br />
{<br />
}<br />
<br />
doc.Save(FileLocation); <br />
<br />
<br />
doc=null;<br />
<br />
<br />
Disposed=true;<br />
base.Dispose(Disposing);<br />
}<br />
}<br />
<br />
public override void Close()<br />
{<br />
<br />
<br />
<br />
Dispose(true);<br />
GC.SuppressFinalize(this);<br />
}<br />
<br />
public override Encoding Encoding<br />
{<br />
get<br />
{<br />
return(Encoding.Unicode);<br />
}<br />
}<br />
<br />
[MethodImpl(MethodImplOptions.Synchronized)]<br />
public override void WriteLine(string Txt)<br />
{<br />
XmlNode oRoot = doc.DocumentElement;
<br />
XmlElement Event = doc.CreateElement("event");<br />
<br />
XmlElement Time = doc.CreateElement("time");<br />
XmlElement Class = doc.CreateElement("class");<br />
XmlElement Method = doc.CreateElement("method");<br />
XmlElement Text = doc.CreateElement("text");<br />
<br />
<br />
XmlText TimeText= doc.CreateTextNode(DateTime.Now.ToString());<br />
XmlText ClassText= doc.CreateTextNode(new StackTrace().GetFrame(2).GetMethod().ReflectedType.FullName);<br />
XmlText MethodText= doc.CreateTextNode(new StackTrace().GetFrame(2).GetMethod().ToString());<br />
XmlText ValueText= doc.CreateTextNode(Txt);<br />
<br />
<br />
Time.AppendChild(TimeText);<br />
Event.AppendChild(Time);<br />
<br />
Class.AppendChild(ClassText);<br />
Event.AppendChild(Class);<br />
<br />
Method.AppendChild(MethodText);<br />
Event.AppendChild(Method);<br />
<br />
Text.AppendChild(ValueText);<br />
Event.AppendChild(Text);<br />
oRoot.AppendChild(Event);<br />
}<br />
}<br />
}<br />
|
|
|
|
|
Hello,
Thanks ! This is a great addition and an answer to people asking for a long time how to append in an existing XML file .
Have a nice day,
R. LOPES
Just programmer.
|
|
|
|
|
I think It very good but...memory and performace?
|
|
|
|
|
I have no idea what is the problem of xml.
1. Use XmlRreader, then we would not need to regard the 'correct' format of xml. So even error format of xml, it can also be parsed.
2. Use File.Open(xxx,append) => Stream.Write() to append log, why use xmlXXX to write xml?? we just need to output some LIKE xml format, for example:
Stream.Write(<error>adfsdf).
So, xml log is not a problem.
|
|
|
|
|
Interesting, mainly because I have implemented pretty much the same thing. Your implementation has a problem that I am trying to solve. That is if the application using ErrorLogger crashes without calling Err.Close the file doesn't have an end tag and thus cannot be read without manually adding the tag. I updated your newFunction as shown below. Then I placed breakpoints in the error logger destructor and it is never hit, which is strange. In my case I hit my destructor but the writer is already closed and I can't wirte the end tag. Do you have any thoughts on this?
private static void newFunction()
{
Console.Error.WriteLine("Look ! I am inside a function !");
int j = 0;
int i = 7/j; //error divide by zero
}
|
|
|
|
|
Hello,
Yes Err.Close never get called in this case.
The problem is quite simple to solve and this is an implementation error I recently fixed when using using the class in my project.
According to Microsoft guidelines, the Close() method should only call the Dispose() method. The Close() method is just meant for the programmer comfort.
That means you should move all the closing tags stuff of the Close() method to the Dispose(bool) method. In that case, when the program will crash and the garbage collector will call the destructor, the tags and file will be properly closed.
Here is an upadte:
<br />
protected override void Dispose(bool Disposing){<br />
if(!Disposed){<br />
if(Disposing){<br />
if(Writer!=null){<br />
Writer.WriteEndElement();<br />
Writer.WriteEndDocument();<br />
Writer.Flush();<br />
Writer.Close();<br />
}<br />
}<br />
Disposed=true;<br />
base.Dispose(Disposing);<br />
}<br />
}<br />
<br />
public override void Close(){<br />
Dispose(true);<br />
GC.SuppressFinalize(this);<br />
}<br />
This solution works for me.
Regards,
R. LOPES
Just programmer.
|
|
|
|
|
Example output:
<?xml version="1.0" encoding="utf-16"?>
<error>
<event>
<time>15:09:28</time>
<class>ErrorLogger.Test</class>
<method>Void Main(System.String[])</method>
<text>Here is my first error !</text>
</event>
<event>
<time>15:09:28</time>
<class>ErrorLogger.Test</class>
<method>Void Main(System.String[])</method>
<text>I should write this inside a catch(exception) statement...</text>
</event>
<event>
<time>15:09:28</time>
<class>ErrorLogger.Test</class>
<method>Void newFunction()</method>
<text>Look ! I am inside a function !</text>
</event>
</error>
Happy coding,
R. LOPES
Just programmer.
|
|
|
|
|