|
As of today my blog about the development of dtpd, the dtp daemon, has been moved to this very location. Enjoy.
|
|
|
|
|
Threads...
It would seem like my thread library is filled with strange, rarely occuring bugs. It's taking me huge amounts of time trying to find and squash them, so I downloaded boost.thread to see how that thread handling library works. I compiled this simple test:
#include <boost/thread/thread.hpp>
#include <boost/thread/xtime.hpp>
#include <iostream>
void thread_func()
{
}
int main(int argc, char* argv[])
{
boost::thread thrd(thread_func);
thrd.join();
}
It segfaults immediately. Soo.... I have a big problem now, these threading issues must be solved soon, I'm falling behind my schedule each hour I spend on debuging my thread library.
|
|
|
|
|
It would seem like I've found the problem with my thread class, it's my condition class that's not working as it is supposed to. I can't see any obvious solution to this problem, but I might find one if I think about it for a while. I've read and re-read the documentation for posix threads without finding out what's wrong, the problem is however best described as a deadlock and could very well be one.
|
|
|
|
|
The thread problems might be solved now, so I've encountered a new problem... A huge memory leak, which could be caused by alot of things in my library, so all I can do is start to debug again. It's pretty unlikely that it's the threading part of the library that leaks, which is good since I'm getting tired of debuging those files.
|
|
|
|
|
I've narroved the problem down to the select function (my library implementation), it occurs when select spawns more than one thread to select the underlaying sockets. As mentioned before, the memory leak is huge, the size of the process increases with about 15-20 MiB every other second.... It's very confusing.
|
|
|
|
|
The select problem is probably solved now, the problem was however not found in the select function at all, but rather in the shared_ptr<> class. All my problems were caused by a small typo in the constructor of the shared_ptr<> class, which meant that it didn't free the memory that it held at destruction.
I'm currently tinkering a bit with the function, adding some features and will later test it some more. When everything seems to be working I'll move on to work with dtpd, testing some locking-patterns and algorithms for thread safety.
|
|
|
|
|
select isn't causing me any obvious problems now, I've found and killed som other bugs as well, so I'm currently feeling good about my self. I'm going to look over my dtpd design a couple of times more before actually starting to write the code. On top of that I'm also planing to take one last look at my library, I've realized that the library design will be more or less frozen solid when I start to use it in a bigger project.
I've also realized, while writing the test programs for the network classes, that flexibility and power sometimes should be traded against simplicity. The socket class are not as easy to use as they ought to be, sure they are (almost) as powerfull as real, non-wrapped sockets handled by a C interface, but not nearly as easy to use as their equivalences in other libraries. My ssl socket class has to be rewritten, not much, but it's a bit to hard to use it right now, which is not good.
|
|
|
|
|
Ah, progress. I'm finally working on the dtp daemon.
I've started to work on a service class which will make the process that's using it a daemon. Sortof. I'm not sure how it will work in Windows, but it looks promesing for Linux. There's currently a small problem; it crashes at some point by some reason, but I'm debuging it and getting closer to the bug. It would seem like an exception is generated when I try to open a file descriptor to `dev/console'. I've come to suspect that C++ error handling doesn't work well with ::fork().
I'm a bit proud of the syntax of main():
int main(int argc, char* argv[])
{
try
{
zinara::service_core<provide_nothing>::instance().install(argc, argv);
zinara::service_core<provide_nothing>::instance().work();
}
catch(std::exception& fatal_error)
{
...
}
}
The policy for service_core is for the moment provide_nothing, which doesn't provide a user defined service as it should (it actually just dead locks). Needless to say, this will be changed in the future.
|
|
|
|
|
I've written some code that should make service work in windows too now, I haven't tested it though. I removed the class template for service too, I realized that it was unnecessary to keep it since it did nothing.
int main(int argc, char* argv[])
{
try
{
zinara::service::instance().install(argc, argv);
}
catch(std::exception& fatal_error)
{
...
}
}
|
|
|
|
|
Today I got some help from mahogny: a very promesing solution to the deadlock problem that has hung above this project as a sword from a thin line.
I'm currently working with the translation of pseudo code to C++ and adding the new idea to the project.
|
|
|
|
|
I've been testing the queue algorithm tonight, it seems to be working fine. I'm moving on to network part now, it's time to test the improved zinara library in action for the first time, and I can't wait to see how it works. I'm hoping that it will be pretty bug-free, I know that it should be, considering that I've tested it extensively.
|
|
|
|
|
I'm working with the dtpd object dispatching system right now, I would like the compiler to complain if the I try to place a object in the wrong todo queue, but that would seem impossible with my current dispatching system that relies on virtual functions (runtime) to decide what should be placed where.
There might be a way of doing it with templates, so I'm going to look in to that now.
|
|
|
|
|
I've realized that I've neglected this blog for quite some time now. I'm strugling with Software licenses right now, hopefully those problems will be solved within the next ten minutes.
|
|
|
|
|
I've been working with smart pointers today. Some bug fixes and some design changes.
|
|
|
|
|
I haven't really prioritized this blog since I started to keep a changelog file, but here's an update of what has happened since the last post:
I've been working a lot on zinara, the reason is simple; each try I try to use it there's something that isn't working, some huge design flaw which makes the library hard to work with etc. So, these are the changes that I've made to zinara
2007-03-18
* /source/detail/basic_address.hpp
/source/detail/basic_address.hpp:
Bug fixes, get_ip() will now return a correct ip address
* /source/detail/basic_multiplexer.hpp
/source/detail/basic_multiplexer.cpp
/source/detail/select_reactor.hpp
/source/detail/select_reactor.cpp:
Classes completed, ready for testing
* /tests:
Cleaned up, deprecated tests removed.
Comment: Everything seems to be working fine now, the library is installed on my system, seems okay to work with, generates no compile errors and does not seem buggy. I have finally decided how the socket (according to me) is updated, a multiplexer will be used (basically a wrapper around some low-level C function).
Time spent: about 1H.
2007-03-17
* /source/ssl
/source/ssl/detail:
Directories + source code removed due to license mismatch,
the OpenSSL license is incompatible with GNU GPL.
* /source/tls:
Added, zinara will use GNUTLS for encryption from now on.
Missing tls implementation right now. (The SSL classes would
have required extensive re-writing anyway due to the major
design changes in the socket part of the library)
* /source/basic_select_reactor:
Replaced, zinara will use a reactor/multiplexer pattern from now on
* basic_multiplexer
select_reactor:
Added class templates
* /source/detail/basic_acceptor.hpp
/source/detail/basic_acceptor.cpp
/source/detail/basic_connector.hpp
/source/detail/basic_connector.cpp
/source/detail/basic_resolver.hpp
/source/detail/basic_resolver.cpp:
Minor bug fixes
* /source/fd_set_adaptor.hpp
/source/fd_set_adaptor.cpp:
Minor design changes
Comment: Unfortunately I've discovered that OpenSSL's License is incompatible with GNU GPL (I do not know why, it's not easy to tell without being a lawyer I guess -> http://www.gnu.org/licenses/license-list.html#OpenSSL ). This means that I will change my secure socket classes so that they use GNUTLS instead, which is released under GNU GPL.
GNUTLS is not yet implemented and I do not think that I'll find the time to write the necessary classes before the first revision has to be turned in either.
Time spent: Including license research about 1 & ½H
2007-03-16
* /source/detail/basic_socket.hpp
/source/detail/basic_socket.cpp
/source/detail/basic_stream_socket.hpp
/source/detail/basic_stream_socket.cpp
/source/detail/basic_datagram_socket.hpp
/source/detail/basic_datagram_socket.cpp:
Major design changes enforced, changing of template arguments,
re-writing of communication policies, etc. Updated doc tags.
* /source/detail/basic_acceptor.hpp
/source/detail/basic_acceptor.cpp
/source/detail/basic_connector.hpp
/source/detail/basic_connector.cpp
/source/detail/basic_resolver.hpp
/source/detail/basic_resolver.cpp:
Updated to cope with the new design of the socket template classes.
* configure.in:
Added dirty hack to add header guards to the config.hpp file
Comment: The old socket classes worked well in my mind and on GNU/Linux with gcc/g++, but they turned out to be hard to port to other Operating systems and compilers. I have therefore changed the design extensivly which I hope will make it easier to port the library and thereby also dtpd to other systems and compilers.
The new classes are poorly documented and writing new doc tags are not a priority right now, I will probably be able to salvage at least halv of the documentation from the old code when I feel there's time.
Time spent: About 2 & ½H.
2007-03-15
* configure.in:
Minor changes
* /tests:
Small testprograms for testing socket for both compile-time errors
and run time errors added. (Some older test moved to this directory
as well)
* /source:
Renaming of some files, mostly preparations for the planned design
change of socket classes.
Time spent: About 30-50 min
2007-03-10 -> 2007-03-14
I used about 1H a day to come up with a new socket design and implementing it.
Time spent: Max 6H
2007-03-09
* /source/detail/basic_select_reactor.hpp
/source/detail/basic_select_reactor.cpp:
Bug-fixes, mainly thread related
* /source/function.hpp:
Bug-fix
* /source/detail/algorithm.hpp
/source/detail/algorithm.cpp:
Bug-fix
* /source/detail/shared_ptr.hpp
/source/detail/shared_ptr.cpp
/source/detail/shared_array.hpp
/source/detail/shared_array.cpp:
Thread safety fixed, bugs removed, race conditions eliminated.
Comment: I discovered that my thread safe smart pointers in fact wan't thread safe, there were several lurky race conditions embedded in the code. I hade to re-write all parts of the classes containing critical sections, but the bugs should (hopefully) be fixed now.
Time spent: About 2H.
2007-03-08
* /source/interrupter.hpp
* /source/interrupter.cpp:
Formerly known as select_interrupter, minor design changes + renaming
* /source/timeval.hpp
* /source/timeval.cpp:
Formerly known as select_timeout, minor design changes + renaming
Comment: Some design changes made, nothing fancy.
Time spent: About 30-40 min.
2007-03-07
Minor bug-fixes
* /source/service.hpp
/source/service.cpp:
Added psuedo-support for windows services, needs to be tested and will
probably not compile right away. (Portability issue)
Comment: Added code that will port daemonize(...) to windows, I wrote it using MSDN, to this date it hasn't even been compiled, so it probably contains some minor errors, they shouldn't be hard to fix when I'm forced to port dtpd and zinara to windows though.
Time spent: About 50 min
2007-03-05
* /source/io_control.hpp
/source/socket_option.hpp:
Classes added to make IO control of the socket as well as setting and
retrieving socket options easier.
* /source/detail/basic_socket.hpp
/source/detail/basic_socket.cpp:
Changed IO control and set/get socket option functions, added typedefs.
Comment: Performing I/O control on the sockets and geting and seting socket options was way to hard, I look around a bit and found a brilliant solution in asio. I re-wrote a couple of things and added it to zinara.
Time spent: About 1H.
2007-03-04
* /source/detail/basic_stream_buffer.hpp
/source/detail/basic_stream_buffer.hpp:
Bug-fixes: discard algorithm changed, several small typos fixed, segmentation
fault fixed and memory leaks removed.
Comment: The buffer wasn't working at all since I had added an allocator policy, so I had to spend some debuging it and fixing the problems.
Time spent: About 2H. (Segfaults are tricky beasts)
2007-03-04
* /source/fd_set_adaptor.cpp:
Minor changes
* /source/detail/debug.cpp:
Minor changes
Comment: Just some tinkering in the actual source code, I did however spend some time contemplating on some problems with the dtpd design and managed to solve one consering relationships between objects and workers.
Time spent: About 1H.
And finally, what have I been doing today?
When finally turning back to the dtpd code I've realized that it some major re-writing, the design isn't good enough. So, I've thrown away the latets design and started on a new, some code can be salvaged.
|
|
|
|
|
Testing a new design for dtpd, it seems to be working out. There's some threading issues that has to be worked out, but other than that it should probably work.
Time spent: About 1H.
|
|
|
|
|
I'm currently documenting my code, both zinara and dtpd. It will probably keep me occupied for quite some time.
Time spent today: About 1 & ½ H
Edit: I'm going to start writing my first draft for my report tomorrow.
|
|
|
|
|
So, I've started to write my report... Just scratched a few things down to get a base to write from. I've been commenting files too, reading about GNUTLS and I've collected some information about mrsw mutex (I have no idea how I am suppossed to turn that into plural, so I'm not even going to try) which are needed in dtpd, from the code I've found this far it should be fairly easy to implement.
Time spent: About 1H
|
|
|
|
|
I'm currently writing my report and the resource lock management system for dtpd, I'm planning to work with that today and spent about 1H researching thread concurency problems yesterday.
|
|
|
|
|
I've added windows support for the condition class now, it will probably be error prone and buggy due to the fact that I had to use semaphores. Conditions is a feature that's apperently only available in Windows Vista, I might write support for this some day.
When the condition class is working as it should again, I'm going to use it to write the swmr class (Single writer, multiple readers), it should be fairly simple. The swmr class will then be used to implement the resource lock management system in dtdp. The resource lock management system will be used to make sure that resources are protected with minimal risk of deadlocks in the process.
I'm having some troubles getting started with my report however, it's obvious that the crib sheet handed out by the teachers isn't meant to be used with my kind of project. I should be able to use with some minor modifications though.
Anyway... Time to sleep.
And, time spent today: About 2 & ½H.
|
|
|
|
|
I've been writing on my report today, wrote the first page yesterday (but forgot to write about here). I'm almost done with the result part and will finish the discussion part tomorrow. When that's done I'll probably check trough the code and see if there's something that should be fixed before it's turned in. If not, I'll try to make zinara compile with MSVC++, if it doesn't generate to many, all to b****y error messages that is.
Time spent: 2H.
|
|
|
|
|
I've finished my report and uploaded the source code. So... I'm going to continue to work on the project now.
Time spent yesterday and today: 2 H.
|
|
|
|
|
Soooo....
I got some help reviewing the report, it had some flaws which now have been corrected.
I'm also working on the TLS support to zinara, I've been able to come up with a design that seems to work after reading about one half of the GNUTLS manual.
Time spent: About 3 H (It's a big manual)
|
|
|
|
|
I haven't been working on dtpd the last couple of days, but I have been reading the reports assigned to me. They all look good except some minor semantic and syntactic errors. Comming up with 9 good questions/things to improve could become difficult, but probably not impossible.
Time spent: 55 min.
|
|
|
|
|
I've decided that I don't like the autoconf/automake tools. They might be (more or less) *NIX standard, but they're (more or less) horrible too. I'm therefore switching build system for Zinara and DTPD to SCons which seems a lot nicer. It will take some time to do this, but I'm hoping that it will save me (and others) plenty of time in the future.
Time spent so far: 2H.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
