Design and Architecture

7-Aug-07 4:52

I'm designing a small, off-site backup service for some existing clients. I'm wondering about how to go about managing the transfer and storage of backup sets. One client may have more than one backup file, from more than one source, that they elect to upload to my service daily. Two of the initial design issues are:

1. It seems more workable if I bundle all backup files into a single 'batch' file, using an archive tool.
2. I will probably use FTP and thus need to encrypt backup batches before upload. I would prefer to not decrypt files for storage, but rather only on retrieval, enhancing security. Clients will be provided with the decryption tool to ensure against the even of my service not being available for retrieval.

I would appreciate comment on these two strategies as well as any more general comment.

Last modified: 51mins after originally posted --

"Once in Africa I lost the corkscrew and we were forced to live off food and water for weeks." - Ernest Hemingway

My New Blog

led mike7-Aug-07 5:37

7-Aug-07 5:37

I read that twice and failed to find any question

Luc Pattyn7-Aug-07 5:41

Luc Pattyn

7-Aug-07 5:41

I guess that's why it was not marked as a question, but as a "General" message.

If anything it is quite a pre-anouncement.

Luc Pattyn [Forum Guidelines] [My Articles]

this weeks tips:
- make Visual display line numbers: Tools/Options/TextEditor/AllLanguages/General
- show exceptions with ToString() to see all information
- before you ask a question here, search CodeProject, then Google

Brady Kelly7-Aug-07 5:45

7-Aug-07 5:45

Sorry, it's modified now. I just wanted comments on my two 'question points'. The appropriateness of each of those is a 'question point' to me.

"Once in Africa I lost the corkscrew and we were forced to live off food and water for weeks." - Ernest Hemingway

My New Blog

led mike7-Aug-07 6:02

7-Aug-07 6:02

Still it is unclear what you are actually doing. Are you developing your own software or putting together this "Service" using available software? If the first, why? Is there something lacking in available software to create this type of service? Is it to expensive? If so how much... considering your time and effort to develop test and maintain your own code?

Brady Kelly7-Aug-07 6:14

7-Aug-07 6:14

I'm developing the software responsible for managing the collection and transfer of the files to an existing host. I'm doing this for someone else who wishes to provide the service to clients who have requested it. It is his wish to develop a small, 'in-house' system to provide the service.

"Once in Africa I lost the corkscrew and we were forced to live off food and water for weeks." - Ernest Hemingway

My New Blog

led mike7-Aug-07 6:34

7-Aug-07 6:34

Brady Kelly wrote:
It is his wish

Ok so it is someone else but it doesn't answer "why". I would think plenty of free or cheap software is available to set up such a system as you describe.

Brady Kelly7-Aug-07 22:18

7-Aug-07 22:18

We are looking at Macro Scheduler[^] as a possible client side assist, but then I will still setup up the basic repository and web UI access for clients.

When I have looked at Macro Scheduler later today I will know more, but I would still like some comment on my two design issues.

"Once in Africa I lost the corkscrew and we were forced to live off food and water for weeks." - Ernest Hemingway

My New Blog

POS discount headache

dazfuller6-Aug-07 4:29

dazfuller

6-Aug-07 4:29

Hi,

I'm currently writing an internal system at my employer which handles sales and we need to handle discount schemes which are subscriber, customer, purchased based and with some weird and wonderful variations. Has anyone come across trying to design a solution for this before and how did you get started?

Thanks

led mike6-Aug-07 4:57

6-Aug-07 4:57

RavensCry wrote:
for this

RavensCry wrote:
discount schemes which are subscriber, customer, purchased based and with some weird and wonderful variations.

*sigh* Geez maybe you should put that in as your Google search phrase, what could go wrong?

dazfuller6-Aug-07 5:23

dazfuller

6-Aug-07 5:23

Probably should have mentioned that I'd done that already and couldn't find any useful leads, I wouldn't dare think of posting here without doing a Google search first Smile | :)

originSH6-Aug-07 5:00

originSH

6-Aug-07 5:00

Yep, I worked on an epos system a few years ago. The way we did it was to find out each 'type' of discount scheme they wanted i.e. Buy one get one free, collection of specific products for specific price.

We couldn't think of a way to do it dynamically at the time so we just coded each seperate type, and yeah that was a lot of work :P

dazfuller6-Aug-07 5:33

dazfuller

6-Aug-07 5:33

We've just had one idea about holding all the discount scheme's in an XML file with a description of how to determine if the discount applies but haven't gotten as far as figuring out how to load the code in or if it's even practical yet.

So something like this:

<br />
<discount><br />
  <discount:Template><br />
    <rules><br />
      <rule><br />
        <name>Rule1</name><br />
        <value>IsSubscriber(id)</value><br />
      </rule><br />
    ...<br />
    </rules><br />
  </discount:Template><br />
  <discount:Query><br />
  <![CDATA[<br />
  bool param1 = {Rule1};<br />
  bool param2 = {Rule2};<br />
<br />
  double fee = GetStandardFee;<br />
<br />
  if (param1 && param2)<br />
  {<br />
    fee = fee * 0.6;<br />
  }<br />
<br />
  return fee;<br />
  ]]><br />
  </discount:Query><br />
</discount><br />

led mike6-Aug-07 8:42

6-Aug-07 8:42

RavensCry wrote:
how to load the code in

interface and factory

RavensCry wrote:
or if it's even practical yet.

Yes it is

Robson Félix14-Aug-07 7:01

Robson Félix

14-Aug-07 7:01

Well, I've done POS systems in the past - my first one was probably 13 years ago, in Clipper - and what I would do is create a hierarchy database of formulas that should be evaluated according to the chain of events in your hierarchy. You can use that not only to discounts, but to other stuff. I come from Brazil and we're know to have the most complex tax systems in the world (do a search on Google and you would understand what I am talking about - even SAP has an engine specifically created to work with our out-of-this-world tax rules).

So, on the top of hierarchy you would have the more general items (for example, State, if you want people from out of state to have a different discount, for example) and go down the "tree".

The formula you're going to use is the one that goes to the utmost item (further down the tree). Then you can use any expression evaluator available in the internet (there are tons of them for C# and other languages). You can even create your own language - supporting IF statements, for example - to accomplish more complicated tasks. I've done this successfully in a pretty interesting project in the company I currently work for.

Hope this helps.

Robson Siqueira
Enterprise Architect

encapsulation VS development time

amit.arora14-Aug-07 4:58

amit.arora

14-Aug-07 4:58

HI ,

I had worked on something similar like this , in our case the promotions were flowing from SAP ( eg : buy one get one free , combo , etc ) .There were combination of 5-6 of these ,which covers nearly most of the promotions which are given mostly at POS , so we coded each of them .

amit

Error States

Leslie Sanford5-Aug-07 17:15

Leslie Sanford

5-Aug-07 17:15

It's easy enough to throw an exception from a method in a class you've written when the preconditions of the method haven't been met. You can do so while maintaining the class's invariants. It's harder, though, handling an exception thrown in the middle of the method from a source outside your control, say from an object you've invoked a method on. Enforcing a class's invariants is trickier here.

One approach is to treat the exception as an event. When it occurs, you have code in place that transitions your object to a state appropriate to the exception thrown, i.e. some kind of error state. The question then becomes how should the object behave in an error state? Are there steps to be taken to bring the object back to a useable state? If so, what are they?

I'll give an example. I have a Synthesizer class in my C# Synth Toolkit. It's capable of recording the waveform output as it is synthesized and writing it to a wave file. This takes place on thread other than the main one.

Let's say that an IO exception of some sort happens when attempting to write the waveform data to file. The exception is caught. Now what? I was thinking that it would be appropriate to raise an event on the main thread notifying the user that an error occurred. In addition, the Synthesizer would transition back to a non-recording state. The Synthesizer would continue to function normally.

Another situation has me a bit stumped, though. The Synthesizer uses an OutputDevice class for playing waveform data. If the OutputDevice object being used throws an exception, it's pretty much a show stopper. There's not much the Synthesizer can do if it's OutputDevice isn't useable. So I was thinking that in this situation, an error state would be appropriate. In this state, some of its methods would throw an InvalidOperationException; they just can't be performed if the OutputDevice isn't working correctly. Other operations might be allowed in order to reset the OutputDevice to get it working or use a different one.

I guess what I'm getting at here is that I'm wondering how you deal with exceptions when they pretty much render an object unuseable.

Re: Error States

Luc Pattyn5-Aug-07 22:52

Luc Pattyn

5-Aug-07 22:52

Hi Leslie,

I see an analogy with a file system and a "disk full" situation.

The .NET Framework does not do anything special under "disk full" situations; you may
have created a file, and be happily wiriting to it, suddenly it throws an exception at
you. If you happen to be on a separate thread, you would use some mechanism (an event
probably) to signal it back to the requestor )e.g. the GUI).

The File class does not handle events, but it does not launch separate threads eiher;
they leave it open to the user to make such choices if and when required.
if you do create a thread, obviously it may encounter a problem and I think you should
indeed include an event mechanism to signal those.

Hope this helps.

Luc Pattyn [Forum Guidelines] [My Articles]

Re: Error States

Leslie Sanford6-Aug-07 5:55

Leslie Sanford

6-Aug-07 5:55

Luc Pattyn wrote:
The File class does not handle events, but it does not launch separate threads eiher;
they leave it open to the user to make such choices if and when required.
if you do create a thread, obviously it may encounter a problem and I think you should
indeed include an event mechanism to signal those.

I was thinking about having an Error event that is raised when an exceptional situations happen on a thread other than the main one. The event will be marshaled to the main thread by using the SynchronizationContext class.

However, last night I did some research, particularly looking at Java's checked and unchecked exceptions. A best practises tutorial suggested that you should catch exceptions you can recover from but not those you cannot. Seems like common sense. So in those situations in which I can devise a strategy for recovering from an exception, I should catch it and deal with it. But there seems to be a class of exceptional situations in which there's not much that can be done. In those cases, the exception should not, according to the tutorial I was reading last night, be caught. I'm a little fuzzy on what happens to unhandled exceptions. I suppose there should be a strategy at the top level for dealing with them, notifying the user, and possibly shutting down the application.

Re: Error States

Luc Pattyn6-Aug-07 6:23

Luc Pattyn

6-Aug-07 6:23

Yes, I think along similar lines.

Some more remarks:

1.
you are allowed to catch Exceptions to your hearts content, but not to hide them:
i.e. you should remedy the problem or somehow let it ripple upwards. There are three ways
to do that:
- not catching the exception at all;
- rethrowing the same exception (a simple "throw;" does that);
- and the favorite: throw a new exception, more specific to your class or method, that
holds the original exception as an "inner Exception".

2.
Your background thread probably already has a Done event, thru which it reports results;
you could use that same event to report problems, so there may not be a need to add
another event.

3.
there are situations where exceptions are more difficult to catch, such as in a constructor,
on a thread you have no control over (say a timer tick), in native code, ... .
The framework has some provisions to catch and handle these too, but I haven't grasped it
completely yet. Things related to this are:
- having try-catch in static Main(); I recommend this, it helps during development,
especially is you show its entire ToString() as one always should.
- AppDomain.CurrentDomain.UnhandledException
- Forms.Application.ThreadException

Greetings,

Luc Pattyn [Forum Guidelines] [My Articles]

DynV2-Aug-07 14:13

DynV

2-Aug-07 14:13

I was looking back at my first worthwhile project and I notices that I made a more than a few functions to separate my code as it was getting cramped. While it seems to do the job, it is harder to maintain and is far from the ideals of Object-Oriented Programming (OOP), especially the encapsulation aspect ! Refactoring the code seems attractive but the time expenditure doesn't seem to outweigh the advantages I'd have from fast maintenance and portability as it seems too project-specific. However, I'm looking for a general guideline so that I may approach the solution from a more distanced point of view.

Je vous salue du Québec !

Re: encapsulation VS development time

led mike3-Aug-07 5:10