Database Discussion Boards

First, I am an absolute "database-beginner" Smile | :)

Here is my szenario in my daily business-live:

I have a lot (~30) different datatables which I combine to produce usefull-information.

The main topics are:
a)
My datatables are stored in xml-files. The xml-files are actually generated by reading out information from other tools in my business

b)
For the "production of usefull information" (this is reporting) I use my own proprietary C#-Programs, where I perform "the logic". Means: Filtering, Joining, Combining,.....

c)
The final result is written into Excel-Sheets. The end-users use these excel-files for their needs

Here is my question:

Especially, step b, with my proprietary C#-Programs needs a lot of time for maintenance/updates/.... due to heavily changing environment. It is time-consuming to maintain it.

How do "professional database-engineers" handle such a task?. What is their approach? Which tools are they using?

modified 9-Nov-12 7:14am.

Search a topic like "Business Intelligence" and you will get an idea of what others are doing.

At a high level, you have a central database which collects data from different systems, then you have reporting tools which either you or your user community can write their own reports. Also, sometimes you can have these reports run on a scheduled basis and have their output delivered to whoever needs it.

The cost of implementing such a system can be quite high.

You might want to implement some pieces of this.

For example:
1) Build a tool which will load your data extracts (xml files) into a central database.
2) Create views or stored procedures which "normalize" your data
3) Use a reporting tool to generate the reports you need or show your users how they can connect to the central database via Excel to get their own data. (See topics like Microsoft Query)

Empower the user to access the data and you will spend less time on maintenance.

Good luck.
Thumbs Up | :thumbsup:

Frygreen wrote:
My datatables are stored in xml-files. The xml-files are actually generated by reading out information from other tools in my business

This is just wrong - extracting data from other tools is valid, moving them to xml files is probably the worst platform to work from.

Get hold of sql server, use Express if cost is an issue (has a size limit of 4gb). Your current xml files can be loaded into tables in a sql sever database.

I would seriously think about getting someone to normalise and organise your data for you it is a non trivial job. Although if you have already done this with xml you may be able to handle it.

Use SQL Server Reporting Services SSRS to produce your canned reports (no analysis or drill down) and look into ~~Integration~~ Analysis Services for data analysis beyond the static reporting requirement.

I may sound SQL Server centric b/c I am, it is by far the best tool for departmental database processing. Don't piss about with Access or Excel go straight for the professional tools.

Never underestimate the power of human stupidity
RAH

Thank you for your first replys. I got some ideas.

But, of course, more replys are welcome Smile | :)