|
Hi all,
imagine this scenario:
- a tree structure is to be be built (edit/insert/delete nodes) by a group of several users
- the tree is expected to be quite large once finished (~100.000 nodes!)
- inserting, moving, copying/pasting nodes is required
- child nodes can result from individual requirements ("make a child node named A") or
from applying domain values (e.g. countries, currencies etc): "make child nodes for the
countries I select in a list".
My ideas so far:
- parent-child database table on MS SQL
- tree versioning similar to a sourcecode management system; each "check in" makes a new tree
version, so that older versions are present (for comparison, as copy/paste source)
- display in a multicolumn treeview using Infragistics Ultracontrols' UltraTree (present)
- nesting is done by parentID, and order of siblings (=children of the same parent)
is controlled by an additional integer value, so it can be changed ("move up/down").
Main problems:
- concurrency managment: my initial approach (timestamps for optimistic locking) could lead to
"outdated" branches, so that hours of work could not be saved, as the database status is newer
- key generation: classical parent-child-trees need keys to be referenced. Usually, I use
identity integer values in hierarchical dataset, so new nodes (-1, -2, -3) get their IDs on
dataset.Update, but here, the reference is recursive (not just hierarchical). Creating a branch
has to create the nodes "top down".
Of course, if a commercial solutions provides a way to edit/manage a huge tree with several users, I'd be glad not to have to reinvent the wheel.
Any ideas or hints?
Thanks!
Martin
|
|
|
|
|
That's not hard to ... imagine. But, imagining a single developer implementing this ... who is not already a wizard with an advanced DB that supports the features you describe ... that's hard.
There's lots of $ implementations out there: at the low end, checkout TreePad.
You might look into graph databases like Neo4j, or Amazon AWS Neptune (cloud), if you foresee the need for a highly connected set of entities where creating/manipulating linkages between nodes is critical.
A recent overview of graph DB's: [^]
«Where is the Life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?» T. S. Elliot
modified 11-May-19 22:15pm.
|
|
|
|
|
Thanks, I'll have a look at these.
Maybe my replies to the others' Posts may provide deeper insight.
modified 13-May-19 8:33am.
|
|
|
|
|
atrus2711 wrote: Maybe my replies to the others' Posts may provide deeper insight.
... uhhh ... no ... I keep bumping into too many other blind wise men in the room with the elephant 
«Where is the Life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?» T. S. Elliot
|
|
|
|
|
Sorry, doesn't sound like an application.
atrus2711 wrote: child nodes can result from individual requirements ("make a child node named A") or
from applying domain values (e.g. countries, currencies etc): "make child nodes for the
countries I select in a list".
From what list in which application? Let me guess, a "domain value" can be anything? The reason I'm sceptical is because I've seen this before; a tree that should hold everything (and then the spec stops).
atrus2711 wrote: - display in a multicolumn treeview using Infragistics Ultracontrols' UltraTree (present)
Yuck. A treeview with a datagridview next to it would work better and probably perform better. You want to mix extra data (unknown # of columns) into a tree, displayed in a single control. I'd be going for a simple tree, loading only the expanded nodes, and only fetching the data for the grid on the selected node.
atrus2711 wrote: - concurrency managment: my initial approach (timestamps for optimistic locking) could lead to
"outdated" branches, so that hours of work could not be saved, as the database status is newer
Most people lock the stuff that edited. If they are inserting the same info, then there's a problem with the workflow, not the application.
atrus2711 wrote: - key generation: classical parent-child-trees need keys to be referenced. Usually, I use
identity integer values in hierarchical dataset, so new nodes (-1, -2, -3) get their IDs on
dataset.Update, but here, the reference is recursive (not just hierarchical). Creating a branch
has to create the nodes "top down".
You can still use a unique number, you just can't expect those numbers to stay sequential in the database. People will move branches, delete them; so the numbering won't stay consistent in the db.
Also, users don't like to have to expand over 10 branches just to get to their data. If the only stuff they work on is hidden three levels deep, I'll bet you that they want the tree to open THERE, without including all the stuff they're not using.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
Thanks for your post.
The tree is a chart of accounts for a financial company, i.e. the root items are things like "assets", "liabilities", "Profit and loss" and "Notes", each of them having thousands (!) of sub-branches. The leafs are to be booked on to, the non-leafs are sums.
You don't want to know how this giant tree is built up to today.
Quote: Let me guess, a "domain value" can be anything
The whole tree consists of nodes which are "bookable accounts or "sum positions". Each of them can result from different reasons. E.g. some node might Need breakdown in currencies, another one is detailed in customer groups. The reasons vary, as the lawmakers/regulatory authorities' mind changes.
Quote: Yuck. A treeview with a datagridview next to it would work better and probably perform better.
For input, yes. But the tree has to show several columns and once, which the built-in treeviews in MS VisualStudio do not provide.
Quote: Most people lock the stuff that edited.
Not here, as the whole account plan has to be built and seen. A position like "other" might occur several times, but is not expressive itself: it's only the "fullpath" that Shows what the Position is talking about. "others" might be other customer types, currencies, ...
Quote: You can still use a unique number, you just can't expect those numbers to stay sequential in the database. People will move branches, delete them; so the numbering won't stay consistent in the db.
There's no need for sequential numbers. The Position in the tree would be determined by parentID and "location within siblings". Yet, the Keywould either be created by the database, resulting in a timelag between node Insertion and db roundtrip, or the Client would create a key itself (e.g. a GUID).
Quote: Also, users don't like to have to expand over 10 branches just to get to their data. If the only stuff they work on is hidden three levels deep, I'll bet you that they want the tree to open THERE, without including all the stuff they're not using.
I wish it was like this. Tree partitions are meaningless, as explained just above....
Maybe my replies to the others' Posts may provide deeper insight.
modified 13-May-19 8:32am.
|
|
|
|
|
atrus2711 wrote: There's no need for sequential numbers. The Position in the tree would be determined by parentID and "location within siblings". Yet, the Keywould either be created by the database, resulting in a timelag between node Insertion and db roundtrip, or the Client would create a key itself (e.g. a GUID).
Table names are unique within their database. Meaning each row in each table in each db can be uniquely identified by using the primary key, tablename and databasename. That would give you a key for your tree that has a unique name for each node, one that can be used to backtrace the item it is supposed to represent in the database.
atrus2711 wrote: Not here, as the whole account plan has to be built and seen.
..but not edited in its whole - that's why there's table level locking for example. Again, a problem that has been solved already, there's a lot of databases out there where multiple users are editing data.
atrus2711 wrote:
For input, yes. But the tree has to show several columns and once, which the built-in treeviews in MS VisualStudio do not provide.
Read the UX Guide. A treeview should not have tabular data. It is there to navigate to a node, not to display columns. That kind of data goes into another control that is specialized for that purpose, either a datagridview or a datalistview. See explorer, see regedit, see any decent application.
The idea that you "need" to edit a column as part of your tree in the same control is nonsense.
atrus2711 wrote: the root items are things like "assets", "liabilities", "Profit and loss" and "Notes", each of them having thousands (!) of sub-branches.
Because there is NO other way of making a selection beforehand - it HAS to be part of the tree, and cannot simply be a button saying "Notes" 
Good luck
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
Quote: Table names are unique within their database. Meaning each row in each table in each db can be uniquely identified by using the primary key, tablename and databasename. That would give you a key for your tree that has a unique name for each node, one that can be used to backtrace the item it is supposed to represent in the database.
You are NOT suggesting to make a single table for each tree Version, are you? "Same things go in the same table". All tree items are - tree items.
Quote: ..but not edited in its whole - that's why there's table level locking for example. Again, a problem that has been solved already, there's a lot of databases out there where multiple users are editing data.
hmmmm... It's rather similiar to a "merge" function used in source code Management Systems like git or Subversion: you edit local code (= local changes of the plan), try to commit, and find that someone edited the items you were referring to. Git demands a "sort ort/merge" process. And that's not automated - one has to find out what is code to be accepted or rejected. It's not as easy as "John locked the table - get time to get some coffee..."...
Quote: Read the UX Guide. A treeview should not have tabular data. It is there to navigate to a node, not to display columns. That kind of data goes into another control that is specialized for that purpose, either a datagridview or a datalistview. See explorer, see regedit, see any decent application
Assumed we use a "single column tree": Which column would that be? The description? The ID? The "account number" used in the target (Downstream) system? Whatever it is: the others might by needed. Search can be assisted by a separate dialogue, but this would be - as you Claim - a nonhierachical "list". How would we distinguinsh the ~500 "other"-items, if not by their nesting (fullpath)?
Quote: Because there is NO other way of making a selection beforehand - it HAS to be part of the tree, and cannot simply be a button saying "Notes
Well - a new node in root "assets" will almost always "need" additional, non-determined nodes in the "Notes" section, thousands of accounts far away. Imagine that the new asset Position is e.g. "machines", then there'll be probably (as the authorities demand) "explaining" positions like "machines bought" and "machines sold" or non-sum-up-breakdowns like "of which..."... It's difficult, it's whatever the auditors decree, and it's definitely not cast to stones that the root nodes will be "assets", "liabilities" etc. forever. So, any button will not Show "Notes", but "Root". Whatever its Name is. We do not make up this hierachy. We are bound by law/regulation to implement it.
So, any irony is futile - the Auditors are right. Even if they're not.
|
|
|
|
|
atrus2711 wrote: You are NOT suggesting to make a single table for
I always expect 3NF. Not a suggestion, but simply something that is expected.
atrus2711 wrote: And that's not automated - one has to find out what is code to be accepted or rejected. It's not as easy as "John locked the table - get time to get some coffee..."...
Because someone has to authorize a new branch, because someone is responsible for the entire project (a project leader). That is not the case with simple data-editing. The checkin cannot be automated, because a machine cannot make that decision.
atrus2711 wrote: Assumed we use a "single column tree"
A tree has no columns. You may link any kind or amount of columns to a node though.
atrus2711 wrote: We do not make up this hierachy. We are bound by law/regulation to implement it.
Law does not state that you make a UI as useless as possible.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
You haven't described any "use cases". Nothing about number of users.
Talk of "huge", concurrent and versioning seems premature at this point.
It's actually easier to roll your own than you might think.
"(I) am amazed to see myself here rather than there ... now rather than then".
― Blaise Pascal
|
|
|
|
|
Thanks for your post.
Quote: You haven't described any "use cases". Nothing about number of users.
Yes I did:
- several users (Ok, Detail to ~5-10, all working on LAN in one floor)
- use cases: inserting, editing, deleting, moving, copying nodes.
Quote: Talk of "huge", concurrent and versioning seems premature at this point.
Well, compared to the way it's done until now, it's not premature, it's high time.
Huge = Needs a decent way of keeping track (cannot be memorized by a account Guru).
Concurrent: There's no single Editor who takes care of the tree. A whole Division is working on it.
Versioning: the trees Change over time, whenever the legislators/regulatory authorities decree a new set of reporting demands. Last year, we used Version 8236, this year, we need a new tree with some new Detail positions and some dropped ones.
And, finally, of cause this tree is not "isolated" in the application I imagine. It is fed ("exported") to a Company application which uses business rules to "route" postings (etc) to the positions in the tree. And no, it's not SAP.
"Route interest postings of construction customers in Belgium, caused by loans backed by mortgages meant for constructing shopping malls located in Ghana, to positon thisandthat."
Maybe my replies to the others' Posts may provide deeper insight.
|
|
|
|
|
A WPF or UWP ListView; hosting a custom control for each node (expander icon, text) for the tree; virtualized over the display height.
The data source can be any database.
Your "use case" is CRUD. Generic in the absence of extra detail. "Objects".
The "chart of accounts" ACCOUNT type is a single class; with ONE PARENT; and optional children. It's one of the simplest and most sensible hierarchical structures around.
The COA is not 100K; EXCLUDING the detail posting accounts (e.g. Clients), it's not more than several 1000 (and that's anal); and is STATIC for the most part once created.
A division of 5 to 10 users ... Optimistic locking.
About 1000-2000 lines of C# code; with comments.
Easy if you understand recursion. (And Accounting).
The part everyone usually under values (or over values) and messes up the most is the "account code".
"(I) am amazed to see myself here rather than there ... now rather than then".
― Blaise Pascal
modified 13-May-19 9:52am.
|
|
|
|
|
Quote: The COA is not 100K; EXCLUDING the detail posting accounts (e.g. Clients), it's not more than several 1000 (and that's anal); and is STATIC for the most part once created.
Currently, we have ~30000 non-leafs and ~70000 leafs. As soon as "higher forces" require a leaf to be detailed, it's no leaf any longer. So, if you mean "STATIC" as "unchanged", it's not true. IF you mean static as a access modifier, you might have a point.
|
|
|
|
|
As I said, the account structure is usually what gets messed up.
Your concept of leaf and non-leaf may have no relationship to what one might consider as a "typical" COA for a particular industry.
This isn't about your design; it's about generally accepted accounting practices (GAAP).
Study their financial statements to see what the purpose of the COA is and how it relates. You won't find 30,000 accounts or anything close.
I think you're confusing the COA (i.e. static "codes") with the General Ledger and subledgers (posting accounts). While the two are related, they are separate things; and where DELETES are strictly verboten.
(It's your BLOCK CHAIN ... in other words)
"(I) am amazed to see myself here rather than there ... now rather than then".
― Blaise Pascal
modified 13-May-19 13:02pm.
|
|
|
|
|
I guess that the balance and taxation division of a bank is appropiately educated to see what their accounting application needs. The question was less "how would a proper account structure look like", but rather "how can it be edited". Today,they use Excel! THAT is hell! Any less heated hell ist highly appreciated. (I agree that a newly founded bank would use current software, but as long as I'm not in the board, I can't change the software they use).
So, we do not need to discuss business demands, but their technical implementation. We do have a tree, ~100000 nodes, references (e.g.some asset position has references to notes positions "far away" in the tree), and all positions are caused by regulation, but can be arbitrary deeper detailed (e.g. for easier reconcialiation reports).
|
|
|
|
|
Another "problem" is that "designers" of accounting systems attempt to use the COA to fulfill EVERY possible reporting requirement; not appreciating that "reporting (tree) structures" can be built independently of the COA's structure.
Everybody thinks their situation is "unique", when in fact it isn't. It's just misunderstood.
In realty, most employees are not allowed to "look at the books", and are limited to one small slice (if any) on a need to know business. I see a lot of issues in your future.
"(I) am amazed to see myself here rather than there ... now rather than then".
― Blaise Pascal
|
|
|
|
|
When I edit video recording I backup the video project every time I make a few changes that way if something does not work then I can go back to a previous project backup eg Gardens1.veg Gardens2.veg etc
I am hoping that you can do the same when writing a C# program but can't see any way of doing this.
I normally select 'save all' so that everything is saved.
Brian
|
|
|
|
|
Welcome to the tribe of devs who need version control/back-up software !
imho, it's a strange "empty" that Visual Studio doesn't provide a simple menu command to back-up an entire project/solution ... for the single developer.
This post gives a useful overview of what you might look into to determine what you need: [^].
I sometimes put a shortcut to the current VS solution folder on the desktop, and then duplicate (the contents of) that as necessary: not elegant !
I did create my own solution for this (just code, not a VS extension), but, it was quirky, and I had to use try/catch to avoid access errors for certain locked VS run-time hidden files: I stopped using it
«Where is the Life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?» T. S. Elliot
modified 11-May-19 8:11am.
|
|
|
|
|
Backups and version control are very different beasties!
Version control is pretty easy with all .NET languages - GIT support is built in to Visual Studio these days, and that allows you to use a local of remote (like BitBucket or GitHub - the latter is better IMHO) and that provides you with full source control. Google "Visual Studio GitHub" and you will find loads of explanations.
Backups aren't normally "single project" related, but that's easy enough to do. The only warning I would give is that a backup regimen should be a bit more complex than source control simply because backed up data should be "air gapped" to prevent corruption when things go seriously wrong!
Have a think about exactly what you are trying to achieve: I use source control and backups and it takes both care and thought (as well as time) to get it right.
And always, always, always: do a trial restore of any mechanism to make sure that you can use the data before you assume it's right! There is nothing worse than being confident in your backups and finding out they are rubbish exactly when you need them to be perfect!
See here: Don't Hit Save - The Screwup[^]
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Hi, I'm curious about what you use for back-ups ... assuming you are automating that in some way: fyi: I interpret "version control" as implying some form of back-up/archiving. cheers, Bill
«Where is the Life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?» T. S. Elliot
|
|
|
|
|
AOMEI Backupper and a bunch of 4TB USB drives!
Version control shouldn't be seen as a backup - or at least not a "primary" backup - the data is out of your control (unless it's a local hub and then it's not air gapped and probably on the same machine).
And we've probably all heard "Git lost the lot!" sob stories, though this can help: Oh, sh*t, git![^]
What I do is use Git for source control, and archive "released to client" software to air gapped RAID 5 backups which live in a friends garage when I'm not using them, as well as "normal" regular backups of my whole system as AOMEI images.
And I copy whole folders to my 16TB RAID 5 NAS when I'm feeling especially nervous about making changes.
I think the most work I've completely lost in the last 20 years is about ten minutes.
It's not paranoia if HDD's really are out to get you!
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
That's enlightening, thanks !
p.s. could you please send me one of those 16TB Raid 5 critters asap ?
«Where is the Life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?» T. S. Elliot
|
|
|
|
|
Thanks for your reply and the replies of others to my question on backing up C# code.
I'm thinking that it might be easier to zip up the project folder for the project that I'm working on and add version numbers to the zip file.
If something does go wrong then am I able to restore the zip file or does Visual NET write information elsewhere other than in the project folder?
Brian
|
|
|
|
|
So long as you're not doing anything with source control, everything is in the project folder.
|
|
|
|
|
Hallo there
i want to get the Aveg Temperature of August which has 31 day but sometime there is a data of temp just for 29 or 27 however how could i get the Avag based on how many days has been read and saved as data that could be (29,27,26 or 10 days) ?
here is the code :
public static TheWeather GetAverageForMonth(List<TheWeather> entries, int year, int month)
{
if (entries == null || entries.Count < 1)
return null;
int days = 0;
int index = entries.Count();
double total = 0.0;
double Aveg = 0.0;
for (index = 0; index < entries.Count; index++)
{
if (entries[index].dateTime.Year == year && entries[index].dateTime.Month == month)
{
total += entries[index].Actual_mean_temp;
days = entries[index].dateTime.Day;
}
}
Aveg = total / days;
return entries[index];
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
