|
I'll try to keep the example simple. I am wondering if we should store data differently so that reporting might be easier. For example, let's say we log when a user logs in and out. A normal sql table might look something like this:
| user_id | action | log_date |
|---|
| 1 | Login | 2021-01-01 8:00 AM | | 1 | Logout | 2021-01-01 5:00 PM
| | 1 | Login | 2021-01-02 8:00 AM | | 1 | Logout | 2021-01-02 10:00 AM | | 1 | Login | 2021-01-02 12:00 PM | | 1 | Logout | 2021-01-01 5:00 PM |
So on and so forth.
Let's say the business needs a report of how long each user is logged in each day. I'm pretty sure I could figure out some sql that uses row_number and partitioning and subtract previous value to end up with the difference between each login and logout event and sum for each day. (Ignore night shifts that might start in the evening and run over to the next day.)
However, we don't want to be writing reports but using an ad-hoc report engine, such as PowerBi.
1) Can a reporting engine take the table of data and allow the user to run a report that shows how long each user is logged in each day?
2) As developers, should we store the data differently to make those types of reports easier? Maybe have a separate summary table?
I know PowerBi pretty well but I don't know how to do a report of how long each user is logged in based on just this table of data.
Should we as developers store the data differently, or is it up to the report (BI) side to manipulate the raw data? Thoughts and opinions?
|
|
|
|
|
It's the difference between an "operational" system versus an "informational" system.
Operational / transaction systems are optimized for day-to-day usage.
An informational system (e.g. a "data warehouse" / DW) is designed specifically for info retrieval and could include redundant data to facilitate querying. The data in the info system is extracted from the operational systems.
The operational system may span a year, more or less. The DW could span decades.
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
If I understand you correctly, keep logging it the way we are but we may need a separate system (data warehouse) to make reporting easier.
Thanks for the feedback.
|
|
|
|
|
Correct. You may also not want "ad hoc users" hammering the operational system with inefficient queries that impact overall performance of the day-to-day.
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
And now it becomes a design issue, do you store the DW data in a denormalised structure (this is standard practice for a DW) do you then write 2 distinct reports targeting each system.
How often do you transfer data to the DW?
How often do you purge data from the production system?
Never underestimate the power of human stupidity -
RAH
I'm old. I know stuff - JSOP
|
|
|
|
|
I have setup consisting of several devices that can communicate through high-speed buses. The following devices exist:
A. 1 grandparent.
B. The grandparent has 5 children. I will denote these the parents.
C. Each parent has 1 child. I will denote these the children.
The grandparent can talk directly to the parents and the parents can talk directly to the grandparent and their own child. Furthermore, the grandparent can also talk to its grandchilden, but it needs to do so via the parent and likewise, the children can talk to the grandparent, but needs to go through its parent. I need to visualize in a table (C# DataGridView) how packets are going back and forth and make it easy to understand what's going on in the system at any given time. All devices has a debug UART and I can modify the source code in all the devices to get the debug messages I need. I'm currently writing a terminal application that can handle several COM ports. I was thinking that the grandparent always logs its data with 0 indentation, parent 1 logs with 4 spaces indentation, parent 2 logs with 8 spaces indentation, etc. Then I was thinking about showing the same packet as it progresses through the chain in the same color everywhere. Whenever a log message contains the word error, then I will display that row in red. If something is purely information, but not a packet, then I was thinking about showing that with a specific pattern style (similar to Excel "Format Cell" -> "Fill" -> "Pattern Style"). Does anybody have any comments or suggestions on this? Is there something I can google image search to get some inspiration?
|
|
|
|
|
I don't know what limitations you face when displaying this, but a message sequence chart[^] might work. I generate simple ASCII versions of them in a session processing framework that I developed, and I was impressed by this article[^], which might help you generate professional looking ones.
|
|
|
|
|
Wow, that's impressive! I'm one layer above, though, so I'm only logging whether it's a read/write and the address and number of registers. I don't log the reply packets, but if I don't get a reply packet then it gets logged as a critical error.
|
|
|
|
|
If the system is idle for long time then can we find this using the mouse movement or mouse click event the system's idle time.
|
|
|
|
|
|
I'm working on a Windows forms application that shows a list of PDFs in a given folder, with a toolbar with buttons that carry out operations on the list of PDFs. The form has a data grid view that takes as its data source a list (PDFlist) of objects (PDFentry) that have properties file name, file date, file size, etc. (Like Windows Explorer).
Because the operations are carried out on the entire list of PDFs, the code for the various operations necessarily loops through PDFlist, acting on each item. That's a lot of code, so form.cs is pretty big (900+ lines). It seemed to me that PDFlist could be its own class, inheriting from List and adding methods for the various operations. Then form.cs would just have a line that said PDFlist.Compress(), or PDFlist.Print(), or whatever the operation was.
But then the red flag was that there's only ever one instance of PDFlist. It's not a singleton in the sense that I don't have to prevent a second instance from being created. It's just that the application only uses one PDFlist throughout.
Would this be a case where a static class is appropriate?
|
|
|
|
|
One thing I have painfully learned is that while a static may work now, at some point you will end up using it from multiple threads and then all hell will break loose. If the class has any state, try not to make is global or thread static.
Also, these types of errors are some of the worse to debug.
"Time flies like an arrow. Fruit flies like a banana."
|
|
|
|
|
RobertSF wrote: Would this be a case where a static class is appropriate?
No, a normal class is what would be appropriate. Whether you have one instance of the class or one thousand is irrelevant.
|
|
|
|
|
I have (static) "adapter" classes; similar in concept to a "table adapter" and its "fill" (a table) method, etc.
Static, because often there is no instance data other than the object being operated on. Just makes calls simpler when you don't need a instance reference to an adapter.
Adapters, for me, often make more sense than adding methods to an object, which then operate on "itself".
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
|
What is Google[^]?
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
I'm keeping an eye on this user. Seems potentially spammy to me.
|
|
|
|
|
Class to leave test traits representing handling failures in a live build so long as everybody uses the liveliest google build but complains only about other multinationals like Mozilla when stuff is already messing with their 'logs'.
------
class NoddyXmlStorageBuilder : bust::maybeSingletonInstancesNeverWereThing
{
NoddyXmlStorageBuilder setNest(std::string what, IRegex why) = 0;
IRegex get(std::exception why_not) = 0;
}
class BittenByGoogle : bust::disable_when<NoddyXmlStorageBuilder::isVisualOnly>
{
// add your more classy methods here
// be careful as live testing traits
// will end up dead on wire someplace.
void seedRandomLogFile(std::string someGarbage, enum DebugLevel) = 0;
bool hasBeenBitten() { return true; }
}
|
|
|
|
|
What has this got to do with this thread? Was this meant to be in reply to something else?
|
|
|
|
|
Somebody mislabeled an hyperlink - so I wrote a code joke about log-directory trashing and neophytes views on storage selection.
|
|
|
|
|
Our company is currently running .NET version 4.5 and using VB as the back-sript language. All of our forms are basically using the general ASP.NET forms methods. In other words we are not taking advantage of the any of the MVC core methods.
We are getting ready to start our new design process and wanted to get some recommendations on what framework and mythologies/C# version/database methods/Web UI methods, to use in our new design.
Any advice would be grateful,
Thanks,
Steven
|
|
|
|
|
The answer is: It Depends!
What skills does your team have / how much new learning are they able to do?
Are you intending to run the old stuff concurrently with the new stuff or is it a big bang approach?
What is the problem domain? That will dictate what are the best tools for the job
Are you willing to risk leading edge (which may be buggy) or do you want trailing edge (which may cause upgrade issues in the future) or do you want to play it safe (e.g. go with the next-to-latest version)?
Who are your clients / customers? What will they be most comfortable using?
Are there off-the-shelf packages that can do what you want which could save you a lot of development time?
Stevey T wrote: recommendations on what framework and mythologies/C# version/database methods/Web UI methods, to use
Do not use mythologies: they are not real!
|
|
|
|
|
Hello,
In my company we had a system on .net framework platform. That was made from an orchestator, different rest API services on top of a monolithic oracle DB. With a huge plsql library.
We used distributed transaction for the operation (import of data). Because of the rollback possibilities if one request did fail e.t.c. And used a orchestrator in front of the rest API services (micro services) that organized the different requests all in one transaction.
We had to upgrade the system to .net core.
But .net core don't support DTC anymore !
Have anyone had a similar situation, and what what did you do about it?
Or do anyone have any comments to this issue/problem.? That if one request fails in a
big operation of many requests the database data will not be consistent, if no rollback is done.
When we use many "micro"services on top of one monolithic database.
BR
|
|
|
|
|
I think a DTC implies "multiple" "micro services".
A transaction that spans multiple databases is not what I would can micro.
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
We are thinking of building an internal Kaggle like platform to run Hackathons in our company. I am wondering what would be the best technologies to use for this and how to go about building such a site. Any help appreciated. Thanks.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
