|
So here's the thing. I am a new coder and I am working on my first big project from scratch. My project is using ASP.NET MVC, Google Maps API, Entity Framework, with SQL Server.
I got a very early protype started on my local machine but I'm starting to run into database design complications. I am already new to coding, but somehow have managed to figure out MVC and get a Web API going. However, I am even less experienced than that with databases and database administration. I know how to add/remove database objects and set up tables and all that jazz; that's not a big deal. But I'm having questions such as:
*What is the best way to associate user accounts in my system with their business' information...
*I want my users to be able to add/remove products to their business' inventory table... What is the best way to set up the database to store each individual product.
*What is a good way to pair the information given to me via business owners with the data I pull from Google API.
*Should I create a new object for each inventory item or just list it in a long text string of all inventory items since no details will be needed?
*When do I have my application create an entire new database versus just add a new table in an existing database? For example, should user accounts and businesses be in two separate databases?
And more...
These questions are less coding questions and more database setup/administration questions... Questions I'd love to ask someone who has made a big modern commercial database before that handles accounts and inventory and all that. Obviously, coding questions will come up too but a lot of it is just database design.
So, my real question here is... How do most coders do this? I would iamgine a lot of coders are not DB experts... Should I buy a Database Administration book and read it? Should I consult an expert? Since this will be a data-driven application, database setup is important. Thanks!
PS: Please, don't worry about me being overwhelmed. I mean, if this is something that will seriously take 6 months of dedicated study, it may be better left for me to call up someone else... But even though I just started programming a few months back, I'm actually loving getting into all of this and it is not intimidating at all.
modified 9-Aug-16 4:51am.
|
|
|
|
|
Do NOT get a book on database administration, yet. Get a book on database DEVELOPER, admin and dev are very different roles. There is also as site with sample database, ah found it[^], pick one close to your industry and try and understand why the design was used. He is pretty good and faithful to the rules of DB design.
After you have got a feel for DB design feel free to ask silly, or not so silly question here.
Your first 3 questions are answered by foreign keys
You should create 1 table per CLASS of inventory (if an items attributes are so different to an existing class then create a new class).
Your application should NEVER create a new database except for backup (and this should be an admin job not application).
If your scale is going to so huge (as in multiple of terabytes) that storage is an issue then you could split the data into different databases but I have never had to do that.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
If you're working on enterprise or Web Applications, it's pretty much a given that you'll need to understand how databases work. There's not getting away from the fact that you can mess yourself up badly if you don't understand normalization, concurrency, and relationships.
The current bar is SQL, and if you're interested in web apps I would strongly encourage you to familiarize yourself with relational design; your model class designs should be informed by the relational system and, more importantly, normalization.
Once you understand the relational schema, then make your life easier by having a look at document stores like Mongo.
"There are three kinds of lies: lies, damned lies and statistics."
- Benjamin Disraeli
|
|
|
|
|
TheOnlyRealTodd wrote: I mean, if this is something that will seriously take 6 months of dedicated study
Not sure how to address that part. A database admin who comes into a shop with no other admin should have about 5 years of experience doing just that. So 5 years of a 40 hour a week job.
A place that wants to hack it a bit more with small initial aspirations but whose goal is to deliver a product could get by with a developer who has had 2 years of experience working with databases via a programming language (so familiar with the database but primarily programming is C#, Java, etc.)
TheOnlyRealTodd wrote: How do most coders do this?
Do it every day for years.
TheOnlyRealTodd wrote: Since this will be a data-driven application
Very little that isn't but that doesn't insure that you must use a database.
TheOnlyRealTodd wrote: Please, don't worry about me being overwhelmed...I'm actually loving getting into all
So dive in. You can't learn it if you never start.
There are books about programming with databases. You should find one of those. Database design is a different issue just as design itself is a different issue and I have never found a book that really teaches design well (of any sort.) The basics of programming however does lend itself well to books.
If you don't mind reading then one that focuses on database programming and another that attempts to teach database admin will, over time, help. I have never been a DB Admin but the admin books I do have have been helpful.
As a suggestion try to keep your design as simple as possible.
|
|
|
|
|
Hi,
I am trying to pass a varchar string from C# to MySQL stored procedure to be used in WHERE xx IN (param). I tried it in the basic way below but it's not working and giving no result.
can any one help please.
this is my C#:
sql_connection = new MySqlConnection(serverClass.connectionstring("BlueFile", "BlueFile", "Server"));
sql_connection.Open();
sql_command = new MySqlCommand("sp_populate_memo_country_companies", sql_connection);
sql_command.CommandType = CommandType.StoredProcedure;
sql_command.Parameters.AddWithValue("param_country", Convert.ToString(cboToCountry.EditValue)).MySqlDbType = MySqlDbType.VarChar;
sql_adapter = new MySqlDataAdapter();
sql_adapter.SelectCommand = sql_command;
data_set = new DataSet();
sql_adapter.Fill(data_set, "companies");
DataViewManager dataview_manager = new DataViewManager(data_set);
DataView main_dataview = dataview_manager.CreateDataView(data_set.Tables["companies"]);
cboToCompany.Properties.ValueMember = "location_id";
cboToCompany.Properties.DisplayMember = "company_name";
cboToCompany.Properties.DataSource = main_dataview;
DevExpress.XtraEditors.Controls.LookUpColumnInfo("department_name", 50, "department_name"));
cboToCompany.Properties.PopupSizeable = false;
cboToCompany.EditValue = null;
and this is my stored procedure:
SELECT locations.location_id, companies.company_name, locations.location_name, first_payroll, last_payroll
FROM locations
JOIN country ON country.country_code_alpha2 = locations.country_code
JOIN companies ON companies.company_id = locations.company_id
LEFT JOIN payroll ON payroll.location_id = locations.location_id
WHERE locations.country_code IN (param_country) AND payroll_active = TRUE
GROUP BY locations.location_id
ORDER BY companies.company_name;
Thanks,
Jassim[^]
Technology News @ www.JassimRahma.com
|
|
|
|
|
Split your country string into a temp table and use a join or (select country from newtable)
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
After looking at your code, are you sure the value of the "param_country" is a quoted, comma separated list of values ?
For example an IN clause would look like this: "select * from myTable where location in ('USA','CANADA','MEXICO')
Check the value of the sql_command.Parameters("param_country") right after you populate it with values.
BTW: There may be limitations to the number of values you can have in an "IN" clause. I remember from a while ago Oracle had a 1000 item limit, just something to think about.
david
|
|
|
|
|
I'm trying to determine if the following SQL query represents someone trying to pull information (regardless of what the information is - title, ID, message etc) from all the forum users' unread PMs:
SHOW COLUMNS FROM mybb_users LIKE 'unreadpms'
Is this the case, and why/why not?
|
|
|
|
|
Might be the case, but since it is far from being a valid SQL statement, I would not even worry if this person was sitting behind my machine using SQL Management Studio.
Not even if the other screen would show the MSDN page explaining the syntax for a SELECT statement.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
|
|
|
|
|
Excuse my writing of the statement, it's probably my fault it's not formatted correctly, I'm not that familiar with SQL, I'm just trying to determine if there's a security/privacy issue of someone possibly accessing unread PMs of the forum users.
I suppose another way of saying this, is the query a generic query you'd expect in a mybb forum database, or could it possibly be used to list (maybe not the contents but like a directory or list of PMs that they can then pick out which take their interest) unread PMs that could then be read manually?
|
|
|
|
|
Member 12662448 wrote: I'm just trying to determine if there's a security/privacy issue of someone possibly accessing unread PMs of the forum users.
Yes, by looking at the queries. The query itself won't be showing intent. It may be malicious, it may just be a dev that is testing. Problem is that you do not know the origin.
The fact that you are looking at them implies to me that you do not trust the security of the database, to which I'd have to agree. Instead of looking at a logbook who was in your house, one should be checking the lock and which users have keys. If you are confident about the lock and keys, then it makes little sense to go ask the person who entered the house what their intent is.
If the data is saved in a readable format, then yes, anyone with access to the table could read it. It does not matter what data it is - goes for (read) PM's as well as other tables.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
|
|
|
|
|
SHOW COLUMNS FROM is to get Information about the table structure and not to query data from a table. Detailed information about it you can find i.e. here:MySQL :: MySQL 5.7 Reference Manual :: 14.7.5.5 SHOW COLUMNS Syntax[^]
I assume you like to query data from the table. For this you Need to do something like this:
SELECT title, ID, message, etc...
FROM mybb_users
WHERE Status = 'unreadpms'
In the above I used Status, which you need to replace with the fieldname you are using to store the Information 'unreadpms'.
I hope it helps.
modified 19-Jan-21 21:04pm.
|
|
|
|
|
I suppose what I'm trying to say is, is it trying to access unread PMs in some way?
I understand it might not be pulling the data directly from the PMs verbatim, but in displaying the table structure, would it be showing a list of unread PMs that, say, someone could then access?
|
|
|
|
|
Not sure, whether I get your point. Do you mean something like"Return all PM's which never have been accessed/read by an SQL- SELECT"?
modified 19-Jan-21 21:04pm.
|
|
|
|
|
That's what I'm thinking but, as said, I don't have enough knowledge to make that call.
|
|
|
|
|
|
Richard Deeming wrote: It looks like somebody is trying to find and exploit a SQL Injection vulnerability in your site:
I don't see anything in there to escape the current command executed.
You could still be correct, in line with post #2; if the connection-string is exposed, anyone could use that to issue commands. It would be something used to explore the database, which could still be valid use -
More questions; is there a table with that name? Have there been other commands from the same source? Could it have been generated by a tool? (Ever seen what traffic SQLSMS causes?) 
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
|
|
|
|
|
There's likely a table with that name given it's a mybb forum, although I see where you're coming from, was it custom created for that purpose? Why would there be a table specifically for unreadpms for all users? I hadn't thought about that.
That thought worries me. They entrusted the SQL side of things to members that I personally did not know the reputation of, and I'm not sure how trustworthy they were (I personally hold the ethos you should only entrust to those you can trust).
All I know is information appears to leak from PM but only recently did I put two and two together (I'm not an admin on the site so it's not something I'd spotted conventionally).
Someone made vague insinuations another forum was dealing with an SQL injection, and that remark always bothered me. But I didn't want to prejudice the first hand analysis.
I think best course of action is to assume worse case scenario, assume there is an injection attempt and find a way to lock it down. If there isn't, site gets extra security and I just look daft, and if there is, everyone benefits.
So next question of course is how do you deal with a site that has presumably been attacked by an SQL injection?
|
|
|
|
|
Member 12662448 wrote: it's a mybb forum
Make sure you've installed the latest version of the forum software. There have been numerous vulnerabilities reported:
Mybb : Security vulnerabilities[^]
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Every now and then the forum spews an error specifying the query. It's not in a log, the forum owners are friends and I've done a little bit of SQL for another project but my knowledge is minimal, so when I saw a query asking for unread PMs, it made me suspicious.
The query is verbatim from the error, not correctly formatted I suspect.
They would have no reason to run it, and they aren't SQL savvy enough to construct SQL queries of that nature. So whoever is running it is SQL savvy.
Other forums in our localised community have been hacked, with one fairly recently taken down (the admin's login details were stolen), and we often found information mentioned in PM was somehow 'known' publicly (I conducted experiments between my forum and theirs and found any info in PM on their forum leaked).
I didn't want to preload the dice in my favour by mentioning this, I wanted to see if the query could be an innocent outcome without the prejudicial information to hand.
The fact people are verifying it's asking for all users' unread PMs (as opposed to all PMs in general or a specific users' PMs) verifies my suspicions there's been a database breach.
Do you have any recommendations for securing a database against SQL injections (or maybe some way to test to see if it's vulnerable)?
I don't think extra security could hurt here even if I'm wrong.
Thank you for helping guys, it's all appreciated.
|
|
|
|
|
For testing, you can either manually try entering single quotes in various form fields, or use an automated tool like sqlmap[^].
NB: Make sure you have permission from the site owner before testing the site - preferably in writing.
Testing for SQL Injection (OTG-INPVAL-005) - OWASP[^]
To secure the site, you need to modify the code. You'll need to find every place where it issues a SQL query, and make sure it's using parameters. If it's using string concatenation, or string interpolation, or any other means of inserting data directly into the query text, then that's a potential vulnerability.
If your database has any stored procedures that build and execute a dynamic query, you'll also need to update those to use parameters. In Microsoft SQL Server, you would use sp_executesql[^]; other databases probably have something similar.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Thank you for the help so far.
I'll ask if I have permission to conduct a pentest, then I'll try to setup the software to conduct a check.
|
|
|
|
|
Hi there,
as the title already say, I'm searching for the right database for my use case which should work with the following data:
- 2 key/value "tables"
One which holds a string as key and a normal number as value, the second table should hold also a string as key and a string as value.
The first table should hold billions of string/number values, the second one should hold millions of string/string values. So there should be stored a huge amount of data.
The only operations I need to do are the following
- constantly add new entries in both tables
- before adding a new entry, check if the key (string) is already stored
- search for entries which share the same key in both tables. In a relational DB I would execute this statement (select * from tab1, tab2, where tab1.key = tab2.key) -> This search should be as fast as possible.
I'm experienced in any kind of relational DB like oracle, mssql, mysql, postgresql....
As key/value database I only used redis so far.
I think my use case is not good for relational databases. Some NoSQL databases meight be a better choice. Redis is not good as it is an in memory DB with data size limits of the physically memory. As I'm using lot of data I need something else.
Which database would you recommend in my case?
I'm excited to hear your suggestions.
Thank you for your help and have a nice day,
Kogs
|
|
|
|
|
|
Thanks for your suggestion.
I did some research and it looks that SQLite can handle big data very well. So this could work for me.
Nevertheless I think simple key/values store databases meight be faster. But I can also be wrong as I don't have much experience with NoSQL DBs.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
