Welcome to the Lounge

5-Jul-23 12:57

Back in 2009, I began a personal project, using SQL Server (Express) and -- after a month or two -- got it to a certain point before losing interest.
Around 2014 (if I recall correctly), I had the idea of reworking it to use SQL Server or SQL Server Compact depending on what the user specifies at run-time.
This is not difficult. If I recall correctly, Subversion can be configured to use one of two (?) database backing stores.

I have long sought a project to show off my ideas about implementing an SQL-based application which can be configured to use any one of a number of supported ADO.net providers. I just didn't want to expend a lot of effort on it at that time. Yet it has remained in the back of my mind.

This summer I have some time to have another look at it and I see that SQL Server Compact is currently out of support.
(As with OleDB, I hope Microsoft reconsiders and releases a new version, but I'm not holding my breath.)
As you may know, I don't really care that something is out of support, but finding some other product is still desirable.

What I envision is the ability to have the application, it's data file(s), and whatever DLL(s) is required on a flash drive, and just run it from the flash drive on "any" suitable Windows PC -- not relying on the host to have anything more than a vanilla .net install.

What I am looking for:
SQL-92 compliance
ADO.net provider
No software/driver to be installed on the host

From my few experiments, SQL Server Compact satisfies all of these criteria.
From what I can tell, SQL Server LocalDB does not meet all of these criteria -- but maybe I misread something. Of course, I could add support for that as well anyway.

I have looked around online a bit and not found anything which seems to be what I want.
The term "embedded database" comes up a lot, but I don't want NoSQL or a Key/Value store or anything which must be installed.

I will likely proceed with using SQL Server Compact and add support for other systems as they come to light. This particular application is really just a proof-of-concept anyway.

I just wonder whether or not anyone here knows of something I've missed.

Daniel Pfeffer5-Jul-23 18:24

Daniel Pfeffer

5-Jul-23 18:24

I don't know how SQL-92 compliant it is, but have you tried SQLlite?

Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.

Jörgen Andersson6-Jul-23 3:14

Jörgen Andersson

6-Jul-23 3:14

It's mostly compliant. Biggest deviation is that it's weakly typed and has limited support for triggers

Wrong is evil and must be defeated. - Jeff Ello

PIEBALDconsult6-Jul-23 4:43

6-Jul-23 4:43

Thanks.
I hadn't noticed that an ADO.net provider was available.
I am a little concerned about the lack of SQL-92 compliance, but I don't use triggers and some of the other features (it) lacks.
The typing system may be an issue, yet it may not be.

DerekT-P5-Jul-23 22:14

DerekT-P

5-Jul-23 22:14

Sounds as if (unless I've misunderstood) that you've got two separate requirements:
1. Build an app where you can just "plug in" any suitable datastore at runtime.
2. Build an app that has no dependencies on any dbms already existing, just install and run a single package

#1 is basically around separating out logic and data access layers; the .Net apps I write have a separate data layer that is provider-agnostic and just tweaking a single constant switches between MySql, SqlServer, Postgres etc. The value could just as well be provided at runtime.

#2 Once you've done #1, #2 becomes easier as there's no "dependence" in the app on SQL standards, since that's all the "other side" of the separation. Hence I can swap MySql for SqlServer (though the d/b does need to have stored procs that manage the database logic - queries etc). You could even consider embedded MySql (runs on Win10/11, and Win Server 12/16/19/22)

Telegraph marker posts ... nothing to do with IT
Phasmid email discussion group ... also nothing to do with IT
Beekeeping and honey site ... still nothing to do with IT

PIEBALDconsult6-Jul-23 4:37

6-Jul-23 4:37

DerekT-P wrote:
embedded MySql

Thanks. I'll have a look, I don't recall seeing it my searches.

1) Not entirely, though I could do that. I have done plug-ins before, but this would be that support for multiple providers would be built-in and then selected at run-time.

2) I would want to be in control of which providers are supported.

Plug-ins are generally done with Interfaces, which can lead to a bunch of duplicated boiler-plate code. What I do for supporting multiple SQL providers avoids that. Which is the point of this exercise -- to show a working example of this technique.

DerekT-P6-Jul-23 6:09

DerekT-P

6-Jul-23 6:09

Probably my naff terminology. I have a data layer that uses generic Data objects (System.Data.Common.DbConnection, d
DbDataReader etc) and then a very simple factory class that, dependent on a single parameter (which could be provided at runtime) returns a MySqlConnection, or a SqlConnection, etc in that generic object. So long as the datastore supports stored queries and parameters the app doesn't care what it's talking to. But it does necessitate the database being populated with table structures and SPs.

Probably getting a bit programmy for The lounge...

Telegraph marker posts ... nothing to do with IT
Phasmid email discussion group ... also nothing to do with IT
Beekeeping and honey site ... still nothing to do with IT

PIEBALDconsult6-Jul-23 6:23

6-Jul-23 6:23

DerekT-P wrote:
a bit programmy for The lounge

Possibly.

I program to the Interfaces, and ADO.net takes it from there.
The two things to consider are instantiating the correct IDbConnection, and then any variations in the SQL statements.

chrisseanhayes7-Jul-23 10:12

chrisseanhayes

7-Jul-23 10:12

the biggest drawback to "separating your data layer" idea is that people forget just how much logic is baked into their SQL calls.

people are writing business logic in their SQL. the challenge is to see if you can "separate" that into a layer.

the only idea I had was DSLs. those are really hard to implement.

this is why i can see someone wanting to keep the sql and switch the implementation.

jschell7-Jul-23 12:24

7-Jul-23 12:24

chrisseanhayes wrote:
people are writing business logic in their SQL.

Hopefully because they realized there is a business need for that. One that is solved by that exact logic.

Absolutism such as 'no business logic in the database' can lead to very problematic business solutions. I saw that first hand at a company where the product that was bought as 'database independent' had two guys on site for weeks (and our DBA dedicating a lot of time too) trying to get their app to run anywhere close enough that the company could even conceivably attempt to use it. When I quit that job they were still trying and it was still hours longer than it needed to be. The DBA and I both agreed that the problem was because they were dragging the entire database across the network so they could process it.

chrisseanhayes7-Jul-23 14:14

chrisseanhayes

7-Jul-23 14:14

every time you use sql server for more than crud then you are married to sql server.

joining, aggregating, filtering, projections, unions

these are just a few of the business logic decisions totally married to your database implementation. they are next to impossible to implement solely in code without reinventing the wheel. that's just what all these other databases are doing. No-Sql (like Cassandra), kvp (Redis) outside of crud operations they present just as hard of a curve for creating the above needs.

jschell10-Jul-23 3:31

10-Jul-23 3:31

chrisseanhayes wrote:
every time you use sql server for more than crud then you are married to sql serve

False.

And I have experience in the following
# Applications which used multiple databases. Including stored procedures.
# Migrating from one database to another. In one case I did that (me alone) in one week when switching from SQL Server to Oracle. For that case I did have the advantage that I designed and implemented the original layer.

What ties one to a database is not the database nor the actual interface to the database but rather inexperienced developers who attempt to solve problems with a lack of knowledge about applications and databases.

chrisseanhayes wrote:
joining, aggregating, filtering, projections, unions

You mean of course...SQL. Yes if one lacks knowledge in that and attempts to roll large enterprises with an ongoing lack of knowledge then that will be a problem.

But that is true for everything. If one decides that an entirely new tech stack (with no experience) is the way to go. Or even if one starts a company with only tech guys with no idea how to actually sell or even what the market is.

chrisseanhayes wrote:
they are next to impossible to implement solely in code without reinventing the wheel.

Not even sure what that means.

Databases are in fact written code. So yes code can implement joins, etc. But I have also seen inexperienced people attempt to do that in non database code. And I already provided an example of that in the post that you responded to.

chrisseanhayes wrote:
No-Sql (like Cassandra), kvp (Redis) outside of crud operations

I don't believe either example you mentioned is intended to be a full persistence store solution. So not really sure what your point is.

chrisseanhayes10-Jul-23 8:00

chrisseanhayes

10-Jul-23 8:00

thank you,
everything you said supported what I said, you just slapped some "no"s in there to make it sound like your answer. kudos superman

jschell11-Jul-23 6:44

11-Jul-23 6:44

"every time you use sql server for more than crud then you are married to sql serve"

And that is still absolutism. Which in practical programming does not work.

jschell6-Jul-23 5:42

6-Jul-23 5:42

PIEBALDconsult wrote:
I have long sought a project to show off my ideas about implementing an SQL-based application which can be configured to use any one of a number of supported ADO.net providers.

Seems like a better demonstration would be to use a persistent store which is not a SQL Server variant. Such as MySQL.

PIEBALDconsult6-Jul-23 5:59

6-Jul-23 5:59

Once I have support for two, adding more should be easy.

Martin Hart Turner7-Jul-23 2:10

Martin Hart Turner

7-Jul-23 2:10

Since moving to .NET in 2003, we've been using SQL Server (free edition) and have had *VERY* few problems with it, even in multi-user scenarios.
There are moments, however, when a simple database would suffice, and for these programs we use SQLite. No previous installation required and usable on mobile platforms. I have, however, had several database corruptions (3 or 4 in the last 5 years or so).
Horses for courses!
If you can install SQL Server, I'd go with that, but for mobile platforms of data transfer using a single file, SQLite is a great alternative.

Hope this helps!
Martin.

Peter Adam7-Jul-23 2:45

Peter Adam

7-Jul-23 2:45

Same origin, two different products:

Firebird: Firebird 4.0.2, GitHub - FirebirdSQL/NETProvider: Firebird ADO.NET Data Provider

InterBase: Product Editions - Embarcadero

Steve Naidamast7-Jul-23 6:07

Steve Naidamast

7-Jul-23 6:07

Firebird has both a server-side configuration as well as an embedded on. However, it is a rather difficult database to use at times due to what I perceive as idiosyncrasies.

See my 2018 piece on how to work around Firebird's quirks... https://blackfalconsoftware.com/2018/05/11/the-firebird-database-engine-the-frustrations-of-the-long-distance-database-application-developer/

Your other option is to use the excellent SQLite Database Engine.

As it regards SQL Server Compact Edition (CE), I always thought that Microsoft made a serious mistake in dropping support for it.

Their replacement of it with LocalDB has never made any sense to me...

Steve Naidamast
Sr. Software Engineer
Black Falcon Software, Inc.
blackfalconsoftware@outlook.com

PIEBALDconsult7-Jul-23 6:18

7-Jul-23 6:18

Steve Naidamast wrote:
made a serious mistake in dropping support for it. Their replacement of it with LocalDB has never made any sense to me

Yeah, I do not see LocalDB as a replacement for Compact.

jkirkerx7-Jul-23 7:52

jkirkerx

7-Jul-23 7:52

I can that happening, having the app on a flash drive, plug it in, but have the app just use a cloud database service instead. AWS will give you a year free if you sign up a personal account for learning, and the monthly fee may not be that expensive after the year is over. But consider how much you can learn, and how awesome your flash drive app can be, by using a cloud database provider. Plus you can build in other cloud services as well. There are many choices of database technologies available to choose from, and lots of current support for them.

If it ain't broke don't fix it
Discover my world at jkirkerx.com

PIEBALDconsult7-Jul-23 10:47

7-Jul-23 10:47

NO! Not cloud FFS! No reliance on a network connection either.

jkirkerx7-Jul-23 11:43

jkirkerx

7-Jul-23 11:43

Oh....

Tin Foil hat stuff, sounds cool!
Reminds me of that movie "Enemy of the State"
SQLite, I don't like it, but could like it if I want to keep secrets.
That's something that could be monetized.

If it ain't broke don't fix it
Discover my world at jkirkerx.com