In nearly every project I have ever worked on I have needed a database. Most require a large robust database that
can be backed up, reported off of and able to have hundreds of connections at a time. In those cases a database
like SQL Server, Oracle or MySql fit the bill nicely. However, some projects require an embedded solution.
IoT, Apps and sometimes even little web apps really only need something simple and easy to implement. Initially
I started using SQL Server Compact for this but it became apparent that Microsoft has no plans to continue
development on it. Probably because it was competing with SQLite for the same space and SQLite already
came built in on all mobile devices. So then I tried SQLite, at first it was a pain because it was not
supported by Entity Framework, which is my prefered ORM. Then when Entity Framework Core was released
I implemented it into a project only to find that some of its’ dependencies could not be used with installer
projects or I had to do extra work in the installers to make it functional. And even worse was when I updated some
of the dependancies and the project would no longer build.
So I bravely (foolishly) decided to create my own database from scratch. I spent weeks reeding all the best practices
I could find. I Read all the documentation on all the major relational databases trying to find out how and why they
implemented pieces the way they did. And after all that research I decided on these requirements.
- It has to be able to run anywhere. For this reason I decided to develop it in .net Standard.
- It had to be ACID. (You can look that one up if you don’t know what it means.)
- It had to have a NoSql interface.
- It had to be as easy to use as Entity Framework.
- The data and the logs had to exist in one file.
- It would need to support indexing on multiple columns.
- It would need to support BLOBs.
- It had to be thread safe.
Six months of nights and weekends later I had a working database. I have created hundreds of unit tests to validate
each little piece of the program. Then there are hundreds more integration tests. And finally several project I have
used as dog food tests to make sure everything worked as expected.
This was a very long and difficult process and looking back I think I was crazy for attempting it. But now that it is
done I want to make it available to anyone looking for a good embedded database.