Forward progress.

Looking at making further progress on my DB engine. In no particular order:

  • Case insensitive string indexes. DB needs to be able to handle changes in index order.
  • Removal of current index tag scheme, with more complete “index combobulation” info, able to handle more than just A-B buffering.
  • Complete transactionality, where metadata and data changes can be done in one user transaction: each user transaction consists of multiple changesets. It should then be able to roll forward / roll back multiple sets of changesets to support existing commit / abort functionality. (Pipeline depth).
  • Support for multiple transactions in progress at the same time: instead of A-B buffering, we have Current (Next1, Next2, Next3) buffer sets. Amount of concurrency can be specified at transaction start time. (Pipeline width).

Once I’ve got all that done, then it should be possible to wrap the local DB logic into a distributed database. This will require a “journal of journals”, indicating the various transaction history paths that each each copy of the database has gone through. Depending on consistency level required, you could then do “last write wins” or “cascading abort” semantics, depending n what you want to do.

Check-in app now online!

Hi guys!

So I have now made a little check-in app. It asks you to register your e-mail and a contact e-mail. Once you’ve done that, you need to login or use the quick check-in link once per day. If you don’t (maybe coronavirus has got you, or you’re not well), then it automatically sends a mail to the contact address you specified.

You can try it out here.

N.B Some web browsers will complain about “invalid security certificate”. I like encrypting and keeping your passwords safe, but can’t be arsed to register the website certificate just for a demo, so if you want you proceed, click “Advanced” or “Details”, and then “Proceed to website”.

KnowComment Project progress…

So, I’ve been working on my latest top secret project. In summary:

  • Progress pretty good, but didn’t do much work on it over the hols.
  • New bespoke database engine pretty good (definitely fast enough!)
  • It does Instagram and filters posts and users quite nicely.
  • A bit more work needed to present tweets & twitter data: They’re in the database engine, but more work needed on the display.
  • After basic filtering is done,  working on more advanced analytics: graphs of who posts / replies to who, volumes of posts, hashtags etc.

MCHBackup

A while back I wrote a backup application which allows you to compare folder structures on different disks. I still use it, and it’s useful for keeping Dropbox folders, USB keys etc. in sync.

I haven’t written much documentation for it, so you’ll just have to try it carefully. Best to have a go with some files that aren’t utterly vital, as it’s perfectly possible to delete lots of stuff. 32 bit MSI, and 64 bit MSI available for download.

 

PageMapper

To showcase the parsing code I’ve written, I now have a demo app which allows you to view the structure of pages, not unlike the debuggers in MS Edge, Chrome etc. For example:

It’s not perfect, but it does pretty well on most pages, and I am still improving the underlying code.

If you’d like to get hold of the source code or like the parsers as a library, then feel free to contact me, and we can discuss your educational or research use, or negotiate licensing terms.

Here are the 32 bit MSI and 64 bit MSI installers.

An alternate Streaming System.

Delphi has fairly good support for RTTI (reflection). Quite a few of my programs need to stream sets of classes to disk, and read those classes back. This library leverages the RTTI system to provide automatic streaming of classes and published properties, with an extensible API to perform custom streaming.

The source is here. No documentation I’m afraid, you will have to read code and think.