Progress on MLP3 - April 27th, 2011
The Encrypted File System
TrailsWeb's loan servicing program, Moneylender Professional, uses it's own proprietary file structure to store data. In MLP2 there are some design flaws to the security of the software that have been addressed for MLP3. The new system was written from scratch to handle reading, writing, encrypting, decrypting, indexing, index validation, reindexing, foreign key indexing...
It's built around a dynamic record structure with the intent of allowing users to add and remove fields directly to and from the records. In MLP2 there are some custom fields available for users to relabel and use as they see fit. If a user needs more than two extra fields on a loan, I had to make an attachable ExtendedFields record that was bound to the loan.
In MLP3, if you need a field, you can add it to the definition of the record. For example, you can add a "VIN" text field to your loans to identify the collateral vehicle, or maybe a "Student ID" to help match records with an existing data store. In any case, MLP3 will leverage the flexible record structure to make become a truly dynamic and form-fitting system for each lender.
Once my new code was usable I wrote a program to read and write to the file repeatedly for benchmarking. First I rean consecutive writes followed by consecutive random reads from all the records in the file. Once I had identified as many "slow patches" in the code as I could, I starting running multithreaded writes, reads, and rewrites concurrently to make sure the file system would hold up in a busy server environment.
My computer might have been pretty decent two years ago, but now it's just average. I still got very promising results from the benchmarking. Records with sizes around 32 to 64 bytes and limited complexity could be created, encrypted and written in .1ms or less. Those same records could be read, decrypted, and put into a field-by-field structure in about .01ms.
As the record size increased the encryption started to take its toll. Very large records, in the 400,000 byte range, took about 26ms to write and 52ms to read. I wanted to test the extremes when benchmarking the file system. Records of this size are rarely needed, but nonetheless my structure can churn them out at a rate of about 20 per second.
Then I started running concurrent writes, reads, and reads-followed-by-modification-and-overwrite. Most of my tests used 5 writing threads, 10 reading threads, and 3 rewriting threads. The throughput went up substantially in a multithreaded environment, and all threads had excellent performance times, sometimes better than the sequential tests. I happily noticed all four cores of my processor chipping in with that many threads running, as opposed to about 1.5 cores being utilized in the sequential tests.
And of course, the data is thoroughly encrypted. Every single byte of the file is run through a 256bit cryptography algorithm before being written to disk. If you don't have the password, you're not getting access.
Unintentional Obfuscation System
When testing out some of the obfuscators that are available for .NET code I found almost none of them would even try to obfuscate my software. Most of them saw things defined as public members and didn't want to break functionality. When I would decompile th e"obfuscated" code, it was exactly the same as before obfuscation. And that's just not good enough.
So I wrote my own obfuscator. I pick my assemblies and my obfuscator merges them with ILMerge, runs ILDasm on the result, parses the intermediate language to pick out all the classes defined in the file, starts at the entry point and identifies all code required to run the program, renames everything defined in the file regardless of scope, writes only the code that actually gets run at some point into a new intermediate language file, and complies it with ILAsm back into a usable executable.
Without thorough obfuscation it's pretty easy to crack the activation on just about any piece of software. At least I know I'll have a fighting chance of getting paid for my work with some decent, rigorous obfuscation in place.