The 3 most important things we learned developing a Blockchain application.

One of our primary goals with ProvenDB was to create a platform upon which developers could create their own Blockchain integrated applications with the same ease they can currently develop directly for a database like MongoDB. To demonstrate how such an application could be built we decided to create ProvenDocs.

ProvenDocs is a document storage system in which the integrity, ownership and creation date of your documents are reliably stored on the Blockchain. ProvenDocs is free to use, open source, written using common web frameworks and uses only the features and functions available to any ProvenDB user. Fundamentally, ProvenDocs is us putting our money where our mouth is when it comes to ProvenDB being the platform to build Blockchain enabled applications.

It will serve as an example of what can be built on this new Blockchain enabled database and we’re hoping you will use ProvenDB to build bigger, brighter and better things in the future. Here are some of the key learnings we had from building a Blockchain integrated application from the ground up.

1. Most people who think Blockchain, think Cryptocurrencies.

…we had to state clearly, in plain words and quite often, the concrete benefits that the Blockchain actually imbues on our application, without the buzz words or technical language.

Although almost everyone has heard the word Blockchain, many people still only know it as some underlying magic that allowed Bitcoin to explode into the media as a new commodity. Many people both in and out of technical circles possess only a surface level understanding of what the Blockchain must offer beyond making some people significantly richer (or significantly poorer) overnight.

Some Facebook automated algorithms even assumed we were a cryptocurrency just for mentioning Blockchain in our descriptions, despite us having no token associated and being a completely free to use service!

This meant we had to state clearly, in plain words and quite often, the concrete benefits that the Blockchain imbues on our application, without the buzz words or technical language. It wasn’t enough to list Blockchain characteristics like immutability, we had to make sure we had documentation, clear language and relevant imagery wherever possible. We also wanted to educate at every opportunity so our users knew what they were getting. This lesson manifests itself all over our product, but we think it is best demonstrated in our “Litepaper”.

2. The Blockchain is complicated.

…we have worked where possible to ensure that users don’t need to trust us or our knowledge.

It’s a valiant goal to educate and simplify the Blockchain for our customers, but that’s not worth much if we didn’t understand it inside and out ourselves. There are a lot of new concepts with Blockchain technology, almost all of which our application would touch in initial or expected features, therefore we had to make sure we had theoretical and practical knowledge to match. It wasn’t enough just to understand what a Merkle Tree was, we had to be able to construct, deconstruct and analyse one using code we had written ourselves. Because Blockchain is an emerging technology, we couldn’t just rely on NPM packages or tutorials to tell us the right way to create a document proof.

With a new and complex technology like Blockchain, we had to be more than sure that we were doing the right things, the right way. Additionally we have worked where possible to ensure that users don’t need to trust us or our knowledge.

Some examples of this include:

• Duplicate implementations — Some of our core logic (like hashing) were written multiple times in different programming languages. These duplicate implementations are running as part of our automated tests to ensure our algorithm is consistent.
• Verification tools — Publicly downloadable tools that verify an exported proof or set of data against the Blockchain so users can be sure (independent of our service) that their data is proven on the Blockchain.
• Open Sourcing (where possible) — We’ve open sourced some of our tools and implementations to ensure that any knowledgeable developer can check our logic, suggest improvements or recreate the important parts if you needed to. We plan to open source more SDKs and tools in the future to help developers get the most out of ProvenDB.

3. The Blockchain makes a slow and expensive database.

Users expect to be able to complete a transaction almost instantly without any cost.

At the time of writing this article it would cost me around 75 cents to post something directly to the Bitcoin Blockchain within the next 30 or so minutes. If I wasn’t in a rush I could wait perhaps twice as long for only 25 cents. This may at first seem like a relatively small time to wait and a pretty low cost to get something immutably placed on the Blockchain. However, there are a few factors that make these numbers much less appealing:

• A common use case for the Blockchain is proving that something existed in some form and at a specific time. If that specific time is consistently delayed by up to an hour the value of the proof can be significantly reduced.
• The Blockchain has sometimes been called the worst database in the world but it’s undeniable that there is demand in the market to use it like one. A database that costs 75 cents to insert new data once an hour seems unlikely to succeed.
• The cost of a transaction is related to the price of bitcoin (or some other token), which can be highly volatile.

For ProvenDocs, we knew we would have to combat these problems. Users expect to be able to complete a transaction almost instantly without any cost.

To address the speed, we realised that although users won’t wait an hour for their document to upload, they are happy (in most cases) for the proof to be a little slower, if the feedback is instant. So, we focused instead on making sure users were aware of delays in a proof by giving them visual feedback and diagrams to explain the process.

For a small start-up we had to make sure we could scale well without our transaction costs eclipsing our budget. We couldn’t afford 75 cents on every single upload and we didn’t want to enforce severe limitations on users that might prevent them gaining value from the application. What we decided to do was essentially a type of batching. Without getting too deep into the technical details, each individual document proof was merged with other document proofs, this process was repeated until we had a single “root” proof that we could put on the Blockchain. Instead of 10,000 document proofs costing us thousands of dollars, it could cost us hundreds. On top of our batching, we also used a service called Chainpoint to batch our “root” proofs with others. The overall effect was to significantly mitigate the cost for our end user.

You can read more about this process in our Litepaper.

Of course, we learned a lot more than just these three lessons while building an application with Blockchain characteristics. But these were three of the main things that any developer creating an application on the Blockchain must consider.

The good thing is if you’re a developer reading this now with an interest in Blockchain apps, we’ve tried to do some of the hard work for you! These lessons and many more contributed to the design and implementation of ProvenDB in the hopes that making a Blockchain app in the future will be as easy as making a traditional MongoDB application is now.

If you’re building a Blockchain application and have thoughts, feedback or suggestions (either for this article or for ProvenDB) please leave a comment below, we’d love to hear from you! Thanks for reading.

ProvenDB integrates MongoDB with the Bitcoin Blockchain. Immutable versions of database state are anchored to the Blockchain, delivering an unparalleled level of data integrity. ProvenDB allows MongoDB developers to build high-performance applications that include cryptographic proof of data integrity and provenance without having to understand Blockchain programming complexities.