Looking after your data

There is a phrase regarding security on websites that terrifies developers: “It’s not a question of whether you get hacked, but when.” It’s popular with the media of course―doom-laden articles will always sell, and strictly speaking this terrifying sentence is correct―but only for a given value of ‘correct’.

For example, I don’t think anyone could stop the NSA getting access to private data if the NSA had a thousand years and warehouses of dedicated brute-force script kiddies at their disposal (no doubt backed by law). What we can do however, is make cracking data so bloody hard that the whole process becomes untenable. For LDC Via, we consider the “base data type” for all migrations to be financial transactions from large corporates―in other words, we do not play fast and loose with this stuff.

As we built LDC Via we had to consider the usual suspects in terms of attack vectors, and the defensive layers we need to provide. This is compounded by the fact that IBM Domino has always provided such good, simple security. We provide various options when using LDC Via, from simple data conversion to fully-converted applications with reasonably complex security requirements―and so there are appropriate decisions to be made.

Thankfully, different members of LDC take care of different areas of the application, and this enables us to impose a global security policy with each member taking care of a number of the constituent layers.

We break down the defensive layers into two categories:

1. “The Usual Suspects”
2. “Just For Us”

###“The Usual Suspects”### These are the layers that all web apps have to deal with. Quite a lot of them are irrelevant for on-premises solutions, but take on huge importance in our cloud offering.

Ports and firewalls. An easy one―only those ports absolutely necessary are opened. By default this means SSL (port 443), with the option of port 80 if non-secure transactions are required. That is it: no database ports, no remote access ports etc. We separate dedicated client servers from each other with firewalls as well, so there is no inter-communication between your dedicated boxes and those used by our other clients.

Operating system. We use a hardened Linux build for our production servers, with proper patching and all the things that keep administrators happy.

Hardware. Where provided by our hosting providers we offer hardware encryption. Now that most decent physical storage is based on solid-state technology, we have found that hardware encryption is not the performance horror it once was. If this is a requirement for your implementation, please tell us.

NSA. A new player in the security arena is the National Security Agency in the US. Their curious assertion that they have the right to take any data they deem fit for their purposes from anywhere on the planet means that even if your data is not stored in the continental USA, if the hosting provider has an office registered there then the NSA can pressure that organisation to provide access to your data (with the hosting provider and LDC Via legally obliged not not to tell you about it).

Well, we are registered in the UK, and we have access to multiple hosting providers, both local and international, to meet the different data security needs of clients.

Databases. All databases offer a variety of security options, and the methods we implement will vary according to the back-end server involved. In our current offering (based on MongoDb) we use both standard MongoDb users and roles for security (rather than those wide-open “service accounts” that so often lead to wholesale database leaks in the case of a security breach).

However, that’s not enough: MongoDb only really offers granular security down to a “collection” level, so that leads us on to the the next level of security…

###“Just For Us” Readers and authors. One of the core tenets of IBM Domino security, and something that is a devil to reproduce in other systems. Readers and authors fields, controlling document- and field-level access to data. Very few systems and databases provide document-level security, so to effect this level of security, we have a “data wrapper”: we have constructed a wrapper that goes round any attempt to access the database from the application (a Java driver wrapper is also provided). The meta-data stored in the database is checked against the requesting user and the various levels of access are granted on that basis. This includes group access as well as individual rights.

Configurable security. At any time, administrators can modify which fields in a collection are used to define security, and of course, will always be able to see all documents (you won’t lose anything!)

So there you go, some of the background to security in LDC Via. As Mark says, it’s like arguing about politics: never-ending, often quite heated, but one day we will build a better world (if the rest of the buggers stop talking rubbish).

More on the why...

We’ve spoken about the whys and wherefores of LDC Via already, but some of the use cases we mentioned in our last post probably bear some expansion. So let’s do that.

1. Re-homing historical data from servers that are being retired A number of organisations maintain Domino infrastructure purely so that they can reference data held thereon—no new development, no real activity, but they need to “keep the lights on”. These servers may be poorly-supported, may not have robust back-up, and can even be running out-dated or un-patched infrastructure (I think many of us have heard about Domino 6 or 7 boxes still running in production). So why not kick those boxes into touch and serve up your data in a modern, cost-effective manner (with support)?

2. Archiving data from Notes & Domino applications that have grown too large for Domino There are all manner of tips ’n’ tricks to make your applications scale. Ultimately however, there are some limitations in our venerable platform, and it is not the solution for every hosting use-case.

The mark of a successful Domino application is one that has lasted years and become overloaded with documents. Why not give the old app a new lease of life? Move historical data off to a platform like LDC Via, and let your Domino application’s view indices breathe once more.

(Incidentally, if this use-case is especially of interest, do look out for our LDC Via-XPages integration demo on this very site)

3. Integrating Domino-based data with other applications Domino can be a bit of a silo. Sure, there are ways of surfacing its data in other environments, but that can get tricky fast. Pushing selected data to LDC Via, with its REST API layer and pleasing user interface could be a solution here. All modern application platforms play well with REST-ful systems, and Via provides a platform for customisation and extension as you see fit.

4. Selectively making Domino-based data available outside the firewall without exposing your Domino infrastructure A lot of your data and tooling sits behind the corporate firewall, and that’s for very good reason. However, you may have a business requirement to open up a sub-set of your data to selected third parties. Why not migrate that data directly to Via, and use its API to access it the way you want? A simple, modern user interface comes out of the box, but you have the option of coding your own functionality using our REST APIs and your Javascript frameworks of choice.

5. Migration of Domino applications to other stacks for increased interoperability and performance Current environments include the MEAN stack, and this has a vibrant user-base. Why not take advantage of these skills and opportunities? This means granting Domino data access to a new generation of developers.

6. Exposing ‘IBM Notes’ application data to non-Notes clients (e.g. web and mobile) As per item #4, a lot of corporate data may only be accessible via IBM Notes clients inside the firewall. You could Domino-enable such applications in a bid to move away from fat clients. Or perhaps your organisation has a target platform for projects that doesn’t include Domino? Well, LDC Via could be an option.

Beta Program

You’ve heard us talk a little about the Why of LDC Via and hopefully that’s got you interested. But we’re very aware that what we’ve created is a significant change from running your Domino server.

What we need are people to kick the tyres of our NSF migration tools, our admin interfaces, security and also the development API. If you think you’d like to be involved then get in contact with us (some of you already have, thank you!) If you’ve been involved with a beta program before then the process will be familiar, we’ll get you set up with our beta environment, help you get started and then let you play. In return we’ll be asking you to give us feedback, as brutally honest as possible, about the good, the bad and the things that need to be changed before LDC Via is production-ready.

In return for all of your effort you will get our eternal gratitude and very preferential pricing when we switch from beta to live.

Here’s a little more detail about the three main areas of LDC Via we want you to look at:

First we have the migration process. This is split into two parts, either our online tool which will talk directly to your internet facing Domino server, or the on premises tool which acts as a conduit between your internal Domino server and our cloud based servers. We want to be able to migrate your full Domino databases and make sure that we are getting 100% fidelity. We’ve obviously done significant testing of this ourselves already, but we can always do more.

Second we have the admin interfaces. These allow you to see your data. Think of the main admin screen as the equivalent of an “All Documents” view combined with the “Document Properties” dialog box (only more useful, hopefully!) You can manage user accounts and security for your organisation too, which includes modifying document-level security settings.

Finally, for the developers out there, we have our API. This is what allows you to create new applications or integrate with your existing applications and access your data in whatever way you need. Here we’re wanting to make sure that the APIs we have created are sufficient, and to identify new APIs that will make your life easier.

So if all this sounds like it’s something you want to get involved with, please let us know and we’ll add you to the list. We aim to be starting the process in the next few weeks.

Why oh why oh why (Part Two)

In part one of these posts we took a look at why we created Via. In this post let’s explore what that means for you.

First, let’s revisit what Via is, in a nutshell:

• A modern hugely-scalable and highly-performant database platform
• A secure application layer that gives you all the Readers / Authors / Roles / Groups goodness you’ve come to love in Domino
• A set of tools to migrate Domino data into Via

Sound good? So where does that fit in your plans? Here’s where we see Via as a viable (see what we did there?) future for your Notes and Domino databases:

• Re-homing historical data from servers that are being retired
• Archiving data from Notes/Domino applications that have grown too large for Domino
• Integrating Domino-based data with other applications
• Selectively making Domino-based data available outside the firewall without exposing your Domino infrastructure
• Migration of Domino applications to the MEAN stack for increased interoperability and performance
• Exposing ‘Lotus Notes’ application data to non-Notes client (e.g. web and mobile)

The list is endless. Well, long anyway.

The NSF data structure is a truly excellent piece of software engineering for its time. And it was way ahead of its time. But the rest of the world has finally caught up and, arguably, overtaken. Except in the area of security, where NSF’s Readers / Authors security model is still unique in its power and elegance. Or, at least, it was until now: LDC Via.

Why oh why oh why (Part One)

We thought it would be helpful to explain why we’ve created LDC Via, and why you might want to use it.

Between us at LDC we have about 70 years of experience working with IBM (Lotus) Notes and Domino. We’ve been there through all the ups and downs and ins and outs of the product’s history since version 3 back some time in the late 20th century (when you were a mere lad or lass, of course).

What we’re seeing now is that Notes and Domino are viewed (whether correctly or not—that’s a separate discussion!) as being somewhat old-fangled, uncool and—that horrible word—“legacy”. The upshot is that some organisations are reducing, or even stopping, investment in this platform. For email they may well be migrating, or have migrated, to Microsoft or Google. Those that are staying with IBM may be considering moving to IBM SmartCloud for email. So, for many, the on-premises Domino servers have become a liability, a problem, and an unwanted overhead. But—and it’s a big but—there are important business records and data locked away on those servers.

So that’s the first reason why we built LDC Via. We believe that many organisations need a route away from Domino as a strategic data store. That may not mean they’re dropping Notes / Domino entirely, of course. But it may mean that they want to downscale or eliminate internal infrastructure, it may mean that they want to retire the Notes / Domino applications but have the data available, or it may mean that they want to redevelop the applications on an alternative platform.

Alongside this gradual shift in focus away from Domino, has been a recent and fairly rapid rise in “NoSQL” databases. The irony, of course, is that Domino’s NSF data structure is itself a NoSQL database. But it’s an ageing one, with undoubted issues of performance and scalability when compared to some of the younger contenders. And with the rise of these databases has been a parallel rise in acceptance of new development architectures to make the most of them, most notably the “MEAN stack”. Here at LDC Via we opted to develop first against the MongoDb back-end, as that is arguably the most accepted and widespread of these modern NoSQL databases.

Where the Domino NSF database has always been a market-leader is in its security structures. Readers fields, Authors fields, encryption keys, database encryption, etc. There just isn’t anything else like it. Except, well, now there is: LDC Via.

We hope that’s given you a glimpse into why we’ve built this product. It’s because we believe you need it. In part two we’ll explore why we believe that.