james
Posts: 2,205
Likes: 955
|
Post by james on Aug 14, 2015 10:31:15 GMT
One of the interesting things I've encountered at work is cases where database administrators have deleted all contents of the database server and all of its backups. For example, in one case all that was recoverable was a few weeks of transaction logs. In others, nothing. Startups are particularly vulnerable because there will often be a lot of control placed in the hands of one or two people. There's also the issue of ransomware, which goes in and encrypts computers and all connected storage then demands a ransom to decrypt it.
If you want to save your business and avoid having to try to work out who owns what part of what loans and who owns the cash, I suggest that you consider these approaches:
1. Regularly have backups made to unerasable media like DVD or blu-ray disks. Have someone other than the person responsible for making them verify that they are retrievable and contain current data. Since the media can't be erased it can't be deleted completely. This is extremely cheap and there's minimal excuse for not doing it.
2. Have a person other than the usual person maintain an off-site database server at another data centre (or at your home as a founder or similar) and use a replication or mirroring solution to have all data automatically sent to this server. Assorted solutions are available for the various database server types. Have backups regularly made from this.
3. Don't discard transaction logs, archive them instead. If all else fails you can recover from these.
4. Don't have one person keep the backup media. Spread it around so it's not easy to take and microwave all the DVDs.
5. Remember your data protection obligations, since these backups will normally contain personal information and might contain actual passwords if you're haven't moved with the times and are still storing them or encrypted versions of full passwords (I know of NO technical justification for storing even encrypted versions of full passwords, other than not wanting to spend what it costs to do it right). A box at home might need something like a connection to an external drive in a safe to be sufficiently secure.
6. Save reports of things like who owns what and balances, as well as database contents. You may not want to recover from these reports but having them beats not having them when you're desperate. Plain text has a lot of value when it comes to being readable.
7. Providing customers with regular statements can be useful, particularly if you email them (which some would assert is insecure) or, less reliably, make it easy to download them. At lest some customers would be able to show their balances and holdings at some point in time so it won't all be just a pile of money and loans with no records that you have to start from.
I hope we never see a P2P firm suffering from this but it does happen and you need a plan to make the data recoverable, somehow, in the face of an attack by one or a small number of insiders or outsiders. As a founder of even a tiny P2P firm a weekly backup taken home and locked in a safe can save a huge amount of trouble.
This post is not prompted by anything at any P2P company. It's just the sort of thing that I regard as the minimal requirements for not losing everything.
|
|
webwiz
Posts: 1,133
Likes: 210
|
Post by webwiz on Aug 14, 2015 10:40:13 GMT
I am interested, generally, in your comment on passwords. What is the alternative to storing them?
|
|
james
Posts: 2,205
Likes: 955
|
Post by james on Aug 14, 2015 10:50:30 GMT
Use a salted cryptographic hash of the password and throw away say a quarter of the hash. Throw away enough so that anyone trying to reverse the hashing will have a very large range of possible passwords. If the target is sufficiently valuable dictionary attacks on possibly matching permutations will be tried first, then just all possible permutations of letters, digits and punctuation in a target language/languages. The idea is to make this search area impractically big.
For cases where it is not desired to ask for the whole password, a very good idea, store the partial cryptographic hashes of a range of combinations. Say 15 or 20 different permutations of three characters from the password. Even storing and comparing just a byte or two of a cryptographic hash will be good enough to have high confidence that the correct three characters have been provided, so it's pretty cheap in storage terms to just pre-generate and use permutations like this.
When you've done that, what can you get if you break in and have full access to everything on all of the servers? That's the case that it has to be robust enough to withstand - including an attack by insiders. Email addresses, lots of financial and personal data, but not usable passwords that can be combined with the email addresses to get into popular webmail accounts and attack lots of other servers...
Cryptographic hashes because their designs mean that they tend to do an extremely good job of distributing variations in input data throughout the whole hash, so the number of bits/bytes you need to have high confidence of a good match is greatly reduced.
|
|
webwiz
Posts: 1,133
Likes: 210
|
Post by webwiz on Aug 14, 2015 11:20:22 GMT
Interesting, thanks for that. This would seem to give some protection against theft of stored data, but there seems to me to still be a vulnerability in the system because the password is still received in clear, or simply encrypted, so can be seen by insiders who would know how to decrypt it..
|
|
james
Posts: 2,205
Likes: 955
|
Post by james on Aug 14, 2015 11:29:10 GMT
Yes, there is vulnerability to catching the passwords as they are initially being received or later. That can in some cases be mitigated by having the client do the hashing and such before it even sends over an original password, though not all clients are going to be capable of doing this sort of work. So the host might never see a password, just a partial hash and partial hashes of various permutations. Of course for spear phishing that client side can itself be attacked sometimes. But I'm really usually more concerned about having every account on a server compromised than this sort of thing. |
|
markr
Member of DD Central
Posts: 766 
Likes: 426
|
Post by markr on Aug 14, 2015 17:44:57 GMT
1. Regularly have backups made to write-only media I've certainly used write-only media in the past (DAT tapes seemed particularly prone to turning write-only at the worst possible moment), but I wouldn't recommend it for backups. I suspect you meant write-once.
|
|
james
Posts: 2,205
Likes: 955
|
Post by james on Aug 14, 2015 18:35:19 GMT
1. Regularly have backups made to write-only media I've certainly used write-only media in the past (DAT tapes seemed particularly prone to turning write-only at the worst possible moment), but I wouldn't recommend it for backups. I suspect you meant write-once. Thanks. Oops. I changed that to unerasable. I didn't have good experience with DAT either.
|
|
jonah
Member of DD Central
Posts: 2,031
Likes: 1,113
|
Post by jonah on Aug 14, 2015 18:44:27 GMT
They used to be called WORM for write once read many. CD-R or DVDR. Cheap as chips (insert pun about computers here) these days. Not 100% reliable though, so I would suggest as great for a secondary, maybe offsite backup, but not as the primary or only one.
|
|
james
Posts: 2,205
Likes: 955
|
Post by james on Aug 14, 2015 19:17:18 GMT
Agreed.
Primary would be just a normal database backup and restore that would be used to recover from routine things like failed disk drives or natural or other disasters, like fires that destroy everything in the primary data center. Or for bigger places, a standard databse failover solution so the customers won't even notice if the problem is just in the database end with one server. For small places, with some database server software it's possible to have two or more copies of the database server running on one box and this can provide a quick and easy first recovery possibility if there's data replicating between primary and secondary copy. Won't help if the data centre burns to the ground but will help for lots of routine failures. Later the secondary can be copied to another box/data centre for a more robust solution.
|
|
markr
Member of DD Central
Posts: 766
Likes: 426
|
Post by markr on Aug 14, 2015 20:26:11 GMT
They used to be called WORM for write once read many. CD-R or DVDR. Cheap as chips (insert pun about computers here) these days. Not 100% reliable though, so I would suggest as great for a secondary, maybe offsite backup, but not as the primary or only one. I'm a hoarder of all sorts of computer junk; this little beauty took 1GB phase change WORM cartridges... 
|
|
webwiz
Posts: 1,133
Likes: 210
|
Post by webwiz on Aug 16, 2015 11:04:38 GMT
James, can you give similar advise to p2p/p2b investors? What steps should we be taking to protect ourselves in the event of platform failure or disappearance?
|
|
james
Posts: 2,205
Likes: 955
|
Post by james on Aug 16, 2015 23:45:58 GMT
James, can you give similar advise to p2p/p2b investors? What steps should we be taking to protect ourselves in the event of platform failure or disappearance? Keeping records of the contact information and individuals involved as well as your holdings and balances. FCA registration can help with that in part because there is some prospect that the FCA would look to assist in the process of tracking down those involved in any vanished platform. the regulatory capital requirements that come once a platform has been approved should reduce the chance of failure.
|
|
james
Posts: 2,205
Likes: 955
|
Post by james on Aug 17, 2015 11:41:30 GMT
While it's not specific to P2P here's a summary of what I do for personal backups, including my P2P information. There are four broad classes of information that I use:
A. My personal things that aren't large. Core financial information like account details and statements.
B. My personal things that are bigger like photos and videos that are not replaceable from other sources.
C. Things that I can replace with some hassle and that are typically big, like various types of media.
D. Operating system backups and similar that are for restoring but can be easily replaced if necessary.
A is periodically copied to an online source, from a download site to a gmail attachment. Encrypted for this. This is where I go if my home burns down and I'm robbed and everything there is lost. Copies are normally made of every file that is changed every day and those copies retained until I delete them eventually, periodically. You can also do things like giving a locked safety deposit box containing DVD copies to family members at different homes or use a safety deposit box.
A and B are always primarily stored on removable media, like a USB stick or USB SSD drive. This is so that I have very easy access to the things that I need day to day if the main computer fails, all I have to do is move the device to another one. There's at least one other copy of this because it's small and that one isn't attached to any computer, for ransomware and burglary protection.
A and B have copies made in compressed 7-zip form twice a month and copied to two drives on the main computer and one drive on a network-attached computer. These are retained indefinitely to provide go back in time capability. They are also copied to the removable media used for A and B.
D is done periodically and stored on at least two disks.
A, B, C and D are mirrored on to a USB attached drive that is normally turned off to protect against ransomware issues.
The external drive that I use for mirroring is currently a Seagate Backup Plus Desktop Drive, 8TB. Cost around £200. Inexpensive and so much storage that I don't need to wonder or worry about space. I could have gone with an external RAID device but it just didn't make economic sense compared to this.
My system is mainly Windows though I use linux at work. Windows has moderately easy to use mirroring software in the form of robocopy.
robocopy.exe D:\ G:\robo\mirror /MIR /FFT /Z /W:5
will make a complete copy of everything on the D drive to the G drive directory robo\mirror . Run it again and it'll update the mirrored copy, adding new files, removing old ones and updating changed ones. This one is similar but it'll do nothing unless the G drive is attached:
if exist G:\robo\mirror\*.* robocopy.exe D:\ G:\robo\mirror /MIR /FFT /Z /W:5
Put that in a file called makemybackup.bat and it'll do the backup for you whenever you double-click on it. I have this run routinely to make mirrored copies of the removable drive that holds a and B data to a remote network drive on a different computer and two local drives. I could add the big 8TB drive but I handle that a little differently.
You don't have to use a whole drive, you can replace D: with some drive/directory combination if you like. I stick various types of B and C information in specific directories and mirror those to the big external drive. There's no fancy software involved, all of the files in the mirrored copy are available at any time and robcopy has been built in to Windows for many years.
I also have a backup program called fbackup making copies of A and B regularly. As usual to three different drives, two in the main computer and one in a remote computer on the network. Windows has its own equivalent of the Apple Time Machine software in version 8 but I don't use it.
My next step may well be looking into fire-rated safes that can have a drive kept in them and attached to a network. So the burn down or burglary case can be handled more easily.
I've had many drives and computers fail over the years. It's just something routine that you have to expect and be able to deal with when it happens. Lots of belts and braces. I dislike losing information to predictable events with only the timing being uncertain.
|
|
