The Scottish Military Research Group - Commemorations Project

Adam Brown · Posted: Tue Jul 28, 2009 11:47 am Post subject: Loss of the Forum

This is related to the post David has just made requesting copies of photographs.

http://warmemscot.s4.bizhat.com/viewtopic.php?t=4787

At the heart of it is the risk we have of losing all the data we have if the forum hosts can no longer support the forum.

We have skimmed over this subject before but have never found a solution.
(see here for an example: http://warmemscot.s4.bizhat.com/viewtopic.php?t=3945)

The main problem is that we run this forum as a group of volunteers in our spare time and with no funding. Frankly we don’t have the time or money to come up with a quick win.

We are particularly at risk on two counts.

1. We use a photo-hosting solution to display our photos so if a host disappears we lose our photos
2. If the Forum hosts have an issue we will lose everything.

David’s request will cover us for risk 1. Risk 2 is not so easy. The genealogy forum ‘Talking Scot’ has been offline for a few weeks now and I believe they were taking backups of the full forum so it’s not going to be that simple to come up with a solution.

If we can come up with a list of possible solutions we can look into whether we can make them work or not. Any suggestions are welcome at this stage so even ones that may cost a lot of money should be put forward because we shouldn’t rule out anything just yet.

Regards

Adam

Adam Brown · Posted: Tue Jul 28, 2009 3:07 pm Post subject:

As you’ll have guessed I really have no idea what the solution to this is so I’m just going to start by jotting down a few questions we need to ask ourselves.

1. Do we need a backup of the data?

2. The UKNIWM in London has been set up to record the UK’s war memorials. Should we just make sure we pass everything on to the UKNIWM?

3. Can we take a backup ourselves of the forum so if there are any problems with Bizhat we can pass the data onto another forum host?

4a. Can we backup each thread as a web file?

4b. What do we do with saved web pages?

5a. Should we apply for funding to pay for a solution?

5b. If we apply for funding won’t we need a committee to manage it.

Adam Brown · Posted: Tue Jul 28, 2009 3:11 pm Post subject:

ADP · Joined: 13 Jun 2008 Posts: 467 Location: Edinburgh

TalkingScot took regular backups. When the site hosts shut down without warning and unable to be contacted we were astounded. We had two distinct jobs to do: (1) to retrieve the domain name which was registered through the website hosts, and (2) to find a new host and rebuild the forum (and non-forum webpages).

The domain name was released within a week, having applied to eNom, the registration people for .com domain names. eNom were very helpful, and guided us through the procedure to reclaim the domain name.

Finding a new host was not difficult. There are many of them out there. I already had dealings with a hosting company through another totally unrelated website I am involved in. So we went with that company. They had a suitable hosting package similarly priced to the deal we had with the previous host.

The main problem was rebuilding the forum. The software was fairly easy to download and set up. Then came the challenge - how to upload the backup. The new host uses a completely different control panel. There was no way to use the complete site backup as it was. I had to find software to decompress the backup and find the database file. mySQL would not take the whole backup in one piece. I had to dissect the SQL file into its individual tables, then upload it table by table using mySQL. Some of the tables were too large. They had to be split and uploaded in numerous bits.

It was a swine of a process that caused me a lot of grief. After taking much advice from an IT consultant friend the method above was devised.

If you have the funds and determination, I would suggest you try a disaster recovery. This would be to purchase another database space with your host and rebuild another forum from the backup. I wish you all the best if this is the route you go down. Without going down that route you will never know if your backup will do the job for you, or not.

The other job that we wanted to do at this time was to convert the forum to the current version of the phpBB software (3.0.5). However that software version ran so slowly that it was unusable. It would have put people off using the forum. So we stuck with the old software, which phpBB no longer support. As a longer term issue, we have to decide about more up to date software (supported by its writers). This may be some other version of phpBB or some other software package that the phpBB forum can me migrated into. That is another day's decision. In the meantime we are just happy to have the forum up and running again.

I hope this information gives you something to help you with your considerations about your forum management here.

ADP

Adam Brown · Posted: Tue Jul 28, 2009 10:11 pm Post subject:

ADP

Many thanks for such a detailed reply (and some good news about the TS forum too!)

It looks like a Disaster Recovery option is something we should be looking at in detail then. This would give us a backup of the forum itself rather than saving the data in another format.

Regards

Adam

David McNay · Posted: Wed Jul 29, 2009 8:29 am Post subject:

I should make it clear that as far as we know there is no danger of this forum disappearing.

However, things can and do change. Several internet service provuiders used to provide free web space and then withdrew it. Think of how many AOL-based websites (including Derek's on Stobs Camp) vanished.

Although there is no evidence of Bizhat (the providers of this forum) withdrawing their services it could still happen and we need to be prepared for this.

Jim · Joined: 30 May 2008 Posts: 195

Might it be worth approaching the National Libraries of Scotland or the Ministry of Defence? I appreciate both organisations face budget constraints but the cost of providing some form of back up or mirrored hosting for what is, after all, an invaluable national resource would be modest for each of these bodies.

If explained to them in the right way I reckon both would see that it's the sort of thing they ought to be doing Wink

Adam Brown · Posted: Wed Jul 29, 2009 1:38 pm Post subject:

Jim

Thanks for the input. I would suspect the MoD would point us in the direction of the UKNIWM.

The NLS is not something we have considered. I wonder if this is something they would be interested in. I don't know what their postion on online records is?

There is also the Royal Commission for Ancient and Historical Monuments of Scotland who operate the Canmore database but the information they record is pretty much what the UKNIWM currently records. In fact I was told several years ago that the UKNIWM were going to pass on details of their Scottish entries to the RCAHMS at some point so should we just make sure everything goes to the UKNIWM and it will eventually end up in Edinburgh.

Regards

Adam

Jim · Joined: 30 May 2008 Posts: 195

Adam Brown · Posted: Fri Jul 31, 2009 8:40 am Post subject:

I think this is quite a timely discussion. It looks like the Roll-of-Honour website run by Martin Edwards is now unavailable. There is an error message of 'Bandwidth Limit Exceeded' when you try and access it.

They have / had very few Scottish memorials listed but quite a lot of English memorials.

Regards

Adam

spoons · Posted: Fri Jul 31, 2009 8:58 am Post subject:

kinnethmont · Joined: 19 Dec 2006 Posts: 1649 Location: Aberdeenshire

Martin's site is fine at the end of August, probably a month end problem in July as Paul suggested.

It may be worth asking Bizhat how the data can be backed up or contact another party using their Forum system to see how they have approached the problem.
Such a backup could not account for private images hosted on photobucket, etc over which they have no authority or control.
_________________
Jim

If ye break faith with us who die
We shall not sleep, though poppies grow
In Flanders fields.

www.kinnethmont.co.uk