Major breakdown on Cactus-Mall web server

For the discussion of topics related to the conservation, cultivation, propagation and exhibition of cacti & other succulents.
Forum rules
For the discussion of topics related to the conservation, cultivation, propagation, exhibition & science of cacti & other succulents only.

Please respect all forum members opinions and if you can't make a civil reply, don't reply!
Locked
bcss
BCSS Publications & Web Manager
Posts: 228
https://www.behance.net/kuchnie-warszawa
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Major breakdown on Cactus-Mall web server

Post by bcss »

At approximately 4pm this afternoon the Cactus-Mall server went down. They tell me both SSD drives in the server failed simultaneously (sounds improbable). This affects about 100 cactus related websites including some highly used ones but not BCSS related sites which are hosted on a different server. It will also render quite a lot of email systems to fail. This is going to be very difficult and laborious to fix.
Tony Mace
Publications Manager
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

Further advice on Cactus-mall server breakdown. Fasthosts engineers are looking into how to restore services but solutions may take some time. Meanwhile if you you need to contact Suzanne or Tony Mace please use bcss@bcss.org.uk which is mostly still working. Mail to cactus-mall.com, paperweight-mall.com. mesemb.org.uk, asclepiad-international.org will not get through.
Tony Mace
Publications Manager
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

I have set up temporary solutions for the following:

www.mesemb.org
www.asclepiad-international.org

Suzanne is now contactable at msg@mesemb.org or treasurer@asclepiad-international.org.

We are still working on the main server problem.
Tony Mace
Publications Manager
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

A server rebuild is currently underway. This is taking longer than expected. Once this is complete all the backed up data (over 1Tb) will have to be copied back to the server before website restoration can commence.
Tony Mace
Publications Manager
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

The server operating system has finally been restored by Fasthosts this morning at 8.30am. It has taken fractionally under 2 days after various glitches in the restore system. I have access to the system. We now have to restore all the backed up data to the main drives. Likely to take about 8 hours, after that we can start restoring websites and other services.
Tony Mace
Publications Manager
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

I am afraid the file restores have gone very wrong. Files corrupted. Big setback. Yann Cochard and I working on it.
Tony Mace
Publications Manager
User avatar
Tony R
Moderator
Posts: 4016
Joined: 20 Apr 2009
Branch: CAMBRIDGE
Country: UK
Role within the BCSS: Member
Location: Hartley, LONGFIELD, Kent

Re: Major breakdown on Cactus-Mall web server

Post by Tony R »

Apologies too, to the Haworthia Society members who are currently unable to access their website http://www.haworthia.org/
Tony Roberts
Treasurer, Haworthia Society
Chairman, Tephrocactus Study Group
Moderator, BCSS Forum
Kent
(Gasteria, Mammillaria, small Opuntia, Cleistocactus and Sempervivum are my current special interests)
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

The file corruptions from yesterday evening have been fixed. There are still some configuration errors to be resolved but I hope we will see some sites restored tomorrow.
Tony Mace
Publications Manager
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

Quite a number of sites are now back on line. There are still residual problems assicaited with mysql logons and php include files, we hope to have most of these fixes during the next 24 hours.
Tony Mace
Publications Manager
bcss
BCSS Publications & Web Manager
Posts: 228
Joined: 11 Jan 2007
Branch: BRISTOL
Country: UK
Role within the BCSS: Website Administrator
Contact:

Re: Major breakdown on Cactus-Mall web server

Post by bcss »

Most of the problems I am aware of have been cleared up. However I sure there may be some I have missed. At this point I would welcome an error reports (and you can now use cacti@cacti-mall.com again).
Tony Mace
Publications Manager
Locked