Results 1 to 5 of 5

Thread: Aaaand we're back

  1. #1
    Administrator Rob Galanakis's Avatar
    Join Date
    Jun 2008
    Location
    Reykjavik, Iceland
    Posts
    1,242

    Default Aaaand we're back



    August 4, 2012 Update:
    ------------------------------------------
    Alright sorry for the downtime (again). We should be good now.

    Basically, on Wednesday or so, the server became basically unusable- besides just being slow, the host was trying to solve the RAM problems, which meant frequent restarts, which combined with the slowness meant lots of downtime. On top of that, response from their Tech Support was abysmally slow in some instances. So we just kept the server down until we found a solution.

    The solution, for now, was creating a new server with the same host, which they finally set up late last night. Dr. Bob and I stayed up into our respective wee hours of the morning setting it up and fixing the issues that came up. Fortunately we had backups of most things, but getting anything off the old server (such as our relatively short backup scripts, which ironically weren't backed up) was a pain.

    The site should be working now, with no lost information or unplanned outages. The Wiki is down right now but should be up before the end of the weekend.

    Fortunately we're more prepared now (hey, this is really our first sys admin gig), with backups of things we missed, more documentation of how to set up a server, and the acquired knowledge one inevitably gets, like deleting some MySQL log files can fix a slow Workbench startup... If we keep running into problems with this host, we should be able to migrate to a new server relatively easily.

    Original post:
    --------------------------------------------------------------
    Sorry for the extended downtime. Fortunately we didn't lose anything and it wasn't our fault.

    Our host had a server migration on July 27-28, and the tech-artists.org server went down late on July 28th (I have been on vacation the past few weeks, so haven't been checking TAO or my email much/at all). Once I was told the server was down, I tried to restart it, to no avail. Due to super-slow customer service, it wasn't until about 12 hours later our host fixed their issue, and I was able to RDP into the server. However we found a poorly performing server and a broken website. After a full day of tweaking things on a super-slow server, Dr. Bob finally solved the problem and the site is back up. Don't ask us what the problem was, we don't know, but it has something to do with PHP and IIS.

    However we still have a terribly performing server due to some phantom RAM usage, so the site may be slow or go down unexpectedly. We'll keep you updated, and I'll post on my G+ page and TAO's twitter feed if the site is down.

    In other news, I completely forgot to honor TAO's 4th birthday. The site is more than 4 years old!
    Last edited by Rob Galanakis; 08-05-2012 at 05:07 AM.
    Forum Administrator
    www.robg3d.com

  2. #2
    struct btribble's Avatar
    Join Date
    Feb 2009
    Location
    SF Bay Area, US
    Posts
    360

    Default

    Fixing something that was broken even though you don't know why it was broken, or how exactly you fixed it is the normal state of affairs for tech art.

  3. #3
    Administrator Rob Galanakis's Avatar
    Join Date
    Jun 2008
    Location
    Reykjavik, Iceland
    Posts
    1,242

    Default

    Alright sorry for the downtime (again). We should be good now.

    Basically, on Wednesday or so, the server became basically unusable- besides just being slow, the host was trying to solve the RAM problems, which meant frequent restarts, which combined with the slowness meant lots of downtime. On top of that, response from their Tech Support was abysmally slow in some instances. So we just kept the server down until we found a solution.

    The solution, for now, was creating a new server with the same host, which they finally set up late last night. Dr. Bob and I stayed up into our respective wee hours of the morning setting it up and fixing the issues that came up. Fortunately we had backups of most things, but getting anything off the old server (such as our relatively short backup scripts, which ironically weren't backed up) was a pain.

    The site should be working now, with no lost information or unplanned outages. The Wiki is down right now but should be up before the end of the weekend.

    Fortunately we're more prepared now (hey, this is really our first sys admin gig), with backups of things we missed, more documentation of how to set up a server, and the acquired knowledge one inevitably gets, like deleting some MySQL log files can fix a slow Workbench startup... If we keep running into problems with this host, we should be able to migrate to a new server relatively easily.
    Forum Administrator
    www.robg3d.com

  4. #4
    Administrator Rob Galanakis's Avatar
    Join Date
    Jun 2008
    Location
    Reykjavik, Iceland
    Posts
    1,242

    Default

    Also, please post here if the site seems slow.
    Forum Administrator
    www.robg3d.com

  5. #5
    Administrator Butters's Avatar
    Join Date
    Aug 2008
    Location
    Vancouver, Canada
    Posts
    249

    Default

    Wow what an "Adventure"!
    Rob Butterworth
    - Everything is relative... relatively speaking -
    www.techartblog.com - LinkedIn

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •