Blackboard's back up.
What happened? A utility we use to keep blackboard's records up to date
stopped running, and when I checked it out, the server was misbehaving
in unexpected ways. The server, though idle, would report 100% cpu
usage for certain slices of time.
(https://status.uvm.edu/ustat/bbprd1/cpu/cpu_14400.png)
I suspected that a restart of this virtual machine might be in order
(bbprd1 is a xen guest), however it failed to shut down cleanly and
needed to be destroyed. After that, it wouldn't start, and it was
apparent that the underlying xen host was the source of the problem.
This problem also affected the other two xen hosts housing the rest of
the cluster.
Everything's been restarted and that seems to have cleaned things up.
I'll be trying to figure out what caused this problem in the first
place.
Ben
Benjamin Coddington wrote:
> Minor work caused a serious problem with the Blackboard cluster this
> morning. Blackboard is currently down, and will be back shortly.
>
> Ben
>
|