[orbit-user] Sandboxes

Mon May 26 12:47:08 EDT 2008

Hi Rick,

I can't find any problems with sb7; node1-2.sb8 needs a new disk and we
are still working on sb5 console.

Regards,

Ivan.

________________________________

From: orbit-user-bounces at orbit-lab.org
[mailto:orbit-user-bounces at orbit-lab.org] On Behalf Of Rick Correa
Sent: Monday, May 26, 2008 12:01 PM
To: Orbit user discussion mailing list
Subject: [orbit-user] Sandboxes

It seems like some of the Atheros based sandboxes are having problems
today. 

The console on sandbox 5 is down.
Both nodes on Sandbox 7 come up as unavailable.
Node 1-2 on Sandbox 8 is down.  It eventually times out during imaging.

----------------- Error Messages Below
------------------------------------------------------------------------
---
**********  Sandbox 5 *************************************************

Log from Sandbox 5
RixMac-2:~ rick$ sshsb5
ssh: connect to host console.sb5.orbit-lab.org port 22: Operation timed
out
RixMac-2:~ rick$ sshsb5
ssh: connect to host console.sb5.orbit-lab.org port 22: No route to host

**********  Sandbox 7 *************************************************

Log from Sandbox 7
Imaging nodes: '[1,1..2]' with image 'baseline-8.3-dev.ndz'
(Domain:  default from hostname)
(Timeout:  800 sec.)
 INFO init: NodeHandler Version 4.2.0 (1272)
 INFO init: Experiment ID: sb7_2008_05_26_11_48_57
 INFO Experiment: load system:exp:stdlib
 INFO prop.resetDelay: resetDelay = 210:Fixnum
 INFO prop.resetTries: resetTries = 1:Fixnum
 INFO Experiment: load system:exp:imageNode
 INFO prop.nodes: nodes = [1, 1..2]:Array
 INFO prop.image: image = "baseline-8.3-dev.ndz":String
 INFO prop.pxe: pxe = "2.0.3-omf":String
 INFO prop.domain: domain = nil:NilClass
 INFO prop.timeout: timeout = 800:Fixnum
 INFO stdlib: Waiting for nodes (Up/Down/Total): 0/2/2 - (still down:
n_1_1,n_1_2)
 INFO whenAll: *: 'status[@value='UP']' fires
 INFO exp: Progress(2/2/2): 0/0/0 min(n_1_1)/avg/max (10) - Timeout: 780
sec.
 INFO exp:  ----------------------------- 
 INFO exp:  Imaging Process Done 
 INFO exp:  - 2 node(s) failed - See the topology file:
'system_topo_failed_sb7.rb'
 INFO exp:  ----------------------------- 
 INFO Experiment: DONE!
 INFO ExecApp: Application 'commServer' finished
 INFO run: Experiment sb7_2008_05_26_11_48_57 finished after 0:24

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: offHard
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: on
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------
ricm at console.sb7:~/current$ 

**********  Sandbox 8 *************************************************
Log from Sandbox 8

Imaging nodes: '[1,1..2]' with image 'baseline-8.3-dev.ndz'
(Domain:  default from hostname)
(Timeout:  800 sec.)
 INFO init: NodeHandler Version 4.2.0 (1272)
 INFO init: Experiment ID: sb8_2008_05_26_11_43_53
 INFO Experiment: load system:exp:stdlib
 INFO prop.resetDelay: resetDelay = 210:Fixnum
 INFO prop.resetTries: resetTries = 1:Fixnum
 INFO Experiment: load system:exp:imageNode
 INFO prop.nodes: nodes = [1, 1..2]:Array
 INFO prop.image: image = "baseline-8.3-dev.ndz":String
 INFO prop.pxe: pxe = "1.2.1-omf":String
 INFO prop.domain: domain = nil:NilClass
 INFO prop.timeout: timeout = 800:Fixnum
 INFO stdlib: Waiting for nodes (Up/Down/Total): 0/2/2 - (still down:
n_1_2,n_1_1)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down:
n_1_2)
 INFO exp: Progress(0/0/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
620 sec.
 INFO whenAll: *: 'status[@value='UP']' fires
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
610 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
600 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
590 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
580 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
570 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
560 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
550 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
540 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
530 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
520 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
510 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
500 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
490 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
480 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
469 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
459 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
449 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
439 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
429 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
419 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
409 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
399 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
389 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
379 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
369 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
359 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
349 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
339 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
329 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
319 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
309 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
299 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
289 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
279 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
268 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
258 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
248 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
238 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
228 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
218 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
208 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
198 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
188 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
178 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
168 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
158 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
148 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
138 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
128 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
118 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout:
108 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 98
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 88
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 78
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 67
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 57
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 47
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 37
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 27
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 17
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 7
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: -3
sec.
 INFO exp:  ----------------------------- 
 INFO exp:  Imaging Process Done 
 INFO exp:  - 1 node(s) failed - See the topology file:
'system_topo_failed_sb8.rb'
 INFO exp:  - 1 node(s) timed-out - See the topology file:
'system_topo_timedout_sb8.rb'
 INFO exp:  ----------------------------- 
 INFO Experiment: DONE!
 INFO ExecApp: Application 'commServer' finished
 INFO run: Experiment sb8_2008_05_26_11_43_53 finished after 13:28

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: offHard
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: on
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------
ricm at console.sb8:~/current$ 