[orbit-user] Sandboxes

Rick Correa rcorrea at gmail.com
Mon May 26 12:01:07 EDT 2008 

It seems like some of the Atheros based sandboxes are having problems today.
The console on sandbox 5 is down.
Both nodes on Sandbox 7 come up as unavailable.
Node 1-2 on Sandbox 8 is down.  It eventually times out during imaging.



----------------- Error Messages Below
---------------------------------------------------------------------------
**********  Sandbox 5 *************************************************
Log from Sandbox 5
RixMac-2:~ rick$ sshsb5
ssh: connect to host console.sb5.orbit-lab.org port 22: Operation timed out
RixMac-2:~ rick$ sshsb5
ssh: connect to host console.sb5.orbit-lab.org port 22: No route to host




**********  Sandbox 7 *************************************************
Log from Sandbox 7
Imaging nodes: '[1,1..2]' with image 'baseline-8.3-dev.ndz'
(Domain:  default from hostname)
(Timeout:  800 sec.)
 INFO init: NodeHandler Version 4.2.0 (1272)
 INFO init: Experiment ID: sb7_2008_05_26_11_48_57
 INFO Experiment: load system:exp:stdlib
 INFO prop.resetDelay: resetDelay = 210:Fixnum
 INFO prop.resetTries: resetTries = 1:Fixnum
 INFO Experiment: load system:exp:imageNode
 INFO prop.nodes: nodes = [1, 1..2]:Array
 INFO prop.image: image = "baseline-8.3-dev.ndz":String
 INFO prop.pxe: pxe = "2.0.3-omf":String
 INFO prop.domain: domain = nil:NilClass
 INFO prop.timeout: timeout = 800:Fixnum
 INFO stdlib: Waiting for nodes (Up/Down/Total): 0/2/2 - (still down:
n_1_1,n_1_2)
 INFO whenAll: *: 'status[@value='UP']' fires
 INFO exp: Progress(2/2/2): 0/0/0 min(n_1_1)/avg/max (10) - Timeout: 780
sec.
 INFO exp:  -----------------------------
 INFO exp:  Imaging Process Done
 INFO exp:  - 2 node(s) failed - See the topology file:
'system_topo_failed_sb7.rb'
 INFO exp:  -----------------------------
 INFO Experiment: DONE!
 INFO ExecApp: Application 'commServer' finished
 INFO run: Experiment sb7_2008_05_26_11_48_57 finished after 0:24

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: offHard
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: on
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------
ricm at console.sb7:~/current$




**********  Sandbox 8 *************************************************
Log from Sandbox 8
Imaging nodes: '[1,1..2]' with image 'baseline-8.3-dev.ndz'
(Domain:  default from hostname)
(Timeout:  800 sec.)
 INFO init: NodeHandler Version 4.2.0 (1272)
 INFO init: Experiment ID: sb8_2008_05_26_11_43_53
 INFO Experiment: load system:exp:stdlib
 INFO prop.resetDelay: resetDelay = 210:Fixnum
 INFO prop.resetTries: resetTries = 1:Fixnum
 INFO Experiment: load system:exp:imageNode
 INFO prop.nodes: nodes = [1, 1..2]:Array
 INFO prop.image: image = "baseline-8.3-dev.ndz":String
 INFO prop.pxe: pxe = "1.2.1-omf":String
 INFO prop.domain: domain = nil:NilClass
 INFO prop.timeout: timeout = 800:Fixnum
 INFO stdlib: Waiting for nodes (Up/Down/Total): 0/2/2 - (still down:
n_1_2,n_1_1)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO stdlib: Waiting for nodes (Up/Down/Total): 1/1/2 - (still down: n_1_2)
 INFO exp: Progress(0/0/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 620
sec.
 INFO whenAll: *: 'status[@value='UP']' fires
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 610
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 600
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 590
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 580
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 570
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 560
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 550
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 540
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 530
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 520
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 510
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 500
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 490
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 480
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 469
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 459
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 449
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 439
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 429
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 419
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 409
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 399
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 389
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 379
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 369
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 359
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 349
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 339
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 329
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 319
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 309
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 299
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 289
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 279
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 268
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 258
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 248
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 238
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 228
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 218
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 208
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 198
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 188
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 178
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 168
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 158
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 148
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 138
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 128
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 118
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 108
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 98
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 88
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 78
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 67
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 57
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 47
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 37
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 27
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 17
sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: 7 sec.
 INFO exp: Progress(1/1/2): 0/0/0 min(n_1_2)/avg/max (177) - Timeout: -3
sec.
 INFO exp:  -----------------------------
 INFO exp:  Imaging Process Done
 INFO exp:  - 1 node(s) failed - See the topology file:
'system_topo_failed_sb8.rb'
 INFO exp:  - 1 node(s) timed-out - See the topology file:
'system_topo_timedout_sb8.rb'
 INFO exp:  -----------------------------
 INFO Experiment: DONE!
 INFO ExecApp: Application 'commServer' finished
 INFO run: Experiment sb8_2008_05_26_11_43_53 finished after 13:28

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: offHard
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------

 INFO Topology: Loading topology 'system:topo:all'.
---------------------------------------------------
 Testbed : grid - Command: on
 Node n_1_2 - Ok
 Node n_1_1 - Ok
---------------------------------------------------
ricm at console.sb8:~/current$
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://orbit-lab.org/pipermail/orbit-user/attachments/20080526/f68eeee1/attachment.htm>

More information about the orbit-user mailing list