James is working on a second generation inventory script.

Currently on: Writer


It's plan is to be simpler and less ambitious than it's predecessor, but still respect the sql table structure ("as much as possible.") 

As I see it there should be 3 parts to this script:

 1. executer: runs either wget or apt-get install and copies the latest version of the other parts of the scripts then 
    executes them
 1. gatherer: collects information using only  operating system based facilities (dmesg, lsusb, lspci, ifconfig, /sys).  
 1. Writer: checks the mysql repository for changes from the current state. If different changes them.


The sql structure is a bit of a bit mess, the major tables of interest are:
 1. motherboards - List of things that can be connected to, has its own id used to tie other tables to it
 1. devices - List of deviced "connected" to mother boards 
 1. device_kinds - type identifier for connected devices (an attribute of a device). 
 1. locations - Converts x,y coordinates to a single integer that maps directly to a mother board.
 1. inventories - records the start and stop time of the inventory pass.
 1. testbeds - gives a testbed id for the specific domain, thus disambiguating node1-1

A lot of the tables are full of unused colums. I guess we'll just ignore them for now. 
The basic crux of an update should be the following:
 1. examine our IP to determine our current location
 1. We gather information about the mother board:
  1. Disk Size (dmesg)
  1. Memory Size (dmesg)
 1. Gather information about attached devices:
  1. 2 wired Ethernet addresses (ifconfig, /sys)
  1. 2 wireless Ethernet addresses (ifconfig, /sys)
  1. any usb devices (lsusb, /sys)
 1. get the mother board id from the location table
 1. update mother board information if diffrent, and stamp with current inventory number
 1. update devices if diffrent and stamp with inventory number
 1. profit.

----
Gatherer:
The disk size and memory size are a quick scan from dmesg. The disk size matches, but the memory size is a little off. It probably has to do with the way dmesg reports memory vs /sys reports memeory. It would be nice to find the /sys entry for consistency. 

In /sys/devices/pci0000:00 are the sub directories correlated with the specific Ethernet hardware. In each directory that correlated to an Ethernet device there will be a symbolic link with the operating system name of the device. This will allow us to match up the pci address(name of the subdirectory of /sys/devices/pci0000:00) to the mac address (from ifconfig). lspci can tell us the associated pci address and a hardware identifier string.

lsusb on the otherhand offers a direct correlation to the device kind table, the ordered pair of numbers xxxx:yyyy directly correlated to the tables vendor and device ids.  And the Bus xxx Device yyy number fits into the addres category of the device table.

=== 9/29/09 ===

I may have discovered the cause of the device / vendor discrepancy. Joe seems to be looking at /sys/class/net/devincename/device... perhaps this points to a different device id. I'll have to check it out.  

That being said I have a working protoype:
{{{
ssugrim@external2:~/scripts$ ruby gatherer.rb
ssugrim@external2:~/scripts$ more /tmp/external2.xml
<external2>
 <ip_adds>
  <10.50.0.12 iface='eth1' host='external2.orbit-lab.org'/>
  <127.0.0.1 iface='' host=''/>
 </ip_adds>
 <motherboard mem_size='1048512' disk_size='156301488' cpu_num='4'/>
 <Devices>
  <pci>
   <eth0 device='1229' bus_add='01:03.0' mac='00:e0:81:26:70:16' str='Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet Controller (Copper) (rev 01)' vendor='8086'/>
   <eth1 device='1010' bus_add='04:01.0' mac='00:e0:81:26:76:9c' str='Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet Controller (Copper) (rev 01)' vendor='8086'/>
   <eth2 device='1010' bus_add='04:01.1' mac='00:e0:81:26:76:9d' str='Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet Controller (Copper) (rev 01)' vendor='8086'/>
  </pci>
  <usb>
   <0 device='0001' bus_add='001:001' str='Linux Foundation 1.1 root hub' vendor='1d6b'/>
  </usb>
 </Devices>
</external2>
ssugrim@external2:~/scripts$
}}}



----
=== 10/2/09 ===

Minus error checking for failed commands, the gatherer is complete. I'm now moving onto writer.  I'm going to keep them in the same script for now, so I don't have to deal with reimporting the data and extracting it from xml, at some point that'll be a todo, so that way we can call just the gatherer if we want to. 

Fow now, I need to determine what node I am based on the resolved host name. The scheme is nodex-y.testbedname# I can extract the x and y cooridnates from the node part, and then The testbed name will have to be a lookup. (this should probably be in gatherer as parameters. 

Once I have that I can look up my unique mysql id from the mysql databse. This id will then allow me to correlate devices with the ones I have.

----
Following the instructions on http://support.tigertech.net/mysql-duplicate

I copied the mysql database from invetory1 to inventory2.

One Caveat is noted on http://forums.digitalpoint.com/showthread.php?t=259486
{{{
In the top of the database file you are trying to dump you will see that :
CREATE DATABASE `gunit_pimpjojo` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Just remove this from the dump ( Notepad or wherever you have the dump)
Then re paste the file
You just need to remove that line....and you will be good to go
}}}

=== 10/5/09 ===
I've revamped the internal data types to facilitate the way xml-simple outputs, and re imports. Any multi argument results (eth, usb, ip) return and array of hashes. This creates clean xml. I also unfolded the cords has to single instance variables, they all get wrapped up into a single attribute.

The new xml format looks like so.
{{{
<opt x="1" y="1" disk_size="156301488" domain="sb7" cpu_num="1" mem_size="491456">
  <pci device="0013" name="ath0" bus_add="00:09.0" mac="00:60:b3:ac:2b:92" str="Ethernet controller: Atheros Communications, Inc. AR5212/AR5213 Multiprotocol MAC/baseband processor (rev 01)" vendor ="168c" />
  <pci device="0013" name="ath1" bus_add="00:0a.0" mac="00:60:b3:ac:2b:66" str="Ethernet controller: Atheros Communications, Inc. AR5212/AR5213 Multiprotocol MAC/baseband processor (rev 01)" vendor="168c" />
  <pci device="4320" name="eth0" bus_add="00:0b.0" mac="00:0f:ea:4a:8b:56" str="Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13)" vendor="11ab" />
  <pci device="4320" name="eth1" bus_add="00:0c.0" mac="00:0f:ea:4a:8b:57" str="Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13)" vendor="11ab" />
  <usb device="6001" name="usb" bus_add="001:009" str="Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC" vendor="0403" />
  <ip ip="10.17.1.1" host="node1-1.sb7.orbit-lab.org" iface="eth1" />
  <ip ip="127.0.0.1" host="" iface="" />
</opt>
}}}


---- 

Working on the writer I've created a internal data type called Xmldata, it's got exactly the same fields and Info and populates them from the generated xml file. 

Working on the mysql part of I have to examine the code that lives in 
{{{
ssugrim@internal1:/opt/gridservices2-2.2.0/lib/ogs_inventory$
}}}

NOTE: mysql query strings should be crafted prior to the actual execusiton of the query, since they don't always form the way you think they do. Also the %&string& formulation of strings is very helpfull in getting the quotes correct.