==== GENI Configuration assembly ==== [[TOC(Hardware, Hardware/hNodes/cNodeVer3*,depth=6)]] Requires: * Console ([wiki:Hardware/hNodes/cNodeVer3/aMain/cLV-67J LV-67J]) * 3 Nodes ([wiki:Hardware/hNodes/cNodeVer3/bMobile/cLV-67K LV-67K]) -- LV-67J -- Assemble as with G except: (put link to node specific page) * 1 160GB HDD on tray, with standoffs (add picture) * Power button in back panel * No CM * Intel 1000T NIC adapter -- LV-67K -- Assemble as with G except: (put link to node specific page) * **DO NOT CONNECT 4 PIN ATX12V POWER CABLE * Mobile node uses connector for 12V dc input, with 24 pin connector acting as output. Connecting both will damage board. __ * Connect new CM, attach to back panel with angle bracket, #440 screws. Choose beveled or pan head appropriately. * Use 60gb SSD on tray, with standoffs. (add picture) ===== Installing the base OS ===== ====== Console Machine Disk Partitioning ====== While setting up Ubuntu server, partition the disk as follows: * 20 GB Primary boot partition * 20 GB Home directory * 116 GB create a new mount point /Export for images * 4 GB Swap . Set the 'location' as end, and set 'use as' to swap area. ===== Network setup and configuration ===== * Three networks * Data * CM * Control * 3 switches, 1 partitioned switch, or single cable out. Setup networking as follows on the console host {{{ # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The CM network interface auto eth1 iface eth1 inet static address 10.1.0.254 netmask 255.255.255.0 broadcast 10.1.0.255 # The Control network interface auto eth3 iface eth3 inet static address 10.10.0.254 netmask 255.255.255.0 broadcast 10.10.0.255 # The Data network interface auto eth0 iface eth0 inet static address 10.20.0.254 netmask 255.255.255.0 broadcast 10.20.0.255 # The primary network interface auto eth2 iface eth2 inet dhcp }}} ====== One unmanaged switch per subnet ====== [[Image(4Node_1.png, 600)]] ====== Single managed switch ====== [[Image(4Node_2.png, 600)]] ====== Single trunk per node ====== [[Image(4Node_3.png, 600)]] ===== Software Configuration ===== Add HOSTNAME and CNAME information to /etc/hosts/ {{{ root@console:~# cat /etc/hosts 127.0.0.1 localhost console xmpp console.geni.net xmpp.geni.net }}} Add the following line to the /etc/apt/sources.list {{{ deb http://pkg.mytestbed.net/ubuntu precise/ }}} Follow the steps on http://mytestbed.net/projects/omf54/wiki/Installation_Guide_54 to install OMF 5.4. Please setup DNS to resolve locally rather than use the domain setup by the dhcp client. This will help create your own sub-domain for OMF services such as XMPP,inventory and wimax-rf. Remove any "search" entries from /etc/resolv.conf as well as remove the "request domain-*" parameters from /etc/dhcp/dhclient.conf ====== Setting up DNSMASQ ====== Edit /etc/dnsmasq.conf and at the end of the file add {{{ interface=eth0,eth1,eth3 dhcp-range=10.1.0.0,static,12h dhcp-range=10.10.0.0,static,12h dhcp-range=10.20.0.0,static,12h dhcp-option=3 dhcp-option=option:ntp-server,10.10.0.254 dhcp-boot=net:control,pxelinux.0 enable-tftp tftp-root=/tftpboot }}} Edit /etc/dnsmasq.d/omf_testbed.conf and add separate entries for the CM,control and data interfaces {{{ # CM dhcp-host=00:20:4a:d5:94:28,10.1.0.1,cons1-1 dhcp-host=00:20:4a:d5:94:2a,10.1.0.2,cons1-2 dhcp-host=00:20:4a:d5:94:27,10.1.0.3,cons1-3 # CTRL dhcp-host=00:03:1d:0c:d3:7e,10.10.0.1,node1-1 address=/node1-1/node1-1.geni.net/10.10.0.1 ptr-record=node1-1.geni.net,10.10.0.1 dhcp-option=00031d0cd37e,12,"node1-1.geni.net" dhcp-host=00:03:1d:0c:d3:61,10.10.0.2,node1-2 address=/node1-2/node1-2.geni.net/10.10.0.2 ptr-record=node1-2.geni.net,10.10.0.2 dhcp-option=00031d0CD361,12,"node1-2.geni.net" dhcp-host=00:03:1d:0c:d3:63,10.10.0.3,node1-3 address=/node1-3/node1-3.geni.net/10.10.0.3 ptr-record=node1-3.geni.net,10.10.0.3 dhcp-option=00031d0cD363,12,"node1-3.geni.net" # DATA dhcp-host=00:03:1D:0c:d3:7e,10.20.0.1,dnode1-1 dhcp-host=00:03:1D:0c:d3:60,10.20.0.2,dnode1-2 dhcp-host=00:03:1D:0c:d3:62,10.20.0.3,dnode1-3 }}} In /etc/default/dnsmasq uncomment the following line {{{ IGNORE_RESOLVCONF=yes }}} ====== Setting up XMPP service ====== 1. In the /etc/java-6-openjdk/security/java.security file, comment out the following line: {{{ security.provider.9=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/nss.cfg }}} 2. Open /etc/init.d/openfire file in an editor and modify line 27 to point to the correct java directory. In our case, it is /usr/lib/jvm/java-6-openjdk. 3. Start openfire using command {{{ /etc/init.d/openfire start }}} 4. Check if openfire is running: ps aux | grep openfire . Startup can take a while, please be patient. 5. Direct your web browser to http://localhost:9090 and begin the setup wizard. If you are unable to connect to http://localhost:9090, then do the following: 6. Choose your language and click continue 7. Enter the AM's hostname in the Domain field and click continue 8. Choose the embedded database and continue. 9. Choose the default profile and click continue 10. Enter an admin password and click continue, then wait until the installation is finished. In most cases, the username and password is 'admin'. ====== Setting up CMC service ====== In /etc/omf-aggrmgr-5.4/available/cmc.yaml file , replace original contents with the following and replace inventory URL with the correct hostname {{{ --- cmc: # inventory_url: URL to the Inventory OMF inventory_url: 'http://console:5054/inventory' default_off: "hard" }}} ====== Setting up Inventory service ====== Copy over the latest inventory service from internal1. Install missing dependencies {{{ data_mapper dm-sqlite-adapter net-scp net-ssh oml4r open4 rparsec rufus-scheduler snmp dm-do-adapter dm-mysql-adapter actionmailer -v '3.2.11' (higher version need ruby 1.9.3) rmagick (for captcha) using command: GEM_HOME=/usr/share/omf-aggmgr-5.4/gems/1.8/ gem install --no-rdoc --no-ri Before installing dm-sqlite-adapter, make sure you have libsqlite3-dev installed using command: sudo apt-get install libsqlite3-dev sudo apt-get install libldap2-dev,libsasl2-dev }}} After installing the ogs_inventory package to /usr/share/omf-aggmgr-5.4/omf-aggmgr/ Install response.rb to /usr/share/omf-common-5.4/omf-common/ Create user omf in mysql and give it "grant all privileges" Create database "inventory" in mysql Define the testbed {{{ wget -qO- 'http://console:5054/inventory/resource_add?name=geni.net&type=testbed' }}} Modify the addNode.rb file to point to the correct inventory database {{{ @host = "http://console:5054/inventory/" }}} Add all the nodes to the inventory database using {{{ ruby addNode.rb }}} ====== Installing Wimaxrf Software ====== This installation is meant to work with OMF 5.4 on Ubuntu 12.04 1. Run the following commands as root. {{{ cd /usr/share/omf-aggmgr-5.4/omf-aggmgr git clone https://github.com/gpauusa/wimaxrf.git ogs_wimax rf cd /etc/omf-aggmgr-5.4/enabled ln -s ../available/wimaxrf.yaml }}} 2. Copy the attached config file to /etc/omf-aggmgr-5.4/available/wimaxrf.yaml. ====== Setting up Login Service ====== * From exernal1, get ogs_scheduler from /usr/share/omf-aggmgr-5.4/omf-aggmgr and scheduler.yaml from /etc/omf-aggmgr-5.4/enabled and save it to /usr/share/omf-aggmgr-5.4/omf-aggmgr and /etc/omf-aggmgr-5.4/available,respectively, on your console. * Similarly, copy ogs_loginCommon from external1 from /usr/share/omf-aggmgr-5.4/omf-aggmgr into the corresponding folder of console. * Enable the scheduler service. This can be done by going into the cd /etc/omf-aggmgr-5.4/enabled/ directory and running the following command. {{{ ln -s ../available/scheduler.yaml }}} * Please run the following commands to get the missing dependencies. {{{ sudo apt-get install graphicsmagick-libmagick-dev-compat sudo apt-get install libmagickwand-dev sudo GEM_HOME=/usr/share/omf-aggmgr-5.4/gems/1.8/ gem install --no-rdoc --no-ri rmagick sudo apt-get install libldap2-dev sudo apt-get install libsasl2-dev sudo GEM_HOME=/usr/share/omf-aggmgr-5.4/gems/1.8/ gem install --no-rdoc --no-ri ruby-ldap }}} The scheduler.yaml file is shown below: {{{ scheduler: # need in /etc/apache2/sites-available/default: ProxyPass /userManagement "http://:5054/scheduler/userManagement" umURL: 'http://www.geni.net/userManagement' # need in /etc/apache2/sites-available/default-ssl: ProxyPass /loginService http://:5054/scheduler/ sslURL: 'https://www.geni.net/loginService' # need in /etc/apache2/sites-available/default-ssl: ProxyPass /schedule http://:5054/scheduler/ShowScheduler # not mandatory - schedulerURL can be determined by sslURL+'/ShowScheduler' #schedulerURL: 'https://www.geni.net/schedule' schedulerURL: 'https://www.geni.net/loginService/ControlPanel' sleep: 180 approveTime: 600 database: # User name and password for scheduler database user: 'geni' password: 'genipwd' host: 'internal1.geni.net' database: 'newScheduler' approveSeparatly: true # calculate time spend od domain separetely (only for domains listed in listOfDomains) listOfDomains: #grid,outdoor, list of domains to calculate user time separetely, all other domains accumulate - grid auth: class: LdapAuth homeDir: '/home/' nullHost: null.geni.net autohomehost: home.geni.net autohome-opts: '-fstype=nfs,hard,intr,rsize=8192,wsize=8192,nodev,nosuid' servers: - # primary LDAP server secret: 'srishti523' host: 'ldap2.geni.net' base: 'dc=geni, dc=net' - # alternate LDAP server secret: 'srishti523' host: 'ldap1.geni.net' base: 'dc=geni, dc=net' email: # email server configuration host: 'geni.net' port: 25 admins: - 'native' scheduler: prefix: 'sc4' email: # sender and administrastor email configuration sender: 'schedule@geni.net' sender_name: 'GENI Schedule' emailAdmin: 1 layout: # colors myres: '#5E7FB1' mypastres: '#A0A1A1' otherres: '#D2DDEC' otherpastres: '#CFCFCF' pending: '#E4DC04' blackout: '#6F292D' conflict: '#FF0000' registration: email: # sender and administrastor email configuration sender: 'account-manager@geni.net' sender_name: 'Account manager' emailAdmin: 1 captcha: temp_image_dir: '/tmp' ttf_file: 'images/font21.ttf' }}} Follow the steps given below to setup the scheduler. * ''' Configuring apache web server ''' 1. In the following we are assuming that is the internal hostname or IP address of the machine running the login AM, and is the IP port on which the service is running (default is 5052 for OMF 5.2services). For example, forn internal IP 172.16.250.7 and port 5025 the : would be: 172.16.250.7:5052. ServerName {{{ ProxyPass /userManagement "http://:/login/userManagement" Order allow,deny Allow from all }}} Note: If you change baseURL and publicURL in login.yaml it is necessary to change corresponding values in apache default file. 2. For SSL services (in /etc/apache2/sites-available/default-ssl) we need: /scheduler only if we want to have it as a separate link, anyway it can be accessed trough ControlPanel and will open as a default page {{{ ProxyPass /schedule/ "http://:/login/ShowScheduler" Order allow,deny Allow from all }}} {{{ ProxyPass /loginService/ http://:/login/ AllowOverride None order allow,deny allow from all :/login/*> AllowOverride None order allow,deny allow from all }}} * From internal2, get ogs_login from /usr/share/omf-aggmgr-5.4/omf-aggmgr and login.yaml from /etc/omf-aggmgr-5.4/enabled and save it to /usr/share/omf-aggmgr-5.4/omf-aggmgr and /etc/omf-aggmgr-5.4/available,respectively, on your console. The login.yaml is shown below. {{{ login: approveTime: 600 database: # User name and password for scheduler database user: 'geni' password: 'genipwd' host: 'internal1.geni.net' database: 'newScheduler' approveSeparatly: true # calculate time spend od domain separetely (only for domains listed in listOfDomains) listOfDomains: #grid,outdoor, list of domains to calculate user time separetely, all other domains accumulate - grid auth: class: LdapAuth homeDir: '/home/' nullHost: null.geni.net autohomehost: home.geni.net autohome-opts: '-fstype=nfs,hard,intr,rsize=8192,wsize=8192,nodev,nosuid' servers: - # primary LDAP server secret: 'srishti523' host: 'ldap2.geni.net' base: 'dc=geni, dc=net' - # alternate LDAP server secret: 'srishti523' host: 'ldap1.orbit-lab.org' base: 'dc=geni, dc=net' email: # email server configuration host: 'geni.net' port: 25 admins: - 'native' scheduler: prefix: 'sc4' email: # sender and administrastor email configuration sender: 'schedule@geni.net' sender_name: 'ORBIT Schedule' emailAdmin: 1 }}} * Modify login.yaml and scheduler.yaml according to your testbed. ''' Setting up LDAP client ''' NOTE: EVERYTHING THAT IS ORBIT-LAB.ORG SHOULD BE GENI.NET 1. Install ldap libraries using command {{{ apt-get install libpam-ldap libnss-ldap }}} This process will ask you a bunch of ldap related questions that will be used to prime the ldap.confs (there are 2). It's not critical to get them correct as we're just going to replace those conf's any way. (Accepting the defaults is ok). 2. Edit the /etc/ldap.conf file to configure the ldap client. {{{ base dc=geni,dc=net uri ldap://ldap.geni.net/ ldap_version 3 binddn rootbinddn cn=admin,dc=geni,dc=net pam_password md5 nss_initgroups_ignoreusers backup,bin,daemon,games,gnats,irc,libuuid,libvirt- qemu,list,lp,mail,man,messagebus,news,ntp,postfix,proxy,root,sshd,statd,sync,sys,syslog,usbmux,uucp,www-data }}} You could edit this file and change all the values to reflect this (leaveing the rest as comments) or simply replace the one you have with this one​. You can also check the non-comment lines by executing: {{{ egrep -v "^#|^$" /etc/ldap.conf }}} 3. The /etc/ldap/ldap.conf is the second file that guides the client. {{{ BASE dc=geni,dc=net URI ldap://ldap.geni.net }}} Again you can edit it directly or copy this version​. 4. The /etc/nsswitch.conf file should have the follow non comment lines: {{{ passwd: files ldap compat group: files ldap compat shadow: files compat hosts: files dns networks: files protocols: db files services: db files ethers: db files rpc: db files netgroup: nis }}} 5. Copy or create the /etc/ldap.secret file. It should contain the ldap password in clear text. 6. Finally add the follow line to the /etc/sudoers file (note this is done with the visudo command). {{{ %admin ALL=(ALL) ALL %sysadmin ALL=NOPASSWD: ALL }}} This has to be done manually. 7. Install other software: {{{ apt-get install cfengine3 emacs ntp }}} 8. We need to make sure machine have fully qualified host name (some services depend on it). So: {{{ echo "console.geni.net" > /etc/hostname }}} 9. Install Apache web server {{{ apt-get install apache2 }}} 10. Install apache's proxy package and enable it. {{{ apt-get install libapache2-mod-proxy-html a2enmod proxy a2enmod proxy_http }}} ''' Setting up LDAP server ''' 1. Decide on the organizational structure for ldap: {{{ geni.net => dc=geni,dc=net }}} 2. Install the server and utilities: {{{ apt-get install slapd ldap-utils phpldapadmin }}} 3. Modify the /etc/default/slapd file and edit the SLAPD_CONF and SLAPD_PIDFILE values to {{{ SLAPD_CONF="/etc/ldap/slapd.conf" SLAPD_PIDFILE="/var/run/slapd/slapd.pid" }}} This is our slapd.conf file, please change it according to your organization {{{ allow bind_v2 include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema include /etc/ldap/schema/ldapns.schema include /etc/ldap/schema/openssh-lpk_openldap.schema oglevel 256 modulepath /usr/lib/ldap moduleload back_bdb backend bdb database bdb index objectclass,entryCSN,entryUUID eq suffix "dc=geni,dc=net" rootdn "dc=geni,dc=net" directory "/var/lib/ldap" access to dn.base="" by * read access to * by dn.regex="cn=admin,dc=geni,dc=net" write by * read access to attrs=userPassword by dn.regex="cn=admin,dc=geni,dc=net" write by anonymous auth by self write by * none }}} 4. You can download openssh-lpk_openldap.schema from ​ http://code.google.com/p/openssh-lpk/downloads/detail?name=openssh-lpk_openldap.schema 5. Set sizelimit to 10000 by editing slapd.conf file {{{ sizelimit 10000 }}} 6. Start the service: {{{ /etc/init.d/slapd start }}} 7. Without the pid file, the start/stop script fail to start/stop the daemon. This can be resolved by editing the /etc/init.d/slapd file and search for the function start_slapd(). Add this to the last line of that function before it exits: pidof /usr/sbin/slapd > "$SLAPD_PIDFILE" 8. Make sure it is running by checking that ldap server is listening on both ports: {{{ netstat -an | grep 389 netstat -an | grep 636 }}} 9. Stop the service: {{{ /etc/init.d/slapd stop }}} 10. Import initial content (first group and account that will be used as administrators for the login service): This is a brief reasoning behind the entries in the ldif file. Please change the file as per your organizational needs. Each organizational unit(ou) has a PI who is the admin for the OU and a group which has all the accounts for the OU. So that is why your first organization and first acount have to conform to such a structure. OU admin can only manage accounts for that organization. Any person that is a member of sysadmin group in LDAP and admin group in login.yaml will be able to use ControlPanel of the ogs_login service to manage ALL accounts. 11. Run the following commands. {{{ cp /usr/share/slapd/DB_CONFIG /var/lib/ldap/ slapadd -l init.ldif -f /etc/ldap/slapd.conf chown openldap:openldap /var/lib/ldap/* }}} 12. Start the service: {{{ /etc/init.d/slapd start }}} 13. Make sure you can access the service with admin credentials: {{{ ldapsearch -x -b "dc=geni,dc=net" "objectClass=organizationalRole" ldapsearch -x -b "dc=geni,dc=net" "objectClass=organizationalUnit" ldapsearch -x -b "dc=geni,dc=net" "objectclass=organizationalUnit" ldapsearch -x -b "dc=geni,dc=net" "objectClass=posixGroup" ldapsearch -x localhost -D "cn=admin,dc=geni,dc=net" -W -b "dc=geni,dc=net" uid=* }}}