close Warning: Can't synchronize with repository "(default)" (/common/SVN/wimax does not appear to be a Subversion repository.). Look in the Trac log for more information.

Changes between Version 14 and Version 15 of Old/WiMAX/05/01


Ignore:
Timestamp:
Dec 15, 2010, 3:07:53 PM (10 years ago)
Author:
hmussman
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Old/WiMAX/05/01

    v14 v15  
    353353[[Image(WiMAX/00:fig 7  121010_ WiMAXSystemOverview.jpg, 100%)]]
    354354
     355Procedure provided by Evan Zhang in Summer, 2010:
     356
     357= This page contains all the information you need to understand the current OMF lab setting and how it was built =
     358
     359
     360
     361== Reference ==
     362
     363http://omf.mytestbed.net/wiki/omf/Installation_Guide_52
     364
     365http://omf.mytestbed.net/wiki/omf/Ubuntu_Baseline
     366
     367
     368
     369== Related Ticket ==
     370
     371http://groups.geni.net/syseng/ticket/1004
     372
     373
     374
     375== OMF System Architecture with WiMAX Base Station ==
     376
     377[[Image(omf_gpolab_arch.png)]]
     378
     379
     380
     381== Current Lab Setting ==
     382
     383[[Image(omf_gpolab_internal.png)]]
     384
     385
     386
     387== Installation and Test Instructions (You should always look at the reference instruction first,http://omf.mytestbed.net/wiki/omf/Installation_Guide_52) ==
     388
     389{{{
     390Note: The OMF/OML 5.2 software(our current build target, 5.3 is on the way) mainly includes following packages(debian packages):
     391omf-aggmgr-5.2(Aggregate Manager) omf-expctl-5.2(Experiment Controller) omf-resctl-5.2(Resource Controller) oml-server oml-client openfire_3.6.4_all.deb(XMPP server).
     392omf-aggmgr-5.2, omf-expctl-5.2, oml-server, openfire_3.6.4_all.deb could either be built on the same machine or dedicated machine for each.
     393omf-resctl-5.2 and oml-client should be installed on each experiment node. We choose to build omf-aggmgr-5.2, omf-server, openfire_3.6.4_all.deb in one machine
     394(actually VM:herculaneum.gpolab.bbn.com) and omf-expctl-5.2 in another machine(actually VM: drasnia.gpolab.bbn.com).
     395For the internal test purpose, we made two kinds of the experiment node: VM nodes and physical nodes. VMs are used for the purpose of quick experiment test(without disk loading).
     396Physical nodes are used for testing disk image loading. For the network design, OMF requires that AM, EC and all the nodes are IP connected as control path. And between nodes
     397there should be experiment data path, either through wired, wifi, or wimax. For our current setting, two VM nodes are wired through eth1 as experiment data path. The physical
     398node 3 is a NC10 sumsung netbook which is a wimax enbaled node, so the data path suppose to be through our wimax station.
     399}}}
     400 
     401=== Installation of the Aggregate Manager machine(VM) ===
     402
     403==== Network Setup ====
     404
     405 * Different from the reference instruction, we also have two interfaces eth0 and eth1 on AM machine. eth0 is control interface with internet accessibility and [[BR]]
     406eth1 is used to run our own DHCP server for the disk image loading purpose. Both eth0 and eth1 have fixed IPs and domain names assigned by GPOlab's infrastructure.
     407
     408==== Configuring dnsmasq (DHCP, DNS and TFTP server) ====
     409
     410 * edit /etc/dnsmasq.conf and I add the following lines at the bottom of the file:
     411
     412{{{
     413# Added by Evan for OMF installation
     414interface=eth1
     415dhcp-range=128.89.91.147,128.89.91.150,12h
     416dhcp-option=3,128.89.91.145
     417dhcp-option=121,128.89.91.30/32,128.89.91.146     <----- for dhcp assigned IP to ping 128.89.91.30, please refer to http://groups.geni.net/syseng/ticket/1004 for details
     418dhcp-option=option:ntp-server,128.89.91.30
     419dhcp-boot=net:control,pxelinux.0
     420enable-tftp
     421tftp-root=/tftpboot
     422}}}
     423
     424 * create a new file /etc/dnsmasq.d/testbed.conf and add your node DHCP details here:
     425
     426{{{
     427dhcp-host=net:control,00:24:54:08:67:68,ini-wimax-dhcp-147,128.89.91.147
     428}}}
     429
     430 * /etc/init.d/dnsmasq restart
     431
     432==== Set up PXE booting ====
     433
     434 * Follow the reference instruction and change the related IPs and MACs mentioned in the instruction to ours.
     435
     436==== Inventory installation ====
     437
     438 * Follow the reference instruction and manually populate the inventory
     439
     440{{{
     441One testbed info is added to table 'testbeds' with domain name 'sandbox1'
     442Three nodes info is added to the inventory (need to update four tables for each node: devices, motherboards, locations, nodes). They are,
     443
     444node[1,1]: control IP: 128.89.91.31  (vm)
     445node[1,2]: control IP: 128.89.91.32  (vm)
     446node[1,3]: control IP: 128.89.91.147 (NC10 physical node)
     447
     448For more detailed please go to: http://128.89.91.30/phpmyadmin, user:omf, pw:omf
     449}}}
     450
     451
     452
     453=== Configuring the Aggregate Manager (AM) ===
     454
     455 * Follow the reference instruction and carefully change the IPs to ours.
     456
     457 * Test the CMC (Dummy) service with
     458
     459{{{
     460 wget -qO- http://128.89.91.30:5052/cmc/allStatus?domain=sandbox1
     461
     462<TESTBED_STATUS>
     463                <detail>
     464                        <node name='n_1_1' x='1' y='1' state='POWERON'/>
     465                        <node name='n_1_2' x='1' y='2' state='POWERON'/>
     466                        <node name='n_1_3' x='1' y='3' state='POWERON'/>
     467                </detail>
     468</TESTBED_STATUS>
     469}}}
     470
     471
     472=== Installing the Experiment Controller (EC) ===
     473
     474 * We can not find /etc/omf-expctl-5.2/nodehandler.yaml by following the reference instruction, we need to do,
     475
     476{{{
     477sudo cp /usr/share/doc/omf-expctl-5.2/examples/nodehandler.yaml /etc/omf-expctl-5.2/nodehandler.yaml
     478}}}
     479
     480 * My nodehandler.yaml:
     481
     482{{{
     483---
     484:econtroller: 
     485  :config:
     486    :default:   
     487      :domain: 'sandbox1'
     488      :repository:
     489        :path: ["../repository", "/usr/share/omf-expctl-5.2/repository"]
     490
     491      :inventory:
     492        :url: 'http://128.89.91.30:5052/inventory'
     493
     494      :web:
     495        :host: '129.89.91.31'
     496
     497      :communicator:
     498        :type: 'xmpp'
     499        :xmpp:
     500          :server: '128.89.91.30'
     501          :password: 'gpo2010'
     502
     503    :debug:
     504      :repository:
     505        :path: ['../repository']
     506      :communicator:
     507        :type: 'mock'
     508
     509}}}
     510
     511=== Configuring XMPP ===
     512
     513==== Installing Openfire ====
     514
     515{{{
     516'wget http://www.igniterealtime.org/downloadServlet?filename=openfire/openfire_3.6.4_all.deb' doesn't work, should be
     517wget -O openfire_3.6.4_all.deb http://www.igniterealtime.org/downloadServlet?filename=openfire/openfire_3.6.4_all.deb
     518
     519username: admin, password: gpo2010
     520}}}
     521
     522==== Creating the system nodes ====
     523
     524{{{
     525omf_create_sysnode-5.2 128.89.91.30 128.89.91.31
     526omf_create_sysnode-5.2 128.89.91.30 128.89.91.32
     527omf_create_sysnode-5.2 128.89.91.30 128.89.91.147
     528}}}
     529
     530
     531=== Preparing the experiment nodes ===
     532
     533 * For VM experiment nodes (which is not mentioned in the reference instruction), the instruction is similar as preparing the baseline disk image. Reference: http://omf.mytestbed.net/wiki/omf/Ubuntu_Baseline   
     534
     535{{{
     536My nodeagent.yaml:
     537
     538# This is an example configuration for the Resource Controller at NICTA
     539#
     540# NOTE: use only 'spaces' to indent !
     541# ('tab' indents are not supported by the ruby yaml parser used to read this file)
     542#
     543---
     544nodeagent:
     545  # Communication settings
     546  comm:
     547
     548    # How many seconds can we go without a message from the node handler
     549    # before we assume we have lost connectivity and need to reset
     550    handler_timeout: 40
     551 
     552    # Number of consecutive handler timeouts before a 'handler lost'
     553    # will be declared.
     554    timeout_count: 2
     555 
     556    # Number of seconds to wait between consecutive RETRY requests
     557    retry_interval: 3
     558 
     559    # Number of seconds between consecutive HEARTBEAT messages
     560    heartbeat_interval: 10
     561 
     562    # Pause between resending previous messages
     563    resend_interval: 0.1
     564 
     565    server_port: 9026
     566    # local_if: Control Interface used with TCPServer Comm.
     567    local_if: eth0
     568    xmpp_server: 128.89.91.30
     569
     570}}}
     571
     572 * For physical nodes (e.g. NC10 netbooks), we need to load disk image to it. How? We have two ways,
     573
     574{{{
     575A. Run frisbee server and client manually:
     576
     5771) Download the target disk image from Winlab. Gautam scped nc10-wimax.ndz to herculaneum.gpolab.bbn.com:/var/lib/omf-images-5.2/nc10-wimax.ndz
     5782) Boot NC10 into pxe image. Refer to 'Set up PXE booting' section of this document.
     5793) Run frisbee server at AM server(herculaneum): > frisbeed -p 7000 -i 128.89.91.146 -m 128.89.91.147 /var/lib/omf-images-5.2/nc10-wimax.ndz          <------Important: -i option must be specified here
     5804) Run frisbee client at NC10(a shell is offered by pxe image): > frisbee -p 7000 -i 128.89.91.147 -m 128.89.91.146 /dev/sda                          <------At both sides -m option IP could be replaced by a muticast IP e.g. 224.0.0.1
     581}}}
     582
     583{{{
     584B. Load disk image through Experiment Controller       
     585
     5861) Same as A.1)
     5872) At Experiment Controller, run: > omf load "[1,3]" nc10-wimax.ndz
     588}}}
     589
     590
     591 * After the nodes are up. The Resource Controller will report node's liveness to XMPP server on hourly basis. You can go to http://128.89.91.30:9090/user-summary.jsp (user:admin, pw:gpo2010) to check.
     592
     593
     594=== Using the testbed ===
     595
     596At this point, you may want to run a test experiment. We choose to run a modified(get ride of the wireless configuration) hello world experiment with our two VM nodes.
     597Based on this page http://omf.mytestbed.net/wiki/omf/Basic_Tutorials_for_Developing_Experiments, we developed our experiment description,
     598
     599{{{
     600 # A) Define the 'source' group, which has the unique node [1,1]
     601 #     Nodes in this group will execute the application 'test:proto:udp_sender'
     602 #
     603 defGroup('source', [1,1]) {|node|
     604 
     605   node.prototype("test:proto:udp_sender", {
     606     'destinationHost' => '192.168.0.2',
     607     'localHost' => '192.168.0.1',
     608     'packetSize' => 256, # in Bytes
     609     'rate' => 8192 # in bits/sec
     610 
     611   })
     612 
     613   # mode can be 'adhoc' or 'managed' or 'master'
     614# Commented out by Evan for wired test
     615   #node.net.w0.mode = "master"
     616 }
     617 
     618 #
     619 # B)  Define the 'sink' group, which has the unique node [1,2]
     620 #     Nodes in this group will execute the application 'test:proto:udp_receiver'
     621 #
     622 defGroup('sink', [1,2]) {|node|
     623 
     624   node.prototype("test:proto:udp_receiver" , {
     625     'localHost' => '192.168.0.2'
     626   })
     627 
     628   # mode can be 'adhoc' or 'managed' or 'master'
     629# Commented out by Evan for wired test
     630   #node.net.w0.mode = "managed"
     631 }
     632 
     633 #
     634 # C)  Configure the wireless interfaces of All the Nodes involved in
     635 #     this experiment
     636 #
     637
     638# Commented out by Evan for wired test
     639 #allGroups.net.w0 { |w|
     640 allGroups.net.e0 { |w|
     641   #w.type = 'g'
     642   #w.channel = "6"
     643   #w.essid = "helloworld"
     644   w.ip = "%192.168.0.%y" # the '%' triggers some substitutions
     645 }
     646 
     647 #
     648 # D)  When all the nodes are turned On and the all the applications
     649 #     are installed and ready, we can start to perform the experiment
     650 #
     651 whenAllInstalled() {|node|
     652 
     653   wait 30
     654 
     655   allGroups.startApplications
     656   wait 20
     657 
     658   Experiment.done
     659 }
     660}}}
     661
     662You can run the experiment by doing:
     663
     664{{{
     665> omf exec /path/to/hello_wire.rb
     666}}}
     667
     668When I first ran hello_wired.rb experiment, I encountered a error at the resource controller side saying:
     669
     670From /var/log/omf-resctl-5.2.log
     671{{{
     6722010-08-11 11:11:04 DEBUG nodeAgent::EthernetDevice: configure cmd: /sbin/ifconfig exp0 192.168.0.1 netmask 255.255.0.0
     6732010-08-11 11:11:04 ERROR nodeAgent::EthernetDevice: While configuring ip with 192.168.0.1 - CMD reply is: ''
     674}}}
     675
     676Then I found out that in source file '/usr/share/omf-resctl-5.2/omf-resctl/omf_agent/agentCommands.rb', there is mapping between net.e0 <--> exp0,
     677
     678So I changed it to net.e0 <--> eth1 in order for the experiment description to assign IP to eth1.
     679
     680Now I can run our first experiment successfully.
     681
     682
     683=== Instrumenting your experiment ===
     684
     685Basically, you need to insert some code into your application's source code to utilized the measurement library offered by OML. [[BR]]
     686Then the measurement data will automatically collected and sent to OML server during the runtime of your experiment.
     687
     688For more detailed, please refer to: http://omf.mytestbed.net/wiki/omf/UsingOML
    355689
    356690== 6.8  OMF/OML Configuration ==