wiki:Internal/SystemPrototyping/Evaluation
close Warning: Can't synchronize with repository "(default)" ("(default)" is not readable or not a Git repository.). Look in the Trac log for more information.

End-to-End Evaluation

Contents

  1. 1. ORBIT Evaluation
    1. 1.1. Deploying MobilityFirst on ORBIT nodes
    2. 1.2 MF Disk Images for ORBIT
    3. 1.3. Inside a MobilityFirst Image
      1. 1.3.1. Code Base
      2. 1.3.2. Binaries, Configuration Files
      3. 1.3.3. Boot Script
    4. 1.4. Updating or Customizing an Existing MF Image
    5. 1.5. Configuring and Running MobilityFirst Experiments
    6. 1.6. Sample OMF Scripts for ORBIT
      1. 1.6.1 Test Config 1: Sender-Router-Receiver
      2. 1.6.2. Test Config 2: Multiple Senders and Receivers
    7. 1.7. Steps to run experiment
    8. 1.8. Test Script to MF Image Compatibility
    9. 1.9. Test Script to MF Release Compatibility
  2. 2. GENI Evaluation

1. ORBIT Evaluation

Early versions of the prototype MobilityFirst (MF) network consisting of: (1.) Click-based router, (2.) a distributed name resolution service, and (3.) client network API and host protocol stack are available for evaluation on the ORBIT testbed. The steps involved in evaluating sample configurations are listed below. Before you begin, it is suggested that your familiarize yourself with the ​OMF framework, which is used to setup and orchestrate these experiments.

1.1. Deploying MobilityFirst on ORBIT nodes

The deployment can be done in one of the following ways:

  1. Installing an MF release (tarball or SVN revision) and dependencies on a base ORBIT image
  2. Imaging ORBIT nodes with a pre-established MF disk image

1.2 MF Disk Images for ORBIT

An MF disk image contains all components (router, gnrs, and client stack and network API library - sources and precompiled binaries) and can be installed on nodes using ​OMF tools.

All images listed below are stored at repository1.orbit-lab.rutgers.edu:/export/omf/omf-images-5.2: Images Currently In Use

Image NameCreated onMF ReleaseDescription
mf-proto-rc1_0.ndz 4-18-2012 rc1.0 End-to-end integrated layer 2 with MF Network API, MF host stack and an MF Router with GSTAR, and GNRS interface, distributed 'in-network' GNRS

Loading a particular image is done using the 'load' command: 

> omf load <node-set> <image-name>

e.g., 

> omf-5.X load [[1,1], [1,2], [1,3]] mf-proto-1.0.ndz

'X' in the command above refers to the version of OML control framework. Presently, 5.2 and 5.3 are most used.

1.3. Inside a MobilityFirst Image

1.3.1. Code Base

The image holds the prototype code base under /usr/local/mobilityfirst/code. It has the following top-level directories:

  • click - Router elements implementing storage-aware routing, hop-by-hop reliable link-level data transport, and interface to GNRS service. This also has elements that implement Click-based sender and receiver applications.
  • gnrsd - C++ implementation of a GNRS server, and an interactive GNRS client.
  • client - C implementations of client API and stack that compile for Linux and Android platforms. Also has sample sender and receiver applications using the API
  • eval

1.3.2. Binaries, Configuration Files

  1. bin - compiled binaries go here
  2. conf - config files from across sub projects, incl. click and gnrs configurations
  3. scripts - e.g., to initialize Click execution
  4. topology - definition files used within the MF router to enforce connectivity among nodes

Also installed on this image are the dependencies for the router, gnrs, and client components. A complete list of installed dependencies can be found in the README accompanying the code base.

1.3.3. Boot Script

The image also contains a boot script (/etc/init.d/mf-proto) that can be used to automate the update/compile functions. It updates the local codebase to the latest release from MF SVN, (TODO - auto updating is currently disabled, pending the creation of an anonymous account access to MF SVN), and then compiles and installs Click and other MF component binaries as described above. Excerpt below from the boot script shows the update and compilation of the click router: 

...

MF_DIR=/usr/local/mobilityfirst
MF_CLICK_ELEMENTS_DIR=$MF_DIR/code/click/elements
CLICK_DIR=/usr/local/src/click

#update mobilityfirst prototype code base 
cd $MF_DIR/code

#auto-update disabled pending anonymous account
#svn update

#Compile user-level click after copying MF's click elements into click codebase
rsync -vt $MF_CLICK_ELEMENTS_DIR/gstar/* $CLICK_DIR/elements/local

cd $CLICK_DIR
./configure --disable-linuxmodule --enable-local
make elemlist
make install

...

The output of the boot script is appended to /var/log/mf-proto-boot.log.

1.4. Updating or Customizing an Existing MF Image

The installed MF source can be updated to a latest release either from the SVN or using a release tar ball. If updating to latest SVN version, you simply run the update command from under the /usr/local/mobilityfirst/code dir. If customizing to a particular MF version from SVN or using a newer tar ball, first delete contents under the code dir before installing. Similarly, one can also update 3rd party components like Click while creating an updated MF image.

For compiling an updated code base, we currently have a simple 'make' bash script (/usr/local/mobilityfirst/code/boot/mf-proto) that combines several steps. Usually, the compiled MF binaries are installed under /usr/local/mobilityfirst/bin. However, the 3rd party components we use are in various locations. For example the source for Click modular router is under /usr/local/src/click, and it's compiled binary ends up under /usr/local/bin. Therefore, before building the Click-based router, the MF-Click elements that implement the protocol stack are to be installed under Click's designated source dir (/usr/local/src/click/elements/local to be specific). So, the bash compilation script does all such steps for the router, gnrs, host protocol stack and network API library components, placing the compiled binaries in proper locations.

Once the source has been updated, the following compiles and installs binaries. Alternately, one can also update just individual components and run the local make and copy binaries to proper locations. 

> /usr/local/mobilityfirst/code/boot/mf-proto #compilation script

Once ready with all upgrades and want to create a new MF image, you have to run an ORBIT 'prepare' script to among other things ensures the package manager caches are cleaned, and interfaces are configured appropriately when creating a general disk image that can be booted on a variety of hardware nodes. 

> cd /root

> ./prepare.sh #common ORBIT script to prepare file systems for creating an image

The prepare.sh script above will also log you out and turn the node off, following which you can save the newly created image by using the OMF save command: 

> omf-5.X save [1,1]

The resulting image can be found at repository1:/export/omf/omf-5.X/ and can be used to image nodes for subsequent experiments.

1.5. Configuring and Running MobilityFirst Experiments

While custom scripting can be used to execute an experiment, OMF has all necessary functionality to reliably configure and repeat experiments on the ORBIT testbed. Both the configuration details (what nodes run what applications with what parameters) and the experiment execution control (when to run what) can be specified within a ruby script using omf syntax. Refer to ​OMF User Guide to get familiar with writing OMF scripts. In the next section, we present several sample scripts to get you started with MF network experimentation.

Once you define an experiment script, and have loaded the MF image on the nodes that will run MF components, you run the experiment using the 'exec' command: 

> omf-5.X exec my-mf-expt.rb

The omf runtime reports any failure to bring up components. For more detailed information, refer to the logs created by MF components - all located under /var/log. Additionally, MF components (only Click-based router presently) are instrumented to report key statistics which are then logged to relational tables hosted on an OML server co-located with the testbed. Section on 'OML-based Monitoring' has more details.

1.6. Sample OMF Scripts for ORBIT

1.6.1 Test Config 1: Sender-Router-Receiver

Below is the simple topology: 

         S ---- MFR ---- R

S-Sender Host, MFR - MobilityFirst Router, R - Receiver Host

The topology in these experiments is enforced within the Click implementations by a GUID-based connectivity graph specified by a topology file passed to click. The following lines in the topology file define the above graph: 

#syntax: <node-GUID> <neighbor-count> <neighbor-GUID1> [<neighbor-GUID2>] ...
1 1 2
2 2 1 3
3 1 2

Files: OMF script | topology file

1.6.2. Test Config 2: Multiple Senders and Receivers

Below is the topology: 

                 S2
                 |
                 |
         S1 ---- MFR1 ----- MFR2 ---- MFR3 ---- R1
                                       |
                                       |
                                       R2

S-Sender Host, MFR - MobilityFirst Router, R - Receiver Host

Files: OMF script | topology file

1.7. Steps to run experiment

  1. Choose and reserve a testbed with OMF support and required number of nodes. ORBIT has a 400 node grid, but also has 9 sandboxed testbeds sb1-sb9 more suitable for smaller experiments.
  2. Image the nodes with an established (and compatible, see compatibility notes below) MF image from list of available OR image the nodes with a baseline ORBIT image followed by install of MF release from SVN or tarball
  3. Determine the node set you will use for the experiment and ensure they are available, imaged and working.
  4. Modify OMF ruby script provided to use the chosen nodes. For example, if you choose nodes node1-1, node1-3, node1-4 (ORBIT hostnames, domain part depends on chosen testbed), then modify the statically defined node set in the ruby script as shown below: 
    ...

#static topo with available nodes
#defTopology('static_universe', [[1,1],[1,2],[1,3]]) - original 
defTopology('static_universe', [[1,1],[1,3],[1,4]])
baseTopo = Topology['static_universe']

...
  5. Execute the script using appropriate version of omf tools. For example, the node naming conventions used above (as points in a grid: [1,1]) is valid for OMF version 5.2, but invalid for versions 5.3 and above which use FQDN to identify hosts.

1.8. Test Script to MF Image Compatibility

Owing to feature additions and/or interface modifications, older OMF scripts may be incompatible - despite our best efforts - when used against newer MF versions and images . Here is what we understand at present:

mf-proto-trial3.ndzmf-proto-gec12.ndzmf-proto-rc1_0.ndz
Test Config 1 No No Yes
Test Config 2 No No Yes

1.9. Test Script to MF Release Compatibility

No releases yet.

2. GENI Evaluation

​GENI, an NSF-funded proposal for a global environment for network innovation, is a multi-group collaborative effort to realize an at-scale experimental network infrastructure that is rich (i.e., with wired and wireless resources, commercial and experimental platforms) and allows for deep programmability.

​ProtoGENI is the prototype implementation and deployment of GENI. ProtoGENI is also the control framework for a number of GENI resources currently deployed on the national backbone and at several participating campuses. It is worth noting, however, that there are several GENI deployments that use other control frameworks and experimentation across ProtoGENI and these deployments is currently set up via personnel coordination/manual configuration.

The following links provide the basic information to learn about ProtoGENI and to get started with experimentation:

  • ​ProtoGENI Tutorial with basics on
    • Creating an account with one of the Clearing houses (e.g., Utah Emulab or BBN)
    • Setting up certificate (with managers) and key-based (with individual hosts) authentication and authorization
    • Steps and test scripts for finding and reserving resources on ProtoGENI
  • Quering and Reserving Resources can be done using either of following:
Last modified 11 years ago Last modified on Apr 4, 2013, 2:57:21 PM
Note: See TracWiki for help on using the wiki.