Manual Provisioning the bare metal switch

Loading NOS (Network Operating System) on a bare metal switch

NOS ONIE bare metal big cloud fab bigswitch networks cumulus linux cumulus netwo edge-core pica8 sdn whitebox

Reza Toghraee

We would like to explain the simple provisioning process of a bare metal switch. 

As you might be familiar a bare metal switch is just a hardware with no switching OS or network OS or NOS or whatever you would like to call it. 

Bare metal switches normally are from vendors such as:

  • Edge-Core (Accton) 

  • Dell 

  • HP 

  • Quanta Mesh (QCT) 

  • Penguin 

  • Mellanox

  • Agema

  • Inventec

 The architecture of a bare metal switch:

  

The process

Manually Loading the NOS

Step 1- After un-boxing the switch and powering it up, the switch starts GRUB boot loader. The GRUB default options are:









If there is no OS on the switch, it will boot into ONIE. 

What is ONIE : ONIE is a small footprint Linux which is created for installing a network OS on a bare metal switch. Findout more from : https://github.com/opencomputeproject/onie 


Step 2: Depending on version of ONIE, you might see the GRUB loader of ONIE. Remember ONIE is on a Read only flash in the switch and has its own GRUB and boot loader. You will see the below screen as ONIE’s GRUB 

Choose the Install OS option.







Step 3: After the complete boot of ONIE which takes in seconds, the switch starts looking for IP through its dhcp client. ONIE doesn’t support static IP. 

After getting the IP address from DHCP, if there is not any specific DHCP option for bootloader file, ONIE will start looking for default files and location to download the switch OS. 

By default, it looks for an OS in default gateway (the option 3 from DHCP) and looks for some specific file names which are based on system hardware. In below example, the switch is a Edge-Core AS5712 (48 x 10G , 6 x 40G).





In this tutorial our aim is to provision the switch manually, although it is possible to make it Auto-provision but for a first timers, we explain how to do the manual provisioning.



Step 4: Network OS : We need to get a network OS to load on the switch.

The current available bare metal network OS compatible are : 

  1.  Cumulus Linux (Commercial L2/L3 + Light OVS) 

  2.  Pica8 (Commercial L2/L3 , SDN / OVS) 

  3.  ONL (Open Network Linux) : Open Source (Light L2/L3 and SDN integration) 

  4.  Other fabric ones such as BigSwitch networks, etc 

  5.  OcNoS from ipinfusion

  6.  OpenSwicth : OpenSource , by HP


For testing let’s try with ONL and download ONL from here: http://opennetlinux.org/binaries 

Click on latest-amd64.installer OR the PowerPC one if your switch is not x86 based. 

Save the file on your PC or a server. Switch will access the file over TFTP/ FTP / HTTP. 


Step 5: Setup the server. You can use traditional tftpd32 application or a simple windows IIS to serve the file. We kept the file in tftpd32 for simplicity. 


Step 6: Loading the OS 

On the switch ONIE, use the install_url command to tell the ONIE where to download the OS> 

Issue the command:install_url tftp://172.17.172.103/latest-amd64.installer 

ONIE will start accessing and downloading the file. 

  1. The installer file actually is a zip file with below components: 

  2. Linux kernel for the switch 

  3. The Network OS (SWI file) 

  4. Installer script file 

  5. Platform config files (these are specific files for each switch model, which are mainly to drive the LEds, Fans, ports, etc).


Note: tftp is really slow. It uploads around 3Mbps over wireless and takes long time to upload the 150Mb file. With FTP you can reach higher speeds of uploads around 100Mbps. 

Step 7: Now the installer script has installed the ONL. 

Switch reboots and you will see GRUB is changed to :






ONL boots up and reaches to the login screen. You can login with : 

Usermname: root 

Password: onl 



White-box , Bare Metal Networking

Bare Metal Definition , Hardware and Software

Reza Toghraee

Read Next