MapReduce powering Cloud Computing

Hello Nerd-migos !!!!

The cloud is here... r u (still) waiting for the rain !!!!

Forget the buzzword and the business hype, the cloud is real (hehe).

U need to (first) appreciate Virtual Machines/Computing (at least in theory - for non-nerds), Qemu, VMware, Xen, Virtual Box, ....


Now for fellow Nerd-migos.. we need to acquint with some of the following stuff

MapReduce:

"MapReduce is a patented software framework introduced by Google in 2004 to support distributed computing on large data sets on clusters of computers.[2]
The framework is inspired by the map and reduce functions commonly used in functional programming,[3] although their purpose in the MapReduce framework is not the same as their original forms.[4]
MapReduce libraries have been written in C++, C#, Erlang, Java, OCaml, Perl, Python, PHP, Ruby, F#, R and other programming languages." - Wikipedia


Notable (IMHO) implementation of this paradigm (apart from google itself) are

A
Hadoop

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-avaiability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-availabile service on top of a cluster of computers, each of which may be prone to failures.

B
OpenStack (+ Cloudera) is the (leading - IMHO) OpenSource Cloud attempt !!!!!

What synergy.. Apache's Hadoop powers RackSpace's OpenStack which is leveraged by Cloudera's ingenuity..


And by the way, did u know that Amazons' Elastic Cloud is a fork of Hadoop ????????

VBoxHeadless - VM without GUI

Fellow Geeks,
I have been doing some interesting stuff of late and thought i should share some...

I probably should start with the simplest....

To run a VirtualBox Virtual Machine without GUI on the Host (This is default in CentOS' and Fedora's Virtual Machine Manager) for whatever reason u may use any of the following commands...

VBoxManage startvm "VM name" --type headless

or

VBoxHeadless --startvm

Typical scenario in production/server environments where no one needs to access the GUI of a Virtual Machine from the Server/Host itself.
Instead we access the VMs using VNC or SSH

Virtual Box - Teleportation

Since Virtualbox v3.1

Move a running VM from one Physical Machine to another (for load balancing or fault tolerance)

Virtual Box - Networking

1. Host Only Network
This is a network between the hosts and guests (configured and running)

Host IP
vboxnet0
Link encap:Ethernet HWaddr 0A:00:27:00:00:00
inet addr:192.168.56.1 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::800:27ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:25 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:5697 (5.5 KiB)

Guests
192.168.56.x

2. Internal Network
This is a Virtual LAN between guests.



3. Bridged Network
This uses the host as the network to allow your guests to join the corporate LAN (and obtain IPs in the same range as the host).


4. NAT

Network Address Translation.
The following example will give u an overview of what NATing can achieve.

The following commands will forward TCP traffic that originates from port 2222 on your host OS to port 22 on your guest OS:

$ VBoxManage setextradata "Windows Guest" \
"VBoxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/Protocol" TCP

$ VBoxManage setextradata "Windows Guest" \
"VBoxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/GuestPort" 22

$ VBoxManage setextradata "Windows Guest" \
"VBoxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/HostPort" 2222

Virtual Appliances

These are software appliances (mostly enterprise applications) that run on a Virtual Machine. A virtual machine (VM) is a software implementation of a machine (i.e. a computer) that executes programs like a physical machine.

Virtual Machines are realized using a Virtualization Platform:
1. Sun Virtual Box (Now called Oracle VM Virtualbox)
2. VMWare
3. Parallels Workstation, etc

The main advantages of system VMs are:
i) multiple OS environments can co-exist on the same computer, in strong isolation from each other
ii) the virtual machine can provide an instruction set architecture (ISA) that is somewhat different from that of the real machine
iii) application provisioning, maintenance, high availability and disaster recovery[2]

The main disadvantage of system VMs is:
a virtual machine is less efficient than a real machine since it accesses the hardware indirectly


A virtual appliance is a virtual machine image designed to run on a virtualization platform.

Virtual appliances are a subset of the broader class of software appliances. Installation of a software appliance to a virtual machine creates a virtual appliance. Like software appliances, virtual appliances are aimed to eliminate the installation, configuration and maintenance costs associated with running complex stacks of software.

A virtual appliance is not a virtual machine, but rather a software image containing a software stack designed to run inside a virtual machine. Like a physical machine, a virtual machine is merely a platform for running an operating system environment and by itself does not contain application software.

Virtual Machines

Virtual Box - Cloning Drives

Prerequisite for Fedora 10
dkms package

Cloning

NB: Windows Users will call this copying.

Cloning allows u to use an existing installation as the basis of another installation/config.
For example u can clone the Virtual HDD of a Solaris appliance so that u can have another solaris appliance with Glassfish on top.

Command
#VBoxManage clonevdi soure.vdi target.vdi

The target.vdi will be created when u run the command. If a Virtual HDD exists with the same name the command will fail with an error.