Managing a grid, Part 1: Network and infrastructure

Whether you are an established grid manager, just in the process of developing and designing your grid, or expecting to take over a grid environment soon, there are many different aspects you need to consider.

In a new four-part series I look at the different roles and requirements, and both technological and managerial considerations that you will need to consider.

For the first part in the series:

Managing a grid involves many elements, from the network and hardware you use to deploy your grid to the security, job management, and job metrics and statistics generated during the execution of your grid, which enable you to more effectively manage the work. In this four-part “Managing a grid” series of articles, we’ll look at some key elements of the grid management process, such as identifying hardware and network fundamentals and how that affects your grid process, and how to use metrics information as a scheduling, prediction, and expansion tool. Here, we cover the core elements of grid networking, along with the hardware infrastructure of your grid and how it affects other areas of grid management.

You can read the full tutorial: Managing a grid, Part 1: Network and Infrastructure.

Application virtualization, Part 1: Level 1 — Abstracting your grid infrastructure

If you have an existing application, or you are developing a new one, what’s the quickest way to make that application work within a grid?

Well one solution is to use virtualization techniques - build a virtual layer around your application so that the deployment and distribution of the application, and how you use the different components is exposed through a standardized layer.

Meanwhile, within your potential grid, look at how you can create a virtualized environment. Create the same virtual interfaces to core elements such as security, resources, and then build a system that enables you to deploy your application within this virtualized environment.

Finally, you should be able to build an interface to your grid that works through a virtual layer. The virtual interface to your grid should look the same as the virtual interface used with a single instance of your application, but hidden behind the functionality of the grid.

In this new series of tutorial at IBM, I look at each of these different levels and the theory and practical considerations behind them. The first tutorial in the series I cover the fundamental issues behind converting your application for a virtualized environment:

This “Application virtualization” series is for developers who wish to convert a single stand-alone application into a grid-enabled application through virtualization. By using virtualization, you can remove the proprietary elements and interfaces to your application, and the environment in which it runs in a way that the interfaces and methods for using the application are no longer the limiting factor.
The first step to providing a consistent, virtualized environment for a grid is to create a virtualized interface to the core components of the system and your application. We’ll examine the main components in the hardware that makes up your grid and examine how standard components like an SOA focus can help make your physical systems available as an abstracted virtual layer that can be exploited by standardized applications.
Main topics:

  • Creating a grid black box
  • Abstracting CPU resources
  • Abstracting storage resources
  • Abstraction techniques

You can read the full tutorial: Application virtualization, Part 1: Level 1 — Abstracting your grid infrastructure.

The Windows Start menu – misnomer or misunderstanding?

I was reading the October issue of Scientific American yesterday and as usual I was enjoying Steve Mirsky's column: Drawing to an Inside Flush -- A tale of two toilets.

The article is about the automatic flush toilets at Microsoft that Steve experienced when visiting Microsoft during the conference. The article in itself is interesting (especially for someone who has experienced the facilities at Microsoft's headquarters), but it was an almost throw away comment at the end that got me thinking:

Check your disaster recovery media before it’s too late

I had a disk failure a couple of weeks ago, which I mentioned at the time. The data was non-critical and I had, I thought, a reasonable backup.

Just to be on the safe side, not only did I get out my copy of the backup tapes, I got the duplicate copies that I keep in a safe location a few hundred miles away.

For all my preparation, it seems that there are some problems with the tapes, including a couple which had a bad label, some duplications I hadn't appreciated at the time, and some tapes which simply didn't contain what I'd written on the outside.

The upshot is that out of the 20 tapes that made up the backup set, and over 60 that cover all the various things I backup and archive, about 10 were unreadable or badly labelled. Fortunately the combination of duplicate tapes mean that I haven't lost anything. Better still, I was able to recover the bulk of the files off the disk when it miraculously reappeared this afternoon, which made parts of the process even quicker.

Semantic Web should lead to intelligent sites

As described in this article: Open Enterprise: Rebooting HTML for the Semantic Web, Tim Berners-Lee, inventor of what we now know as the Web, is promoting the next phase of web development, called the Semantic Web.

Tim has made some interesting comments on the development of the web, including the failure of people to adopt XML in a logical way that could affect everybody. HTML and many of the functions in modern browsers have caused problems over the years, leading the confusing mix of standards and 'Built for Netscape' type logos at the bottom of the page.

Gears of War and the power of Internet fandom

Christmas is on the way - someone today mentioned to me how many weeks it was, but I really didn't want to hear. This year will be another console war as the new kids on the block, the WII and the Sony PS3, fight it out with the well established XBox 360.

Up to now I've been avoiding upgrading my XBox because the only game I've wanted to play is Halo 3 - the follow up to two of my all time favourite games, Halo and Halo 2. Halo 3 however is more than a year away.

Finding a potential replacement has been difficult. Many of the games that appeared on the XBox 360 in the months after it's release have hardly taxed the new environment and power that is available. Until recently, at least, when I heard about Gears of War. The game centers around the emergence of an alien race of insect like creatures that take over the planet.

Acrobat 8 Universal and downloadable

Yesterday I purchased an upgrade to Acrobat 8. Acrobat has been one of my staple applications since the first version when I used it to aid the proofing process at an ad agency. By pure accident, I picked up the upgrade after it had been made a Universal application, and thus running at full-speed on the new Intel Macs.

It's also the first time I've purchased a large commercial software product and downloaded the software the live, rather than buying it on CD. I've downloaded plenty of software before, from utilities through to full-blown applications, but never what would be considered a 'box-only' product like Adobe Acrobat before.

iMac 24″ – Plenty of WOW factor

My iMac 24" has just arrived. I ordered it weeks ago, and it seems to have taken an age to be built, shipped, and arrive.

I'm happy to say it's perfect. It's got a tremendous amount of WOW factor. Surprisingly for a display this large, there are no dead pixels. Unless you've seen one in the flesh, it's difficult to describe how big it feels, but it makes the 17" iMac I was given by Apple (for being an Apple Transition Kit developer) positively tiny. The sheer size of the screen is impressive, and it's bright and clear throughout.

Also arrived at the same time is the new White MacBook, and all I can say about that is 'gosh this is cute'. I'm typing on it right now, and I can see that it will already be a popular little addition to the toolkit. Best of all, the little screen is incredibly bright (I'm currently using it in a well lit room with the brightness on the lowest setting before the backlight goes off) and, considering the reduction in size and real estate, surprisingly useful.

Making a laptop portable

Before the end of this week, my second laptop will arrive.

Why do I need two? Because my main laptop spends the bulk of its time on my desktop, wired up to an ergonomic keyboard and a second monitor. When you work on the machine for 10 hours a day, you want to be able to get the best out of it, and have the best ergonomics for doing so. With all that in mind, you might be wondering why I have a laptop, not a desktop. Simple, when I'm travelling for work, I need the full blown laptop for work. In those cases, I'm more than happy to use the laptop keyboard (which is one of the best in my experience) and the smaller screen.