Tag Archives: MCslp Coalface

MySQL UC 08 Keynote Videos

I’m pleased to say that I was able to see these in the flesh, but if you aren’t lucky enough to be here (or just want to watch them again), Zack has posted up videos on YouTube of the opening keynote presentations:

Sadly these are only snippets, but if you like what you see, make sure to book your place for next year’s conference early!

Solaris 08/07 impressions

New versions of Solaris keep coming out apace, and with each new version we see improvements and enhancements, both from the core Solaris group and from the cooperation with the OpenSolaris teams.

Solaris 10 8/07 came out last year, but it takes a while to go through and look at all of the new features, and in the time it’s taken me to fully understand all the new stuff, they’ve come out with another new version, but the functionality in the 8/07 release warrants some closer inspection. You can get the full details with the official documentation.

First up, this is one of the the first releases where I feel comfortable using it pretty much out of the box without modification. Most of the tools and functionality are available with a standard install. You get Firefox 2.0 and Thunderbird 2.0 as standard, and combined with the improvements to the X Window System, we have an OS that can be used as a desktop without any additional installs.

There are still some things I prefer to re-install (perl, for example), but the point is you no longer have to install many things. With Sun Studio being available, there’s little reason not to use Solaris 10 as a good replacement for Linux – in fact, one of the other elements I want to talk about might help there.

Of course, its not all plain sailing. One of the problems I encountered is that I installed onto a machine with three identical drives – but when prompting you to partition disks, it shows you all of the disks without any kind of indication which one is which. You just get the size of the volume to choose from (a little difficult to choose when your machine has three identical drives).

Luckily this was an empty machine that I could trash, but I wouldn’t want to identify the drive during installation just based on the partitions each device had. Actually, it turned out worse than this – after installation, Solaris had actually ended up on my second HD (the first attached to the second IDE controller), not my first. I had to swap the disks around to get it started properly. Really it’s a minor issue, but one to be wary of.

One of the big improvements throughout are the changes and enhancements to the ZFS file system. One of my favourites is that you can now create a ZFS pool and then share it as an iSCSI device to other systems. This means that if I had a big server with loads of space, I could create a ZFS pool, get the benefits of ZFS on the server side, and share portions of the disk for storage over to other machines.

Enabling iSCSI sharing is just a case of setting the shareiscsi

# zfs set shareiscsi=on mainpool/ishare

On a potential client, you add the server sharing the pool to the list of iSCSI targets, and then run format. With iSCSI, you are effectively sharing the physical device, so it just appears like a another disk, which you partition and organize as you want. I can see a real benefit here for providing users with a dedicated disk that they can organize how they like, while still giving you control the disk space and allocation at the server end.

ZFS also has a huge range of other improvements. There’s no ZFS booting in this release (it’s in b84 of the regular OpenSolaris releases), but ZFS support and facilities just keeps going on and up.

Using a single OS is often no longer a reasonable choice. Different companies release their products on different OS, and sometimes you just have to install another OS. Virtualization is one solution (and there’s plenty of news about that in later builds of Solaris), but in 8/07 we have another alternative, BrandZ.

BrandZ builds on Solaris containers – logical divisions in the running environment that allow you to divide, allocate and isolate resources on a system into discrete, well, containers. Each container is effectively running another instance of Solaris (although it’s not as resource hungry as I make that sound).

BrandZ extends that container functionality with an additional layer that provides binary compatibility for other operating systems, without going to the same level of complexity or resource requirements as the virtualization options.

There are limits, of course, to that – you are not running a copy of the corresponding OS kernel, instead, an interface layer is providing the compatibility between the linux kernel and library calls and the corresponding interfaces on the Solaris side.

You can’t run device drivers, and you also can’t run applications that require access to the graphics card; you have to run any X applications by using an X server in the global zone. That means you can’t run X.org in Linux, but since you can attach to the main X service, it still means you can run Firefox and other applications running as Linux binaries but displaying on your main host (or indeed, anywhere with a running X server).

For the most part, that means you can run Linux binaries in a BrandZ zone without resorting to full virtualization. It works fine- I was able to run a whole range of apps without any problems. The downside? Currently it’s x86 only. I’d love to be able to run SPARC Linux apps on my SPARC servers, for testing purposes only it would fill a hole I otherwise have to fill with another machine running another OS.

BrandZ is a convenient way to run Linux applications without running a full Linux distro either on another machine or in a virtualization. This means it will appeal either to the user with a Linux binary legacy application, or the casual Linux application user without the complexities of a VM layer or dual booting.

A good example here is something like Skype, available on Linux, but not Solaris. There are many examples of people using BrandZ on Solaris (see here for an example).

All in all, there’s a lot to like in this release. ZFS is maturing and extending nicely, and I certainly want to play with the iSCSI feature a bit more to see where it can be used effectively (I’ve got a new server on it’s way soon to help with that test) .

BrandZ, on the other hand, will solve the issue of running Linux applications to the point where I dont need VMware or Parallels for some jobs. I wont completely eliminate the need, but for one of my desktops it will make life much easier.

Cheffy 2.0 beta now out!!

I’m pleased to announce that we have release the new version of Cheffy, what we’ve been calling internally, Cheffy 2.0.

The new version comes with a number of new features, some of which I’ll be detaileding over the coming weeks, but as a headline:

  • New layout and structure
  • New Inspire Me section for quick access to selected recipes
  • Your own personal Kitchen area where you can manage your preferences, your cookbooks, recipes, shopping list, and connect with other users.
  • Add your own recipes – you can now add a recipe, have the nutrition automatically calculated, and then share that recipe with others.
  • RSS feeds for your cookbook, recipes, and Inspire me and automated searches
  • Improved searching and nutrition information

And there’s more to come – please start using the site and let us know what you think, or if you find any problems.

New VoiceXML/XQuery Demo

I’ve got a new VoiceXML/XQuery article coming out, and IBM have asked that a demo of the service is live.

The service is an interface RSS reader – you get to choose the topic and the feed (currently only four static feeds are provided), then it will read out the feed content.

You can try out the demo by calling:

  • Skype: +99000936 9991260725
  • US (freephone): (800) 289-5570, then using PIN 9991260725

Occasionally the hosting times out, in which case, please contact me and I’ll check it out and restart or reboot the service.

Comparing 32-bit/64-bit MySQL on OpenSolaris

I’ve been working with the folks working on OpenSolaris for a few months now providing advice and input on getting MySQL and the connectors (C/ODBC and C/J) installed as a standard component. Having got the basics in, the team are now looking at adding both 32-bit and 64-bit packages.

The question raised at the end of last week was whether OpenSolaris should enable 64-bit builds by default in 64-bit installations, and whether there was a noticeable performance difference that would make this worthwhile.

I did some initial tests on Friday which showed that there was a small increase (10-15%) of the packaged 64-bit installations over 32-bit under x86 using snv_81. Tests were executed using the included sql-bench tool, and this was a single execution run of each package for 5.0.56. Transactions are missing because I hadn’t enabled transactions in the tests.

Test (x86, binary packages) 32-bit 64-bit +/-
ATIS 20 17 17.65%
alter-table 18 15 20.00%
big-tables 14 11 27.27%
connect 134 121 10.74%
create 348 348 0.00%
insert 1038 885 17.29%
select 399 257 55.25%
transactions
wisconsin 10 8 25.00%

There are some significant differences there (like the 55% increase on SELECT speeds, for example), but a single execution is never a good test. Also, it’s unclear whether the differences are between the compilations, the platform or just pure coincidence. This requires further investigation.

As a coincidence, Krish Shankar posted these notes on using SunStudio 11 and SunStudio 12 and the right compiler flags to get the best optimization.

I decided to do 10-pass iterations of sql-bench and compare both 32-bit and 64-bit standard builds, the 32-bit standard builds against Krish’s optimizations, and finally 32-bit and 64-bit optimized builds.

Some notes on all the tests:

  • All builds are 5.0.56
  • All tests are run on SunOS 5.11, snv_81
  • Tests are executed on the same OS and machine running in 64-bit. The SPARC tests are on an UltraSPARC IIIi@1.28GHz Workstation with 1GB RAM; x86 are on a Dell T105, Opteron 1212 with 4GB RAM. Of course we’re not comparing machine speed, just 32-bit binaries over 64-bit.
  • All results are in seconds; lower values mean faster performance.
  • In all tests I’m using the built-in defaults (i.e. no my.cnf anywhere) so as to simulate a standardized installation.

Let’s first look at x86 and the 32-bit standard and 32-bit optimized builds:

Test (x86, 32-bit) 32-bit (standard) 32-bit (optimized) +/-
ATIS 15.4 21 -26.67%
alter-table 15 16.3 -7.98%
big-tables 13.7 12.5 9.60%
connect 77.6 133 -41.65%
create 343.7 350.6 -1.97%
insert 760.3 1043.8 -27.16%
select 394.8 384.2 2.76%
transactions 10.8 18.6 -41.94%
wisconsin 6.6 10.1 -34.65%

The standard build uses gcc instead of SunStudio, but I don’t get the same performance increases that Krish saw – in fact, I see reductions in performance, not improvements at all. I’m going to rebuild and retest, because I’m convinced there’s a problem here with the builds that I’m not otherwise seeing. I certainly don’t expect to get results that show a 27% reduction in insert speed. That said, a 10% big-table increase is interesting. I’ll redo these builds and find out if the slow down is as marked as it here.

Here’s the comparison for standard builds between 32-bit and 64-bit standard builds on x86:

Test (x86, standard) 32-bit 64-bit

+/-
ATIS 15.4 13.5 14.07%
alter-table 15 10.6 41.51%
big-tables 13.7 10.6 29.25%
connect 77.6 76.4 1.57%
create 343.7 346 -0.66%
insert 760.3 681.6 11.55%
select 394.8 254.8 54.95%
transactions 10.8 10.7 0.00%
wisconsin 6.6 5.8 13.79%

There are some incredible differences here – more than 50% increase in SELECT, and 30% for the big-tables test show that there is some advantage to having the 64-bit builds on x86 enabled.

Unfortunately I’ve had problems with the 64-bit optimized builds on my machine, so I haven’t completed optimized test comparisons.

On SPARC, Sun Studio is used as the default compiler, and the standard 32-bit and 64-bit show little difference:

Test (SPARC, standard) 32-bit 64-bit +/-
ATIS 28.6 27.5 4.00%
alter-table 27 26.7 1.12%
big-tables 26.9 29.4 -8.50%
connect 166.3 173.6 -4.21%
create 155 143.1 8.32%
insert 1577.3 1572.3 0.32%
select 807.4 761.6 6.01%
transactions 19.5 18.75 4.00%
wisconsin 11.1 11.4 -2.63%

Overall, a pretty insignificant difference here.

Now let’s compare the standard and optimized builds using Krish’s flags on SPARC:

Test (SPARC) 32-bit (standard) 32-bit (optimized) +/-
ATIS 28.6 27.75 3.06%
alter-table 27 26.25 2.86%
big-tables 26.9 25 7.60%
connect 166.3 162.5 2.34%
create 155 145.25 6.71%
insert 1577.3 1551.5 1.66%
select 807.4 769.625 4.91%
transactions 19.5 16.875 15.561%
wisconsin 11.1 10.875 2.07%

The tests here show little significant difference between the standard and the optimized builds, although 6-7% would probably be enough to prefer an optimized build if you wanted to build your own.

Now let’s compare the optimized, Sun Studio 12 builds running in 32-bit and 64-bit:

Test (SPARC, optimized) 32-bit 64-bit +/-
ATIS 27.75 27.3 1.65%
alter-table 26.25 26.6 -1.32%
big-tables 25 25 0.00%
connect 162.5 162 0.31%
create 145.25 154.3 -5.87%
insert 1551.5 1535.1 1.07%
select 769.625 771.2 -0.20%
transactions 16.875 19.1 -11.65%
wisconsin 10.875 10.7 1.64%

The differences are virtually non-existent, and taking the reductions and increases in performance overall, there’s probably little difference.

The overall impression is that on x86 the improvement of 64-bit over 32-bit is significant enough that it’s probably a good idea to make 64-bit the default. On SPARC, the difference in the optimized builds is so slight that for compatibility reasons alone, 32-bit would probably make a better default.

I’ll probably be re-running these tests over the next week or so (particularly the x86 so I can get a true comparison of the 64-bit optimized improvements), and I’ll try the T1000 which I upgraded to snv_81 over the weekend, but I think indications are good enough to make a reasonable recommendation of 64-bit over 32-bit.

Can a website have a box?

I’ve been sent a couple of books to review for O’Reilly, which I’ll get to soon, but I’m intrigued by the text on the front of the book.

Both are for websites – Wikipedia and Facebook – and are both part of the Missing Manual series.

I’m confused though – both also quote the Missing Manual mantra:

The book that should have been in the box

Can a website have a box?

Acorn, Pixelmator and Iris alternatives to Photoshop

I’ve had Acorn in my list of applications to review for months, and I’ve only just got round to it.

I wish I’d got there earlier. Acorn is quick and powerful, and that’s because it employs your GPU to do soe of the processing, and it includes a number of filters (based on OS X’s CoreImage interface), all of which is wrapped up into a nice little application. If you can’t find what you want, there are ObjectiveC and Pythong plugin interfaces, but I haven’t investigated it yet.

Of the alternatives, the most talked about is Pixelmator, closely followed by Iris.

Pixelmator is a closer approximation to the way that Photoshop operates, and in some respects I prefer the functionality and the feel of Pixelmator if I was looking for a Photoshop replacement, but there are other elements I don’t appreciate. The flashy graphics and animations when you do different operations seem superfluous to me.

There are nice touches in both applications – the stamp tool in Pixelmator is particularly good (although I prefer Photoshop), while in Acorn the crop and select tools provide much better feedback during the select operation than even Photoshop.

Iris is less polished, but shows some promise. There are some annoying oddities (I used 1.0b2, 367), like the image opening at pixel resolution, rather than being scaled to screen size, and the lack of specialized cursors can make identifying what you are doing and the potential effects of that process difficult, but the image editing and manipulation is very quick (particularly on stamp and touch up operations). It is, however, a bit memory hungry at the moment.

Any of these solutions would make a good alternative Photoshop and Photoshop Elements if you don’t want to go down the Adobe route.

Of these I currently prefer Acorn – it’s small and lightweight and the interface feels much more polished and easy to use. Certainly I’d consider it as an alternative to the larger packages on a laptop if you wanted something while you were traveling. I can’t get by without Photoshop because of the image scanning and editing I do, but occasionally I want something more extensive than Preview when I’m on the move.

Of course, this could change – all of these tools are being actively developed and so it’s likely that there will be some leapfrogging along the way.

Migrating from WSRF to WSRT

If you’ve been reading the main MCslp blog you will know that I’ve been writing a series on writing a grid solution using WS-RT.

WS-Resource Transfer is an improved version of the WS-Resource Framework (and the associated WS-ResourceLifetime and WS-Transfer). Basically, the developers of WS-RT looked at WS-Transfer and WSRF, and added the functionality found in WSRF missing in WS-Transfer to WS-RT. WS-RT, for me, is basically what WS-RF should have been all along.

If you know WS-RF and want to start using WS-RT, check out Migrating from WSRF to WSRT.

For my WS-RT series, check out the MCslp Grids category.

An introduction to Eclipse for Visual Studio users

I’m seeing more and more people moving to Eclipse as a development platform, even those Windows users who have traditionally used Visual Studio. As an Eclipse user for quite a while now I’m often asked how good it is, or how to use it.

Of course, telling people to simply try it out isn’t enough. Many people just don’t get Eclipse and cannot understand or translate the skills and experience they already have to the Eclipse environment. That’s where An introduction to Eclipse for Visual Studio users can help.

It’s a quick overview of the fundamentals of Eclipse from the perspective of a Visual Studio user. For a more in depth examination, there’s a tutorial Eclipse for Visual Studio developers, and another on migrating your applications from VS to Eclipse: Migrate Visual Studio C and C++ projects to Eclipse CDT.

I can recommend any (or indeed all) of these.

Getting Best out of MySQL on Solaris

I’m still working up some good tips and advice on MySQL on Solaris (particularly the T1000, the new x86 based servers like the X4150 and ZFS, amongst other things), but until then I found Getting Best out of MySQL on Solaris while doing some research.

With the latest OpenSolaris builds (b79, from memory) we now have MySQL built-in, and I worked with the folks on the OpenSolaris Database team to get some reasonable configurations and defaults into the system. MySQL 5.1 and 64-bit support is currently going through the process and will be a in future build.

I’ve also been working with the DTrace people to improve the DTrace support we have in MySQL (documentation will go live this week, I hope). MySQL 6.0.4 will have some basic DTrace probes built-in, but I’ve proposed a patch to extend and improve on that significantly.

We’re in the process of updating the documentation and advice on Solaris (and OpenSolaris) installations and layout too, which is itself part of a much larger overhaul of the installation and setup instructions for all platforms.