SQLStats 1.3 released

I have released SQLStats 1.3 now, and it is available for download from sourceforge. The changes are small:

• Cleaned up the build script a bit.
• Fixed a bug that caused statements to be counted twice.
• Made statement comparison non case sensitive.
• A small fix to the normalizer, that would miss a few statements in some cases.

SQLStats is a plugin for MySQL 5.5 and up that allows live monitoring of SQL statements, without using the general query log and with the added benefit of managing statements with the literals removed. read more in my previous blog-post in the subject.

Enjoy!
/Karlsson

Finally MySQL gets World-class support!

And have we been waiting for THAT puppy! Yeah, right, all over the years MySQL, Sun and Oracle has been providing real crappy support (tm), but now, finally, it is world class. And if you believe this, then may I interest you in a famous New York bridge that I can provide at a real bargain price? Or?

Fact is, MySQL already has world-class support. If you don't trust me, then look at the MySQL website, hosted by no one else than... Oracle! Like this from Mårten Mikkos or this, quoting Rich Green and Jonathan Schwartz (if you remember him).
Both claiming that now, finally, with Sun, is MySQL customers getting "world class support". Strange thing is, the same guys who are now to give world-class support are largely the same guys who has been providing much less than "world class" support.

And make no mistake, dudes, I'm not pointing fingers at the Support guys at MySQL, not by far, quite the opposite, they need some support from us now, when what they have been doing all the time is suddenly turned "world class" just because they have to use an Oracle developer crap support system. And I'm not pointing fingers to Mårten, Rich or Jonathan either at least not much. They were apparently wrong (as the support didn't actually turn World class until now), but we all had great expectations at that time, expectations which didn't turn into reality, but at least spirits were high.

In conclusion, I am much less upset with what Oracle, Sun and MySQL representatives said and how that was said, after all, they, and are most of us, stuck with a way of expressing our selves. In particular, Oracle has the problem of showing that they can add value to MySQL, they just have to, but that is difficult in most cases, so you have to use some meaningless terminology. What I DO worry about is the MySQL tech support guys at Oracle.

Come Oracle, give these guys some cred, they need it and deserve it. And just using Oracle support systems and being employed by Oracle doesn't make support World-class, but the people who make it World-class, and those have been there all the time, so really, it's been world class all the time.

/Karlsson

Slave Read-ahead 1.1 available now

If you read about slave-readahead in my previous post on accellerating MySQL Replication, and if you got to the state of actually trying it out, then you really should download the 1.1 version that I just made available. Version 1.0 really was a pre-Alpha, and it had some memory allocation issues, as well as a simple multi-threading issue from my side, which had a really annying side-effect, mainly making the progrem not work at all in many circumstances.

The new version, 1.1, is in much better shape, really. I have run it for a while now, and except for the fact that it seems to do a better job, it also doesn't crash or consume enormous amount of memory after a while.

/Karlsson

Where is the Exit.. Pleeease!

It might well be me not being in a good mood today. I really should be in a good mood, spring is here, sort of, the sun is shining, and I've started to look forward to waking up the MG and taking it for it's first round for the year!

But no, something got in the way of that. My #1 culprit is Skype but there are more of them, many of them these kinds of chat, social networking style applications. Msn is another of these annoying puppies, mainly on Windows, this is a disease that has yet to spread in the world of Linux that is has on Windows. Come on now, Microsoft, you put some many rules and regulations on a piece of software that is to be certified to run on Windows, why can't you require them to also have an Exit function? Is that something that is difficult?

In Skype you just cannot exit the application. I do not understand why. I could possibly accept it if there wasn't a reason to ever restart or exit skype, but that is not the case. Running Skype on a laptop where you move it from one network connection to the other (i.e. take the computer from the office and home into the docking station), means that Skype gets confused. It runs, but is confused, usually showing that it is live and connected, when it is not. A restart will fix that. But that restart has to be done by opening the Task Manager. If you, for the same reason, change audio equipment used for Skype (like a USB based headset in the office, and a normal Audio headset at home), the same thing happens.

Please get the Exit menu option back, or rather get us an Exit that will really Exit the application. In Skype, "Exit" means "Minimize the window". What the heck is THAT? I do not want more applications like this, I want ALL applications that are started manually by the user to also be able to shut down with a menu option. I don't mind using Task Manager, but I don't understand why I should have to. Is this a difficult thing to implement, an Exit function? Really?

/Karlsson
Exiting for now....

Want to accellerate the MySQL Slave? Here is something to try!

I admit this wasn't my idea. Nor do I claim it will solve all problems. Or that it will always speed things up. But in principle, this should work in many cases.

The deal is this: For certain kind of Slave operations, UPDATE, DELETE and INSERT ... SELECT, the operation is both a read operation, as well as a write. To be frank (but I am not, I'm Anders), this is the case with just about any MySQL operation, but for the ones I list, there is a distinct set of rows, defined by the statement itself, that will have to be read.

The above is something we can take advantage on. As there is just one thread applying the changes from the master in the slave, as the slave operations are serialized, this thread is easily the bottleneck. But there is one aspect that doesn't necessarily need to be serialized, which is the read of the data to be modified. We can, to put it simple, pre-warm the MySQL cache with the data that is used by the statements above.

What is needed to make this work is to read from the MySQL relay log, ahead of the SQL Slave Replication thread, and convert the UPDATE, INSERT or DELETE statement to a SELECT, and then execute that SELECT statements. As I said, I didn't invent this, but I wanted to try it out. Hence you can now download slavereadahead 1.0 from Sourceforge. This is a really simple piece of software that just what I describe above. But before you get started, let me tell you some caveats:

• I am NOT reading directly from the MySQL relay log files, rather, I use the output from the SHOW RELAYLOG EVENTS command, which means you have to be running MySQL 5.5 or up.
• Because of the above, not all statements will be available to cache, as row-based events are not shown properly in this case.
• Also, it might be the case, but I have not tested this, that the overhead of doing this through the server cause quite a bit of overhead. I do not know.

Now, I have not tested the effectiveness of the program much. I know it builds and works on Ubuntu 9.x on 64-bit Intel platforms, but that is about it for hard facts. What I have done to compensate for this is to make the program configurable in many ways. The SELECT statements are for example executed by separate threads, you can configure how many you want. If you run out of these threads, the SELECT will be executed by the "main" thread, which means that you can run with no specific SQL threads at all, if you wish. Also, how much ahead I read and how much I read at the time is also configurable.

The algorithm I use for which statements to pick up and execute is pretty simple, and the conversion from INSERT, UPDATE and DELETE into SELECT is also not very fancy. For us here at Recorded Future, it seems to work though. Any errors in the conversion to / execution of SQL statements are silently ignored.

I'd be really glad if someone would try out the usefulness or not of this program. It comes with the standard GNU autobuild setup, i.e. configure / make, and should be easy to get going with. There is also a basic documentation PDF included.

So, download from Sourceforge and try it out!

Cheers
/Karlsson

What? An iPad2? Me?

I got my iPad about a year ago, long before they were available here in Sweden (I got mine in the US). The iPad is a brilliant piece of engineering, it's really beautiful, and the software pretty much matches it. So now, going back to the US for the MySQL UC again, will I get an iPad2? Kaj apparently IS getting an iPad2! Well, I'm not!

Why, you ask, unless you read my views on Apple and their take om media, in which case you know why. That Apple is cencoring smut isb't something that bothers me that much, really. It bothers me a lot more that they are cencoring perferclt legit media that is not to their liking (i.e. articles that doesn't suckup to Steve Jobs, articles that claim that Anroid rocks etc). Also, it bothers me that cetain technologies are not to their liking, such as Adobe Flash (now, I'm not discussing the merits or non-merits of Flash here, I'm nt debating it it really is Steves task to determine that).

And you might say that I should get over it, and accept that fact that Apple really can do it because it's their technology. Well, tell you what, the iPad I bought is mine! And it cae with the promoise of Apps and content. And noone was saying that certain content should be censored on it. Or rather again, censorship has a certain place, for content that is harmful to someone or to society as a whole, in many cases. What I am questioning is if it is Apples taks to do this censoring? I think not.

Democrazy and free speach is important to me, and however much I admore Apple hardware, software and technology as a whole, I don't really want them to touch my content. In particular, I don't want my daily morning newspaper (Dagens Nyheter here in Sweden) to provide me with a paid-for app where they don't even say WHAT has been censored, if anything.

So, sorry Apple, but Android really rocks. 3G Wifi router built in, Flash support and much more, at a lower price...

/Karlsson

How to explain things in the world of IT

You see them all the time. The words and the expressions that are used all over the place, and which usually means something completely different from what that particular word means in real life (which is any life outside IT. Which in turn means any life at all). And it's not just Sales and Marketing folks that use these words, you see them used by any kind of IT professional, in particular when something just s*cks or plainly just doesn't work at all and the IT Professionals Dictionary is picked up and out comes a string of meaning less words, all with a fuzzy interpretation even when used correctly, strung together and mixed up with some technology term and a few acronyms.

Another thing here is that all terms used are fuzzy and non-committal, and typically passive terms are used. When something works, it is We created this exceptional groundbreaking SSD based RDBMS with world-leading performance! And when that groundbreaking technology fails, it's no longer "We created", rather it's "The system" or "The Technology" or some even more passive fuzzy terms is used, say The SSD part number X-561 671 has, due to conditions not under our control (Osama Bin Laden for example. But anyone but us. Despite the fact that when the product was released 2 months ago it was WE who had created just about every part of the product. But as the product actually s*cked, all responsibility is not with anyone else but us) show suboptimal performance under certain situations. Also the data quality has in some extreme tests shown signs of deteriorating. In short: The thing s*cks, data is corrupted, the product is ours and ours alone, but we are NOT responsible. Sort of.

Now, you can in an announcement usually find words that are meaningless, and at the same time positive, that are used in these situations. Words that make something look good, but not good in any specific kind of way, and above all are completely non-binding. Words and terms such as ground breaking, world class, excellent, seamless, high performance, user experience etc.

To these words you add a spice of acronyms of choice. These are a bit more difficult, and they have the role of making something look "modern" or "high end" or something like that, it means that these acronyms are more tied to the current times and to what is considered "HOT" when the something is written. And this changes with time of course. Todays acronym is SSD I think, at least when it comes to IT products. Every single product announcement has to mention SSD at least 10 times it seems, from razors to servers, from car radios to laptops, all benefit from SSD technology. Just HOW they benefit we don't know, but they do. It is these acronyms that makes an announcement look dated after a year or so, but in an announcement that is a few years old, say a 1980's software announcement where the writer wets his pants over SNA, you can do a search-and-replace for SNA-to-SSD and a new, modern, announcement is made.

You may think that a changing a software technology acronym like SNA for something that is a hardware one, such as SSD, isn't going to work, but that is not true. It will work just fine. The reason is that whatever is being announced, it is neither, hardware, nor software or anything substantial like that. All you know about it is that it is something that is Ground breaking and provides great Value for money! That the bl**dy thing doesn't even work, and was never intended to work, unless the provider had figured out a way to work around Einsteins Theory of Relativity (and make no mistake, this is something a marketeer or a Sales Engineer does on a daily basis) but now we are talking reality. I.e. the product is great and works perfectly, it's just that when applied in the real world it somehow fails. But real world is typically one of those things that are beyond our control. So again, besides the minor point that it doesn't at all and never will work in reality, it works just fine, What's you problem mr. Customer?

Take one particularly disgusting terms that has survived the world of IT for ages and ages, and which is still used (I had it replied to me the other day): Seamless integration. This is an interesting term in that it can be used both when something is announced, and when it is announced that it doesn't work. The reason for this aspect of this term, that makes it so useful, is that it introduces things beyond our control, i.e. when you are integrating, it typically means you are integrating with something, which is a good thing and a selling point.

On the other hand, when that integration fails miserably, which is always does, I have yet to see a product really achieving seamless integration, I guess one reason is that you really do not WANT integration (anyone using Windows and has seen the so called "ribbon" interface in MS Office knows what I am talking about. They introduce an interface which is distinctly worse and completely different from what already exists in Windows itself, which it really should integrate with. And both of these things which really SHOULD provide "seamless integration" doesn't, despite both coming from MS and despite the fact that they were once much BETTER integrated). So no, we don't integration, but that doesn't mean we can't use Seamless Integration to sell a product, as well as the major, highly important, reason to explain why it failed to deliver on the promises in certain specific customer situations. That last part can be interpreted by you newcomers to the world of IT as This huge pile of manure you call software really s*cks!

/Karlsson