MyQuery 3.0 Alpha available now!

MyQuery 3.0 is now available, as 3.0.0 Alpha, for download from Sourceforge. I am aware that this is probably surprising for many MySQL users, as the original plan was to release 2.4, and now we end up with 3.0 instead, which is the reverse of what MySQL is up to now (going from 6.0 to 5.4).

Now,. jokes aside, the reasons for 3.0 instead of 2.4 is that 3.0 really contains a whole bunch of new features and bug fixes. Among the cool new features are:

• Tabbed editing - This allows you to edit several files at the same time, in different tabs. The old "History" function is still there, although of less use now I guess.
• Dual connections - MyQuery can now keep two connections active at the same time, one for issuing queries, and one for controlling the whole thing. This allows the controlling connection, for example, to issue a KILL QUERY if you have a long running query that you want to stop.
• Status monitors - You can now monitor global and session status easily, and I have included "lap time" and "total time" monitoring, and some nifty means to choose exactly what status you want to see.
• MySQL Statements monitoring - This shows statements from SHOW PROCESSLIST, both in just a plain processlist format, as well as a specific dialog which keeps a history of statements, having any literals removed. You want to check up which statements are running most and takes most time? This is the feature for you (although this is not as effective as the MySQL Enterprise Monitor Query Analyser, as the latter will track all queries, even ones that are very short, whereas I poll SHOW PROCESSLIST).
• Dictionary dialogs - Dictionary for tables and routines, that you may use as reference and to get at the CREATE statements for them, copying them to the editor or to the clipboard.
• Non-modal dialogs - One thing that the multiple connections allows me to do is to have multiple windows open at the same time, without them blocking each other. I use mutexes to control access between different windows sharing a connection also.
• Keyboard accelerators - These were there before also, but now there are more of them, and they are configurable by the end user.
• Editor keyword files - The keywords used by the editor for syntax highlightning are now configurable, by placing them in "keyword files". You may have several different sets of keyword files, and easily change between them.

In addition to the above, I have refactored a few things to make things work better, and some features which have been there before has been fixed up. This goes for example for the Auto-reconnect feature, which has been there for a while, but which never worked really well. Now it works better, and is easier to maintain.

As for this being an Alpha version, this is based on it not being 100% feature complete yet. Also, there is still a fair amount of testing to do. The features to add before GA are a few more dictionary views, and the testing them of course.

If you test it, and get a crash, there is now a crash handling feature included. This will bring up a dialog and an optional emailing Window, allowing you to send the dumpfile to me. Feel free to attach that file and drop it off to me, but it may turn rather large, possibly.

Happy SQL'ing
/Karlsson

Using Windows multiple memory heaps for finding leaks

If you, like me (but sometimes I feel it's just me left) use C and the Win32/Win64 API for Windows programming, you might be interested in this little trick which I use quite often on Windows to make sure I do not have any memory leaks.

Usually, you allocate from the heap using something like malloc for a level of portability, as the Windows HeapXxx calls are Windows specific, and malloc maps to these guys anyway. usually.

In Windows, each process has a default heap. In most cases, DLL data is private, but heap allocations are usually done from the Process heap. This causes a memory allocation issue that is all too common: There may be a memory leak, and again, it may not be, as everyone is allocating data from the same heap, including any libraries, static or dynamic, that you may use, when all you want to know is if you have a leak in your code.

And this what you can do here: Although a process has a default heap, there is no stopping it having several heaps. The memory allocations functions that you mostly use on Win32, like HeapAlloc and HeapFree, take a handle to the heap to allocate from as the first parameter, and this handle can come from two places:

• GetProcessHeap() - Get the default heap for the process.
• HeapCreate() - Create a new heap.

Often, you see code like this then:
LPTSTR pStr;
if((pStr = HeapAlloc(GetProcessHeap(), 0, nLen)) == NULL)
return NULL;
and so on. Not too exciting stuff. Instead, I recommend you keeping a global handle for the heap to use for all you allocations, and use this handle, like this:
HANDLE g_hHeap;
winMain()
...
g_hHeap = GetProcessHeap();
...
LPTSTR SomeFunction()
{
LPTSTR pStr;

if((pStr = HeapAlloc(g_hHeap, 0, nLen)) == NULL)
return NULL;
...
}
This looks like it is the same thing, but it does give you one advantage: You can, when you set your heap to the return value from GetProcessHeap() instead create your own heap, using HeapCreate(), and you need only do this in one place. And using some other nifty calls, you can "walk" this heap, and see how many allocations you have and how big they are. And that is from your heap only! No external DLLs, no nothing but what you have allocated! If you allocate 15 bytes, you will see 15 more bytes allocated from your heap, no more and no less. How can you do this then? The way I do this, if this is a GUI application, I provide a menu option with a dialog that shows me the # of allocated blocks and the size and stuff like that (also, I do thisin such a way, susally, so I can ifdef this menu option away if I want to). So how do you "walk" a heap then? Oh, that's easy, here is a simple sample for you:
PROCESS_HEAP_ENTRY entry;
unsigned int nBlocks;
unsigned int nFreeBlocks;
unsigned int nSize;

...
entry.lpData = NULL;
nBlocks = nFreeBlocks = 0;
nSize = 0;

/* Walk the heap and get info. */
while(HeapWalk(g_hHeap, &entry) != 0)
{
if(entry.wFlags == 0)
{
nFreeBlocks++;
continue;
}
nBlocks++;
nSize += entry.cbData;
}

I find this really useful, and it allows me to easily track down those small leaks (as I know the size AND count, this is much easier), that can, in a production system, end up being major leaks and cause big issues.

/Karlsson

Black box programming in C. Why? and a little How.

I'm not too hot on C++, I admit that, although I am hot on some of the advantages of C++! The way it is possible to program using a class as a"black box", where the developer of that "black box" decides what is public, what is protected and what is private.

The reason this is good thing is simple, it makes it possible to control what gets access in a complex datastructure, and how that is accessed, and that is determined by the developer of that complex datastructure, not the users. Without this, you would have to document what aspects of the structure should and can be used by anyone using it, and not even that might be enough, i.e. who needs to read the spec!

This is particularly important with Open Source software. If I work on an open source project and a simple C struct is available to me, I will use it. Hey, it's there and I can see it. And if there are any ill effects of using a particular member of the struct, that I can check by reading the code! Right?? Nah, not really. It may well be that a particular member of a published struct is distinctly private and is planned for other uses beyond what is currently implemented. And when that happens, your code may break. I think this is an issue that we are fighting with at MySQL right now, but we are not unique, I have worked with other RDBMS projects (written in C), even commercial ones, and have seen the same effect. Just because it is not Open Source doesn't mean you are protected, it just means that you have some more control (you know WHO writes the code, and if they screw up, you know where to find them).

I think this is a very important issue with Open Source in general. How can you be sure that all developers that writes code in your project follows what ground rules? Not that they are unintelligent or mean, but really, getting the grips of a large chunk of complex code like MySQL is difficult. The nice thing with C++ here is that it allows you to document the source itself, sort of, and the compiler will, to an extent, also enforce that.

But for me, as I am no fan of C++ in general, or rather, I don't mind C++ in itself that much, but all the different frameworks, discussions, features that are not agreed on and that work differently (templates) and non-standardized aspects (name mangling for example) drives me crazy.

So, can I achieve something similar in C then? Yes you can! And it works surprisingly well, at least in small / mid-size projects.

To begin with do NOT repeat NOT put struct definitions in header files! This is just plain bad! This doesn't mean that structs shouldn't be published, it just means that the implementation of it should be "hidden" from the users of it, and a pointer to it is implemented in the header file.

mycode.h
typedef struct tagMYSTRUCT *PMYSTRUCT;

mycode.c
struct tagMYSTRUCT
{
int nId;
char *pName;
} MYSTRUCT, *PMYSTRUCT;

This works so well and I use it all the time! Even in my own small projects, where I am the only developer! hey, even I loose track of my own code at times!

Now, this of course means a few things. To begin with, if the whole project resides in one implementation file, mycode.c in this case, this doesn't help at all. So the lesson then is to keep the project split in several files, and implement each function as one or more types, usually structs, that are controlled by code in a simple implementation file.

Another advantage of only having forward declared structs in header files is that the include file ordering gets much less important, and the issue of includefiles X requiring Y, but also Y requireing X, is much easier to solve and is much less of an issue in general.

Disadvantages here is that for every member that is accessed, I must provide an accessor function. This is usually considered good practice with C++ also, but in the case of C++ it is a bit easier, as the implementation can be inlined (performance) and in the header file (documentation). Can we solve this with C then?

Well, yes, to an extent. If we assume that we really would like to have the nId member in my sample struct above public, we could do something like this:

mycode.h
typedef struct tagMYSTRUCT
{
int nId;
} MYSTRUCT, *PMYSTRUCT;

mycode.c:
typedef struct tagMYSTRUCTPRIVATE
{
int nId;
char *pName;
} MYSTRUCTPRIVATE, *PMYSTRUCTPRIVATE;

In this case, in the implementation file I need to cast PMYSTRUCT to a PMYSTRUCTPRIVATE. This is not such a big issue, and this is a workable solution, although this has a few disadvantages, like the multiple include issue. Also, I wrote somewhere above that you shouldn't put struct definitions in includefiles, hey, I'm just trying to be practical here. This way of doing things is a bit iffy I guess though.

Another neat C++ feature is inheritance. This can, to an extent, be done with C! Although this also iffy... I'll get into this one later... But I guess that you have figured it out already, if you have read this far.

/Karlsson
Getting out into the sun. Time for some open air motoring!

Opinionated review on an opinionated book: Joel on Software

I just finished reading Joel Spolskys book "Joel on Software", and I had a good and interesting read I have to say. This doesn't mean that I agree with everything in this, by now a bit dated, published in 2004 and as it is a collection of writings from Spolskys website www.joelonsoftware.com, it is more dated than that.

Spolsky writes in a rather humorous, witty style, and he is very opinionated and has strong opinions on many things. And he sometimes hits some rather minor issue which he dislikes with a baseball bat. But I enjoyed reading the book, it was a fun read and it got me thinking on issues, even though I didn't agree with Spolsky always, frankly, most of the time I didn't (he's not too hot on Open Source for example, and gives examples why it's not going to work, and we now know, as the book is dated, how things went. And he was wrong). But the deal with this book isn't to get you to buy into Spolskys ideas, rather, it's to start you thinking on the issues he writes about.

Before you go on and buy this book though, you should be aware that Spolsky is a big-time Windows fan, there is no doubt about that (and I tend to like Windows as a desk-top OS also). But again, that is not the deal here. The deal is to get those cogs in your brain working.

As for entertainment, this book isn't bad, as is often the case with strongly opinionated writings. There are a couple of annoying things in the book though, once you have learnt to live with Spolsky having a different opinion than you (or me, for the most part). And that is that Spolsky is also seems pretty full of himself at times. The software company that he owns is one that can't do no wrong, anywhere in the book. The only time that it's mentioned to do something at fault, it is to tell how incredibly well they dealt with that problem and fixed it, no doubt better than anyone else.

But that aside, if you want to read a book on software development (yes, there is very little code in the book, but even in that sense I got a few ideas for the next version of MyQuery coming up soon), stuff that are interesting issues to think about, and which you might not agree with, but still interesting to think about.

Also, for all you hard-core Linux folks out there, I think this could be interesting to read, something written by a hard-core Windows guy, to get an idea on how the Windows developers of the world thinks, and why and what their arguments are. And Spolsky actually goes more into taking about differences here than about whats good and bad with Windows, Linux etc.

Also, I said that Spolsky is a hard-core Windows guy, and I am a Windows guy also, in part, but that doesn't mean Spolsky has many kind words for what is going on at Microsoft these days (oh, Spolsky used to work at Microsoft, which he writes about many times. It is no surprise that in THOSE days, little, if anything, was amiss in Redmond). .NET is not high-ranking in Spolskys eyes, and in this sense, it is interesting to see that in some cases he was right (this is complex, and getting .NET out there has been a long process), but that it actually get out there and has a reasonably large following, well that was also expected, but that so many developers actually seems to have embraced .NET with there hearts was apparently not expected.

And just to make things clear: As a Windows developer, I insist on C and on the Win16/Win32/Win64 API. No MFC, no C#, no .NET and above all, no VB.

/Karlsson

Cross platform GUI portability, part 2

So, if you read my first post on the subject of cross platform GUIs, you probably think I missed one aspect, which is that on cross platform GUI toolkits. Don't these guys, in one shape or the other, solve this problem? If you ask me, the answer is no, and I will now set up to make my point here.

The problem is that the main issue persists: Do you want your app to look like Windows on Windows and like Mac on OSX. Always. For all intents and purposes? Or do you want your application to look and feel like "your application", on all platforms? Whichever path you choose, there are solid good reasons for both approaches. And that is the issue. wxWindows, Java, wxWidgets, all fine technologies, but they do not solve the basic issue.

Is the solution to make the GUI adaptable to both approaches on all platforms, and have it configurable to support all these aspects? In a sense, I think that is true, but on the other hand I feel that the effort needed to achieve this is terribly counter productive. It really is. It is cool and it looks nice, but in the end, really does little to drive technology forward, in a philosophical sense.

I have seen a few commercial toolkits doing this, with great technical success. The drawback:

• These toolkits are usually expensive.
• Now you have a third API to program against, not Win32, not OSX, but something else. Not to mention Gnome or KDE then.
• What good does it do to your application, really? I have seen applications built with these toolkits expose a 100% Windows look and feel on Macintosh! And it looks real neat, and achieving the task of having Windows look-and-feel on a Mac is not an easy task, and these toolkits are advanced. But the value for the end user, once the interface they want to use is set? Not much.

To an extent, this reminds me of the 1980s, early 1990 mix of different networking technologies, where some vendors were pushing cross networking APIs. Usefule, well, sort of. Valuealble there and then, well, yes, to an extent. Useful in the long run? Nope. Definitively not at all. And then you have all this code, when you realize that TCP/IP is all you need to care about, written not against a TCP/IP library, but against some other kind of weirdo library that has a cost, but which provides you with cross network portability across TCP/IP, IPX/SPX, BanyanVines and DecNet. Which would be useful if it wasn't for the fact that no one is using anything else than TCP/IP.

Things aren't even close to that situation with GUIs, and I will make a shot at a cross platform wxWidgets based app later this year. But for now, I am pretty much convinced that independent how much I try, I have to make compromises between functionality, OS/GUI integration, usability and code overhead.

• No, I am NOT going to sprinkle #ifdefs across my code.
• Yes, I know I need to make compromises.
• Yes, I am aware that my knowledge on Win16 7 Win32 is of limited use.

Despite this, I want to try it. And to easy the minds of my MyQuery users, no, the target for this will NOT be MyQuery, but something else...

/Karlsson
About to go to bed.

Of GUI conventions and cross platform portability

Well, this post I guess belongsin the "and stuff" category of my blogg. I am reading Joel Spolsky's book "Joel on Software" now, which is a collectioon of Joels writings on different software things. I will post a review in due time, but for now, I wanted to touch on another subject that this book brought up, and which is related to my new version of MyQuery that is soon coming up, which is this: Who controls the conventions of the GUI of an application? There are several possible answers to that question, such as:

• The application - I.e. the application code determines what keys are used for cut, paste, open file, the order of standard buttons (Open,Close) the look of the windows and other things that the application does.
• The GUI - I.e. Windows / KDE / Gnome / OSX or whatever GUI framework you use determines the standard functions as listed above. As for any applicatioon specific functions, that is still left to someone else.
• The user - I.e. the user can determine this. Note that this can be done in several ways, i.e. the user can control the Application or the GUI Framework itself.

Usually, there is some mix of this in most applications. Some keys and conventions are left to the GUI Framework (Cut / Paste / Open File shortcut keys, Dialog button ordering etc), some are hardcoded in the application, and some are configurable by the user. The question is how we determine the mix of these things.

To put it differently. If I have a cross platform application, do I follow the conventions of the different platforms I run on, so it looks like a Windows app on Windows, a Mac app on OSX etc? Or do I create an application that works more or less the same, independent of platform? The latter path is choosen by, for example, Adobe, to a large extent. This means that Photoshop works like Photoshop, not like any other Windows app, assuming we run on Windows. For someone using Photoshop day in and day out, this is great, the application works and looks the same, independent of platform. For someone like myself, who use it now and then, this is a major pain. Things don't work as they do in any other Windows app I know, PS Windows don't have a "close" button on the top right corner (at least not last time I looked) and key combinations that I am used to using doesn't work.

Which path to choose? I don't really know, I am just raising the issue. And while I was thinking about it, I decided to have the new version of the MyQuery GUI much more flexible, with many more functions controllable by the user. This is how I would like PS to work: Have it configurable and provide useful defaults: Work like PS, Work like Windows, Work like KDE etc. To an extent, this is what I am doing in MyQuery now. Now, I don't have all the time in the world, but i have relaized one thing: Having configurable function keys for menu options is real easy in Windows. These are called "Accellerators" in Windows, and I have built a simple framework to support configuration them with easy. I can't beleive more applicatoons don't allow this.

Also, I have included proper exception dumps in MyQuery. This is still on a basic level, but I got the idea on this from Spolskys book. And not to get ahead of myself here, but, I do like some aspects of the book, I do get ideas from it, but it is terribly opinionated (hey, who isn't!) and there are quite a lot of things I do not agree with.

And not even Microsoft got this right. Ctrl-C / Ctrl-V to cut and paste is the Windows convention, right? Try that in aDOS Window in Windows...

/Karlsson
Back to finishing up MyQuery, and spending some vacation time in the Sun