This is one of those seemingly simple concepts that never really gets elaborated. Usually a student learning Java for the first time is told. “Write the following:”
public class Person
{
public static void main(String[] args)
{
System.out.println("Hello World!");
}
}
“Don’t ask why, just do it.” Because of that, I’ve heard this question more than a few times in the last couple years.
(more…)

If you found this site via some sort of Google search, you’re probably looking for answers. I won’t get to that now, but I will warn you that I don’t have them. If you’re a friend or colleague of mine you will have let out at bit of a laugh and settled into your chair with every intent of hating what I’m about to write. Well, I’m glad.

In industrial technology applications we often see projects and ideas labelled with ideas that spark value to other techies.  Terms like ‘robust”, ‘quality’, ‘efficient’, and, to a lesser degree, ‘boffo’. Far less often do we see technology simply described in terms of ease of use.  In fact, I would say it’s downright rare to hear of any steam-powered, hard-left engineers bringing up usability or learning curve when drafting a system designs or applications. Just like football players are interested in beer chugging and cheerleaders, I guess engineers into database normalization and load-balancers.

So what happens when you take that out of the equation? What happens when you take away the pomp and circumstance, when you lose the beer and the cheerleaders? Well, then you have football  or,  in this case, engineering. Take away the sex appeal of massive, scalable availability and mind-boggling complexity; what are you left with? It’s not frequent, sure, but it happens eventually. You’ve got problem solving, pure and simple. Simple calculations, exercises, tasks, and duties.  Sometimes football players have practice, they run drills, lift weights, and sell pizza coupons. Well it’s no different for engineers.

Enter Microsoft Access.

I’m serious, although this is quite a leap (move with me here, I’m trying to keep this short). Access has never, in its entire existence as a RDBMS, won any championship football games, but that doesn’t mean it isn’t a great tool. It’s the football practice dummy and the barbell and the coupon book, but all of this for engineers. Just like them, we’ve got to do the boss’ laundry and earn our keep. We architect the big solutions, sure, but what about the little stuff? What about the parts inventory for the warehouse? What about the customer contact that the boss does once a week and notes in his journal? There’s also that email list that you share information on, but people keep asking the same questions every year or so and nobody keeps any of the information recorded anywhere? What about office supply orders that Debbie does once a week? It’s all little stuff.

Three times, in three different positions, I’ve used Access (or other simple data management tools) to bridge a gap or improve a process that was being done poorly or not-at-all.  Each of these times it’s been a task that I volunteered for, and each time I’ve received more recognition than all of my ‘big picture’ work combined. I didn’t choose Access because it’s fast or robust or sexy (as it is clearly none of these things), but it is definitely quick and easy and portable, not to mention the availability across most corporate IT spaces. It’s not designed to track Walmart’s inventory, but it does get the job done.  After all, who cares about an 18% performance increase on the security log queries when I have this nifty iPhone app that lets me keep track of what I eat every day? Okay, maybe that’s silly, but it’s all little stuff. And the reason it makes a difference is because it effects people.

I’m a proud Access developer. It’s not my day job, and I’m glad for that, but it’s an amazing tool. I pledge to volunteer my services to help improve the processes and daily work of people on whom I rely. I’m also going to use it as a ‘gateway database’; I’ll use it to get into American homes and get kids and adults to try harder stuff like MySQL, Linq, Rails, and Hibernate. But for Dad’s big list of home electronic serial numbers, Timmy’s baseball card collection, and small project CRM…well, I’m on board. Who knows? Maybe even the Microsoft or the EPA will use it.

*Remember: a good developer is an active developer. Please stop engineering for engineering’s sake; it’s not healthy. Put your skills to some good use and fix something or teach someone (or vice-versa).

Sometimes it’s good to have a moment of reflection and take stock of your position in life. I’ve heard that it is when you are at your busiest moments when you need to stop, if only for a moment, and consider all the things in your life. Well, I’m busy. Too busy, in fact, to look at all the things in my life, so I’ll just hit a few of recent technology moves that I’ve made. (Read on…)

(more…)

Subtitle: Makes an A$$ Out of You and Me

I ran into an issue last night with MySQL Workbench that frustrated me. I’m working with a table of medium-low complexity (about 10 Tables and 15 foreign keys) that I’ve inherited in joining a research team.  At this point I’ve reworked the whole thing to be a little more robust, but when I did so I ran into some problems on importing the results into a table.

On some of the tables I was getting an “Can’t create table <x> (errno: 150)”, which turns out to be a foreign key(FK) creation failure. I then realized that all of the tables that were failing were referencing the same core table, but with no visible issues. The primary key on the table was a composite of two  surrogate keys [INT(10) in this case]: one from the parent table and a local one.

I immediately thought that the FK definitions might have been incompatible, and so I combed through some of them for reasons why this would happen. I looked through InnoDB reference documentation in search of limitations in the implementation. Maybe there was a limitation on the number of keys referencing a single table, or on chaining “cascade”, “set null”, “restrict” behaviors, or on the maximum length for foreign key field names. Nothing.

I was up until 2 in the morning in the office last night with this problem with no success and eyelids with exponential density growth. When I got in today I quickly realized that it was just a simple conflict between ‘unsigned’ and normal int values in the foreign keys and relating tables.

My beef: All of the table declarations were made in MySQL workbench. Relationships and everything were formed using the exact same tools for all of it, and yet some of my columns were unsigned and some were not. Additionally, there was seemingly no way to change this value in the interface. Well, it turns out there is.

How to enable zerofill, unsigned, and other flags in MySQL Workbench:

• Go to (Menubar)->Tools->Options
• Open the ‘Diagram’ Tab
• Ensure “Show Table Column Flags” is checked

My biggest beef: A good programmer should never assume that a tool is doing what he wants it to do. If there’s no visible indication that numbers are being given unsigned classification, then that’s your job.  I wasted a few hours on this problem, but my jedi masters should be pleased that I’ve resolved this error in less than 24 hours. PEBKAC ftw.

I was recently visited by a friend who is also an IT professional. Some time during the visit, I casually made the recommendation that programmers should take a drafting course. I was a bit surprised to receive a contradictory opinion, but it was well received and has forced me to further justify my position in my own mind.

My case is rather simple; (more…)

Has this ever happened to you?

Hey, I have this thing you have to see! I’ll send ti via email, it’s 10MBs.
No wait, my email service can’t handle anything that size. It’ll have to go a different way.
Okay, I’ll send it over the IM client. Wait, it says you’re using a different client than me.
Yeah, I’m using Pidgin (read: stupid Pidgin, seeming all helpful)
Okay, log into Skype and I’ll send it as a file.
Okay. Wait why is it only going at 500 bytes a second?!?!? Are you running Windows 3.2? (read: or Linux, har har, but seriously why do Skype file transfers blow?)
Man, I’m going to have to upload it to my FTP server and send you a link. It’ll be faster. Click, click,click,click,click,click,click,click,click,click,click,click,click,click,clickity-clickity-click. Okay, here’s the link.
Man…..that sucked so bad.

I can imagine you, the reader, sitting in your seat and sobbing into a pillow at this point, “Yes, yes I was there….I had to send him a burnnn….a burrnnn…a burned cd!!!” Well, maybe it wasn’t that bad.

So in my own way I fixed it for myself. ShareItNow is a python script that you drag files onto and it will automatically upload said files to the FTP site of your choice and give you a URL to give to your friends and family. It’s simple, but it’s not bad. You do need Python, an FTP server, and a Webserver, but that’s what I have so too bad if you don’t.

Just edit the script to change the value of your server location, credentials, and a web folder the server points to and you will be golden. Once you drag a file or files onto the script it will ask you if each file is binary before uploading. Once it’s done with an upload it will print you a URL that you can use and go on to repeat the process for any additional files.

One last thing, to make Python scripts “Droppable” (meaning you can drag-and-drop a file onto a script to get some magical result) you need to enable a little switch in the registry. I added a .reg file to the project zip using the code in a tutorial on how to make droppable Pyscripts. The funny thing is that this guy wanted to do the same thing I did, but he didn’t share his code. Great minds think alike, I guess.

But I share my code  (Download ShareItNow)   ;D

(more…)

With the Thanksgiving break finally upon me I now realize that I need to be a better behaved blogger. The nice thing about having the blog on my website is that I’m pretty committed to keeping it going despite my occasional/frequent droughts.

It’s been busy for me at school and elsewhere. I’ve started regaining my social life in the midst of my more intense studying aspects in the hopes of finally becoming the well-rounded individual. Also, I’ve finally returned to the developer stage with some interesting projects.

I’ve pretty much abandoned my trivial web projects in favor of a few more serious research and other projects in Python and C++. This is the third time I’ve been programming in C++ and the first real exposure to Python. It’s been fun dealing with interface/facade ideas in these languages instead of using the familiar Java concepts. It’s also interesting to try plugging in these tools into some fun new libraries.

I must say that I don’t despise C++ as I used to. It’s really not much different from Java except for the exaggerated power you feel from managing your own memory.  When I switched over the first time I can still remember how uncomfortable I felt with the lack of the amazing Java library and documentation, but you can overcome that.  The code can obviously be much more elegant at times as the magical pointer garbage can be pretty interesting. The biggest concern is that your code doesn’t start looking like something written by Escher.

Python is fun and different, but I’m not sure that I’m used to it yet. When working with C++ and Python at the same time you start to realize how uncomfortable that void of high level management is. I really have no idea how long it’s going to take me to get really comfortable with the language. You know what I’m talking about; there’s that point where you feel like you’re working on a code assembly line and the code colors and connecting shapes are all in arm’s reach.

The current situation with Python is a lot like that episode of Futurama where Bender is floating through space. A small species of people form on his body and evolve into a fully civilized species and he plays god for a little while. Then he screws up and causes a nuclear war, killing off his little world.  He then meets up with a god-like galaxy and they start talking and figuring stuff out. I think I just encountered god after playing around and blowing up my own mini-universe. Soon I should a good approach for creating life, but right now the options are just blowing my mind.

I’ll keep you up to date and I hope to have some descriptions of my tools and projects soon. Until then, I guess I hope that you’re staying healthy and looking forward to a happy Thanksgiving.

One of the biggest, although non-normalizing, issues that I’ve struggled with is that of the natural versus surrogate keys. Now, I didn’t know what these were a few years ago so on the off chance that you are me in that time period, I’ll briefly explain. A natural key is a unique/primary key that is made up of strictly information that is logically connected with the entity you are storing in the row. The tried and true example is that of a social security number or a detailed title, basically anything that is part of the entity that makes it unique. What makes things hard is choosing the right data to make that storage easier, obviously the same FIRST_NAME occurs in more than one PERSON so you’ll have to use another or more rows (ex. FIRST_NAME, LAST_NAME, BIRTHDATE, BIRTH_PLACE). But it’s a lot easier to use a surrogate key, most often an autoincrementing integer, to just represent each row, because you know that will be distinct.

When instructed in my undergrad database lectures I was told, flat out, to never use surrogate keys. Fortunately, the man I learned this from can now be properly labeled as a “hack” when it came to practical computational theory. He would literally grab his shirt collar and adjust it uncomfortable while making strange, wounded-animal-like noises every time a programmer stuck an extra method call in a constructor or used a surrogate key. His goal was to make us flawless when it came to design practice, but he really just made us terrified that if we did something outside of his strict guidelines he would leap on us with his red pen. Or that the noises he was making was part of the summoning act to bring some giant bird to come and tear our typing fingers off.

Well, we learned to do a pretty good impression of him, and we also learned that what he was saying wasn’t exactly flawless, or even reasonable. The reality of the matter is that it’s OKAY TO USE INTEGER PRIMARY KEYS. First of all, it’s not the end of the world to de-naturalize a piece of data. Plus, not only is it okay, but there are significant benefits in speed and tool interoperability.

The only argument in existence against using a surrogate key is that you are essentially defining rows as being defined by a simple number that has nothing to do with the actual data. Sure, this is a loss, and it’s going to make comparisons between multiple tables a whole lot easier. But what about complex tables that aren’t easy to represent in one or two fields. I remember that same “E” professor made us, for a practical software course, store foreign key relationships to a table using 3 varchar fields. Since we had 2 mappings to said table, this meant 6 varchar fields defining the relationships instead of 2 integer fields that would have been much faster for our thousands of rows and much more readable.

Beyond that, do you know how to fix the entity integrity issue? Really, you can just add a unique key constraint on the same fields you would have used for the natural primary key. By using this method you essentially recreate the exact same restrictions on a faster indexed table for which it is much easier to define relationships.

CAVEAT: But it’s not right to say that a database designer, especially a green one, gets to use surrogate keys off the start. Why not? Because it’s not the best case. To be honest, I think this whole world would (www) be a little bit better off if we could make the initial model work. I will continue to use natural keys whenever it is convenient for the model, but the difference is that I will make a conscious decision about which one to pick.

Call me a glassy-eyed, idealist youth, but I honestly believe that you can test a good programmer not by what he knows but by how he evaluates his options. It’s a fine balance between diving off the board before checking for pool water and hourly, broad-field pH tests, but it’s the one that will make you a solid programmer/architect/designer.

My developer friend, Scott Sloan, has been working on his DB class for some time now and it’s quite a useful tool for doing queries simply. Part of the ongoing movement, between him and myself, is designing a rock-solid set of framework classes that will aid in rapid PHP development. Of course, his project has some stuff to show for it while mine are still awaiting a beta release. But I really do love this class and others from Scott, and I use them in Droplet and contribute as I find need to break them.

The most recent change to this DB class was the addition of exception based error handling, making database connections an entirely simpler creature to deal with. This class does a lot of abstraction, and up until now it’s been virtually impossible to debug it, or any db interactions, without stack traces. Unfortunately, every silver lining comes with a dark cloud.

This cloud, not dealing with Scott, happens to be PHP’s development traditions. Just like most functionality, exceptions are good, maybe even necessary, but the implementation of them was very poor from a security perspective. The fact that you can’t disable the printing of the stack trace from an uncaught exception is inexcusable at best. But I can guess how that conversation went:

-Should we have an option to disable stack printing (specifically of method parameter values) for select Exceptions?
-Why?
-Well, maybe they wouldn’t want the end user to see what was passed in a particular method?
-But you just catch the exception!
-But what if you don’t catch the exception? They see everything!
-Are you suggesting that we write code to protect programmers who are breaking rules? Plus, all production servers have warnings/errors disabled for output, unless their people are idiots.
-Oh….I suppose you’re right.

Here’s my beef: you need to plan on some mistakes. No offense, but haven’t you ever forgotten to catch an exception? Since this is a scripting language, you don’t have the parental compile time warnings or blocks, like Java, to say “Yo, you didn’t tell me to do anything when this code freaks, and believe me it can. Fix yo’ code, homes.” (I assume that a PHP compiler would use a similar compile error vernacular). The reality is that there are many production systems that don’t hide warnings/errors, and even if they did you wouldn’t want password information getting written to a log file whenever you fail to connect to a database.

The key here is a “who needs to know” system, just like I talked about in my blog entry about keys. There should never, ever, ever,ever,ever be a way for the language to “accidentally” print a system password to a user. Even if the developer is a complete idiot! If he passes a password or hash into a function,  he’s not going to think about what would happen if that function would error. He’ll fix that when it happens. It probably sounds like I’m defending the Cro-Magnon programmers of the world, but I’m not…..really.

An even worse PHP prospect is the ability to dump a class with private class values onto a page with one motion (i.e. var_dump). I know that these are all helpful tools in debugging, and that private variables were never meant to be a security constraint in this fashion, but the way they did it DOESN’T MAKE SENSE!! That function should not, I repeat ‘NOT’, be able to print private access variables unless there are appropriate accessor functions. That’s what object oriented design is all about.

I wouldn’t be so hard on PHP if it weren’t for the fact that these examples are the ones that give PHP a bad name. When someone’s data gets stolen on a PHP site it isn’t that PHP is a bad language, it’s that the programmer wasn’t thinking about that specific hole. But there are a lot of spots where developers can not know the rules, or forget a step and accidentally release loads of information to an eager hacker. As part of the group defining how the tool gets made, we need to be careful that the tool doesn’t have a cigar cutter that’s big enough for our “baby developers” to fit their arms in.

Anyways, code smarter not harder!

In a recent article on the blog I link to most, Jeff Atwood took the time to discuss database normalization and its rather blurry scale of reason. As usual, I found myself agreeing with Jeff and questioning a few of my own practices in database design. This is something I encourage all my peers to do, especially those who don’t spend much time contemplating the rules by which you program.

Now, when I say that I agree with the fact that normalization logic is blurred, that doesn’t mean that I abandon it. Quite the opposite, I’m actually kind of a fanatic about getting information nomalized, at least up to about 4NF….because beyond that I think it gets a little masonic, if you know what I mean. But I can also see the reasons that one has for denormalization.  Tools are the biggest one that I deal with because many applications from Oracle and reporting tools cannot deal with complex joins, intricate keys, or other common tricks. There are also speed issues that would get you to commit flattening and other acts of database treason.

The big deal here isn’t the fact that these practices are right or wrong (which they often are…clearly), but rather that there is a lack of openness on the topic in the CS curriculum and among the ‘elite programming circles’. When I got to my first serious development job I saw that people were designing tables with an apparent ignorance to the “Laws of the RDBMS”. It didn’t take me any time to throw my nose up and strut by like a upstart schoolgirl passing a pack of teenage smokers.

The reality is that young developers can’t come out of college thinking that they hold the keys to the ‘Normal Universe’ just because they took a database class or two.  No, you cut your teeth on your first 5 gut-wrenching projects, and at that point you earn the rights to sidestep a database design or two, but not before. It’s not black and white and it certainly has little to do with mathematics that you may have been taught. I’m not saying the math and theory isn’t important or even necessary to get to that point. I’m just saying that your decisions on who to marry comes less from the math knowledge that adding 1 and 1 returns 2, and more about the experience you gain in what works and what doesn’t after that first step.

Now that I’ve compared database design to marriage, theoretical mathematicians to freemasons, and myself to a snobby schoolgirl, it’s time to say that this isn’t the end of this. In fact it’s just the beginning. Hopefully I’ll have a few posts for you on breaking down the illusions that I’ve struggled with in the real world of database applications.

*Two things: 1) Normalcy, especially in the physical universe, doesn’t exist. 2)Just because compared DB laws to smoking and then some laws are breakable doesn’t mean you should go smoke. If you’re of age and you want to fine, but it’s a gross habit folks. Just like ignorance of design patterns. And you don’t want to be loke those freaks, do you?