I got hooked on creative programming during the brief few years I sullied the field of software development by working as a programmer. I had a clear sense of myself as an explorer, taking the Apple II places no one had ever taken a computer before. The industry’s relatively few books on algorithms and program structure were unknown to me, and I wouldn’t have understood them if I had blundered into them. So I learned to program through trial and error, intuition and experimentation. When I got a program to run – and I’m still proud of the apps I created in those days – my triumph was both practical and artistic. For me, programming was a creative endeavor.

I know I’m not alone. I’ve talked to many programmers over the years, and the chance to exercise creativity is one of the things they like best about software development. Many of the developers I talk to these days are preoccupied with mundane tasks like extracting data from mainframe tables, formatting information for output, and patching up apps with new layers of data-security scaffolding. It doesn’t look like creative work to me, but I’ve learned that creativity refers to the way you approach a problem, not the problem itself. Even the most mundane task can be accomplished in a creative way (without compromising the quality of the app or missing the project deadline, of course) by practitioners who see themselves as creative.

I hope you see your work as an outlet for creativity. For me, that was what made programming satisfying. Yes, I liked solving users’ problems. I loved dismantling complex challenges and finding the inner simplicity. But mostly, I saw programming as a means of personal expression.

Everyone who works in a creative discipline draws a blank sometimes. It’s hard to be creative on demand, especially when facing tight deadlines.

Of course, little we do as developers is truly original. For me, at least, a creative triumph was when I arranged well-known, frequently used programming tropes in novel ways that met my application’s needs. You might call this “combinatorial creativity.”

I was reminded of this the other day when I happened upon a letter that Mark Twain once wrote to Helen Keller. It seems that Keller had been accused of plagiarism. The charge was dismissed, but Twain still had more than a little to say about it, and about creativity itself:

Oh, dear me, how unspeakably funny and owlishly idiotic and grotesque was that “plagiarism” farce! As if there was much of anything in any human utterance, oral or written, except plagiarism! The kernel, the soul – let us go further and say the substance, the bulk, the actual and valuable material of all human utterances – is plagiarism. For substantially all ideas are second-hand, consciously and unconsciously drawn from a million outside sources, and daily use by the garnerer with a pride and satisfaction born of the superstition that he originated them; whereas there is not a rag of originality about them anywhere except the little discoloration they get from his mental and moral calibre and his temperament, and which is revealed in characteristics of phrasing. When a great orator makes a great speech you are listening to ten centuries and ten thousand men – but we call it his speech, and really some exceedingly small portion of it is his. But not enough to signify. It is merely a Waterloo. It is Wellington's battle, in some degree, and we call it his; but there are others that contributed. It takes a thousand men to invent a telegraph, or a steam engine, or a phonograph, or a telephone or any other important thing – and the last man gets the credit and we forget the others. He added his little mite – that is all he did. These object lessons should teach us that ninety-nine parts of all things that proceed from the intellect are plagiarisms, pure and simple; and the lesson ought to make us modest. But nothing can do that.

Web recommendation: I know too much about how computers work to consider robots a threat to my life or my job. But this series of articles got me thinking. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He enjoys a shot of Scotch now and then.

I encountered a fascinating Web site today. The care and love behind its construction are evident. The personal messages from the site’s managers reflect their commitment to exceeding users’ expectations. The idea is novel and the implementation, from what I’ve seen, is first-rate.

So what’s the problem? The problem is that the site fills a need that doesn’t exist, as far as I can tell.

The site is compilr.com. It’s an online IDE and compiler for Python, node.js, PHP, JavaScript, HTML, C, C++, Ruby, Java, C# and VB.NET. Think of it as Google Docs for developers.

compilr.com instantiates a bunch of hot buzzwords. It’s cloud-based. It’s virtual. It’s SaaS. It supports collaboration. All good, right?

Yeah, it’s all good. But I’m left with one question. Who needs it?

If you’re a professional, you want your development environment on your machine, or at least on a local server where you can control the installation. You want to know which patches and revisions have been loaded, and which versions of the libraries are installed. At compilr.com, you don’t know what compiler is running in the back end. If the installed libraries are documented, I can’t find the details. These details matter.

Also lacking are version control and facilities for testing and debugging. Judging from the site’s user forum, these features have been on the back burner for a couple years.

I want to like compilr.com, but I think it’s a solution in search of a problem. It’s easy enough for students and casual developers to download free development environments from the Web. Professionals know where to get their power tools. A Web-based development environment sounds like it ought to be useful, but once you think about it for a few minutes, the usefulness evaporates.

Too bad.

Web recommendation: Here is another alarming article, this one from the good folks at Wired. Little by little, piece by piece, a larger picture is becoming clear, and it’s a disturbing one. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He wishes Aaron Sorkin still had a series on the air.

The world is full of people eager to sell you training and specialized information. Authors and publishers push books. Consultants sell insights by the hour. Speakers at seminars and conferences promise to revolutionize your thinking.

I’m the last person to disparage these conduits to learning. Although it is commonplace to dismiss these teachers as principle-free cynics who craft their messages to the marketplace, my experience is that authors, consultants, and conference speakers believe in their messages, very often passionately. In general, the money you spend on this kind of information is money well-spent.

But when I say “become a better developer,” I’m not talking about picking up the basics of a new programming language, or learning how to manipulate a new set of APIs, or grokking how to craft code for a new platform, or seeing how to translate the principles of a development method into practice. Those are all examples of skill acquisition, and skill acquisition is a fine thing. But it doesn’t transform you. If you were a mediocre programmer before a course in Dart or F#, then at the end of the course you’ll be able to code in a mediocre fashion using a new language. You’ve acquired new skills, but you’re the same developer you always were, not a better one.

If you really want to be a better developer, you’ve got to look inward. Understanding lambda expressions may help you write better code in certain situations, but being a better developer is more likely to come from a higher level of commitment to quality, a greater level of enthusiasm, enhanced patience, and more willingness to communicate clearly, frequently, and honestly with colleagues.

That’s the secret behind McCarthy BootCamp, which I wrote about recently. Jim and Michele McCarthy don’t teach any programming tricks during their training sessions (what they teach, mostly, is personal integrity – but you didn’t hear that from me). Nonetheless, they may just help you become a better developer.

Because when it comes right down to it, there’s just one way to be a better software developer: Become a better person, and then develop software in whatever way seems most natural.

Web recommendation: This is an amusing slide show that does a pretty good job of displacing a belief that is held by no one I know. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He’s done some simple home repairs today and is feeling uncommonly proto-masculine.

The other day I told you about NELL, the Never Ending Language Learning system at Carnegie Mellon. NELL is an AI system that is acquiring its knowledgebase by browsing, parsing, and assimilating content from the Internet. NELL is the real-life analogue of one of my favorite characters in modern literature: Helen, the AI system at the heart of Richard Powers’s astonishing, heartbreaking, brilliant Galatea 2.2.

There is no way I can set out for you all of the wonders of this wonder-packed novel. First, you should know that the main character spends a year training a neural network in hopes of winning a Turing test consisting of a comprehensive graduate examination in literature. The book provides more than a bit of detail about successive versions of the net and how they are structured and trained. A former programmer, Powers gets the technical details right.

You should also know that the main character is named Richard Powers, and that he shares many biographical details with the author of the novel. You can never be sure, reading the book, when you are reading fiction and when you are reading autobiography. This ambiguity sounds confusing or distancing in theory, but in practice it adds to the complexity of the novel in a delicious way and supports one of its major themes.

The book is densely layered with allusions to current events, computer science, the novels of Richard Powers, and, especially, literature. The story of Pygmalion and Galatea plays a central role, as you might guess from the title. There are liberal borrowings from Proust as well, and any number of other books.

I don’t think the literary baggage should prevent you from reading the book. It’s a terrific stand-alone read. The allusions and literary in-jokes make the reading experience richer if you get them, but you can enjoy the book just fine without them.

You learn all about these extra layers when you pursue a literature major. For example, I learned something interesting about the plays of Shakespeare and his contemporaries. They were written, as you probably know, toward the end of the 45-year reign of England’s Elizabeth I. Succession was a taboo issue in those days. Elizabeth had no heir and had not named a successor. Everyone knew that she was an old woman and wouldn’t live forever. But talk about succession could get you thrown in the Tower of London as a traitor. Shakespeare knew that these events were on the minds of his audiences. That’s why so many of his plays deal with ambiguities and battles over royal succession. None precisely mirrored the situation with Elizabeth – Shakespeare couldn’t risk execution as a traitor – but he broached the issue delicately, to the delight of contemporary audiences. Most modern theater-goers don’t know these historical details, and it doesn’t matter. The plays are still entertaining. It’s just the same with Powers and his allusions.

It’s fruitless to attempt to name a favorite book. But I have read and reread Galatea 2.2 many times, and recommended it to others who have found that they, too, love it. The writing is lively, erudite, engaging, and creative. The double-plot includes inversions and interplay that draw you in. The characters are memorable and all of them, ultimately, touch you with their humanity.

You can’t spend all your time coding. Pick up a copy of Galatea 2.2. You’ll be glad you did.

Web recommendation: Well. I didn’t realize the P vs. NP problem was so central to computing. This article is fascinating. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. He loves every Richard Powers book he has read, and he has read them all.

One of the most appealing aspects of the Agile movement is the breakdown of development efforts into themes, epics, and stories. I have long thought that the real problems in software development have less to do with algorithmic complexity and more to do with the difficulty of getting everyone on the same page. The theme-epic-story structure helps everyone see his place as part of a tale that has a beginning, middle, and end. We know how stories work.

All of this resonates powerfully with the work I did at Columbia in pursuit of my degree in English and comparative literature. Stories, I have come to believe, are how we make life make sense. Life is, in essence, a series of moments. The events that fill the moments have no intrinsic meaning. It is only our sense of story that gives each moment significance. This is, loosely speaking, the essence of existentialism.

Computers don’t have any sense of their existence as art of a story. To a computer running conventional software, facts have no meaning and events are not judged against values. A word-processing program spell-checks a ransom note as happily as a poem.

But what if computers comprehended the world, understood the relationships among the things whose verbal stand-ins they manipulate so fluently for us?

That’s the goal of NELL, the Never Ending Language Learning system at Carnegie Mellon University. Researchers have given NELL a rudimentary set of categories and verbal relationships, and turned it loose on the Internet. The system, which runs on supercomputers donated by Yahoo! with financial support from Google and DARPA, uses the Internet to check its hunches about relationships – what books an author has written, for instance, or whether a particular phrase refers to an item on a menu or a bill under congressional consideration. NELL’s learning isn’t supervised. The system constructs facts and relationships by assembling webs of information sources and evaluating their credibility. NELL scans and rescans millions of Internet pages and from the assembles a view of the world.

The results are impressive. NELL has composed more than 15 million beliefs about the world, structured as simple statements like playsInstrument(George_Harrison, guitar). Every day, NELL learns to read a bit more efficiently and accurately. It hones previous statements of belief and compiles new proposed beliefs. The researchers don’t see an end-point for the project. They intend to let NELL keep learning and learning indefinitely, and see what happens.

NELL doesn’t always get it right, of course. For a few days, before researchers stepped it, it believed Internet cookies were baked goods. The difference between “She bought the bread with the money” and “she bought the bread with the poppy seeds” still flummoxes Nell. For now.

If any of this sounds interesting, check out the NELL project page at Carnegie Mellon. You can download NELL’s knowledgebase if you want. You can even follow NELL on Twitter, where you’ll receive a continuing stream of the system’s new conclusions about how the world works.

Researchers say that NELL is an attempt to simulate the way human beings learn – cumulatively, gradually, with tentative assertions gradually becoming more certain. It’s true that NELL represents a new approach in that regard. But I don’t think NELL matches the human model. Human learning relies upon an extremely rich broadband stream of feedback. We learn that things fall to the ground by repeatedly dropping them and seeing it happen. NELL evaluates and weighs inputs, but it doesn’t seem to test assertions against reality. It’s unreasonable to think that NELL could perform such tests before its hardware includes robotic extensions. But until that day, I don’t think it’s fair to say that NELL’s belief-construction process mirrors human learning.

As for the differences between NELL’s beliefs, which are factual assertions that the system drops when they are disproved, and human beliefs, which members of our species cling to in the face of overwhelming contradictory evidence – well, that’s a topic for another day.

Web recommendation: This story is both important and chilling. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. Reading about NELL made him itch to do some AI coding.

Organizations concerned about the growing popularity of bring your own device to work policies may be comforted by the release of RADE3. Retriever Communications, a provider of mobile force field automation, released its newest development environment this week. RADE3, which stands for Retriever Application Development Environment, allows developers to create device-agnostic native applications for a variety of business workflows.

Developers, according to Mary Brittain-White, CEO of Retriever Communications, can create applications on RADE3 without having to go outside of the environment to deploy the code.

RADE3 writes an abstraction lawyer above the OS-specific API and then enables the applications to allow end-users to have full control for editing assets, adding information or receiving notifications, Brittain-White said.

Brittain-White said that Retriever’s write-once, deploy natively approach does not include HTML5 mobile Web-based applications that are wrapped, as many of the other development and deployment tools do. She said it is important for these enterprise applications to have a larger database running in the background to allow full support for in the field workflows, such as a contractor who needs to input data into an insurance claim form. These types of workflows cannot be supported by HTML5 mobile Web-based applications, she said, because they need to work instantly and correctly for each and every transaction. Lag time is not acceptable for many organizations creating business applications at the enterprise level, she said.

RADE3 and the Mobile Enterprise Application Platform (MEAP), which powers the write-once deploy natively feature, is available as a free download. These applications are built with a GUI interface that delivers a dynamic user experience even when users are offline or in an area with sporadic coverage.

Microsoft is planning changes to the user interface it will deliver in Visual Studio 11, based on feedback from its early beta testers. They deal mostly with color pallettes, styling, font size and icon usability, but based on the screenshots included in the blog post heralding the changes, they are a marked improvement over the first go-round. There is, of course, no timeframe for the next, RC release. More detail about Visual Studio 11 can be found here and here.

After all of that noise and tumult, the Oracle versus Google lawsuit is reaching the end of its first phase. First phase, I say, because Google is already asking to have a mistrial declared. The jury had been unable to bring a unanimous verdict to the court on all counts, but today, they brought in a major win decision for Oracle.

The jury found that Google violated Oracle's copyright by copying the Java API calls in Android. And, hoooo boy, we could debate for months on the validity of this decision. But at its core, it shows us that all those crazy years of licenses, Java development and TCK's meant basically nothing until right now when a court has actually had to consider all those lawyer-infused pages of licensing agreements and JCP legalese.

In the end, I predict this whole thing will result in money changing hands and no real damage to Android in the long term. But this could also be seen as a chilling realisation for companies around the world that you now must absolutely use only the proper distributed version of a language, or else you could be sued.

Google has issued a statement: "We appreciate the jury's efforts, and know that fair use and infringement are two sides of the same coin. The core issue is whether the APIs here are copyrightable, and that's for the court to decide. We expect to prevail on this issue and Oracle's other claims."

And, as has been the case throughout this trial, Groklaw has covered the whole sordid affair more thoroughly and completely than anyone else could ever hope to accomplish.

Hacker News, as well, has good commentary, which boils down to the following: Google is accused of copying the range checking code for their implementation of a Timsort in Android. Timsort is a standard sorting algorithm, and it's actually one of the default sorting methods in Python. Are we really surprised a jury composed of non-developers didn't quite understand that an algorithm's implementation across different applications is likely to be quite similar. Kind of like how a hamburger is always going to have two buns and meat between them?

The sum total of the jury's decision rests on the fact that the code for range checking in Timsort in both the JDK and in Android is the same:

From OpenJDK:

private static void rangeCheck(int arrayLen, int fromIndex, int toIndex) {

        if (fromIndex > toIndex)

            throw new IllegalArgumentException("fromIndex(" + fromIndex +
                       ") > toIndex(" + toIndex+")");

        if (fromIndex < 0)
            throw new ArrayIndexOutOfBoundsException(fromIndex);

        if (toIndex > arrayLen)
            throw new ArrayIndexOutOfBoundsException(toIndex);

    }

From Google:

private static void rangeCheck(int arrayLen, int fromIndex, int toIndex) {

        if (fromIndex > toIndex)
            throw new IllegalArgumentException("fromIndex(" + fromIndex +
                       ") > toIndex(" + toIndex+")");

        if (fromIndex < 0)
            throw new ArrayIndexOutOfBoundsException(fromIndex);

        if (toIndex > arrayLen)
            throw new ArrayIndexOutOfBoundsException(toIndex);

    }

}

Yeah, they're the same, but if the whole case over infringement and copyright violation comes down to less than 10 lines of code, in amongst millions of other dissimilar lines, it's tough to swallow. I sincerely doubt this lawsuit is truly over.

While I wasn’t looking, the discipline of user-interface design was rechristened UX – short, I am led to believe, for “user experience.” That seems a rather breathless, wide-eyed way of referring to the shopworn set of UI widgets that let users interact with applications, but so be it. The user-interface world has always suffered from stylish overstatement.

UX is like the weather. You know – everybody talks about it, but no one does anything about it. Every programmer I meet is a self-appointed expert in how user interfaces ought to work. Everyone seems to know what makes an application “user-friendly” or “compelling” or whatever the current buzzword is.

At the same time, our industry perpetuates the myth that it supports a specialized cadre of user-interface designers, UX engineers with deep insights into user expectations and how to meet or exceed them. But tell me: Have you ever met a UX engineer? I haven’t. I think there may be some at Microsoft, behind the scenes of the Windows 8 team. Everywhere else, UX is understood to be part of the software developer’s expertise.

There seems to be a conflict between best practices in user-interface design and the incremental development processes we call Agile. UI designers count pixels and specify detailed behavior of program elements before they are coded. The entire discipline of UX depends upon a waterfallish development model in which the UI takes emerges as vapor from the analysis phase and solidifies into the spec during design. Remember analysis and design?

Bloggers across the Internet have attempted to downplay the incompatibility between traditional UI design and modern development models. You can read some of these articles here: InfoDesign, Interfaith Marriage: Experience Design Meets Agile Development (by my old friend Larry Constantine), The Integration of User Experience into Software Development, Great User Experiences Require Great Front-End Development, Twelve Emerging Best Practices for Adding UX Work to Agile Development, Agile + UX: Six Strategies for More Agile User Experience, Adapting Usability Investigations for Agile User-Centered Design, Bringing User-Centered Design to the Agile Environment, Design Chunking with Scrum…well, you get the idea. There’s a lot out there.

What strikes me about all these articles is that the programmers and writers and designers who created them are so earnest and eager to stand on one foot, and squint, and somehow make a discrete UI-design phase seem like a natural part of the Agile development model, not something that is awkwardly grafted on.

UI design has always been a black art. If you’re old enough, you may remember the Babel of UIs every user had to deal with before Microsoft and Apple imposed UI standards on developers. If you think things are bad now, you’ve got a short memory. UIs were really dreadful, then. And while there is always room for improvement, standard UI widgets and guidelines have done a lot to make the user experience – er, the UX, I guess – significantly better.

Web recommendation: Intel introduced the 2012 version of its SDK for OpenCL a few days ago. If you’re looking to write parallel code that takes advantage of the GPU for compute-intensive tasks, this is a significant release. You can learn more about the SDK – and download a copy free – here. J.D. says check it out.

J.D. Hildebrand has written hundreds of articles for dozens of publications and online communities dedicated to software development. His lovely wife is uncharacteristically impatient for him to finish work today.