KWPro.net

Although I've been programming for a while, I didn't get serious until I started programming the web.  After that, I found my career and got fed because of that.  However, 8 1/2+ years later after embarking on that journey, one naturally matures and tries to grow (especially as someone like myself easily grows bored doing the same thing over and over!).  Fortunately, I decided to force myself to embrace a higher standard for myself. 

However, part of me believes that most developers who get onto the web probably don't know a lot more than simple CRUD development.  I mean, it's easy and does take time to get used to initially.  Once you do that, then you can build simple, interactive webpages to your heart's content.  Perhaps, they'll even progress to learn a trick or two in a given language, and maybe progress into a second language and explore AJAX (well these days...).  Maybe even some learn a few things on their Linux desktops .

Despite this, I consider web development by itself far too easy.  I mean, it's almost brain dead in that you merely map inputs to a record in a database and vice versa.  For the most part, I think this might reach the limit of what a lot of web developers understand, especially those from the dot com days.

Of course, some companies challenge this paradigm like Google by having fairly sophisticated interview questions to weed out those who lack fundamental knowledge in computer science.  I decided to peruse some of those questions and realized that many come from brain teasers and algorithm analysis.  With regards to some of the brain teasers, a few are derived from the statistical or discrete mathematical disciplines.  Then when you examine another set of questions, they really come from people's projects in CS at universities involving things like solving recursion puzzles, understanding some areas like graph theory, or even AI.  I wasn't particularly strong in th math part and I didn't do hardcore algorithms and AI, but I started doing some self study on these topics and began realizing what Google was asking.

That said, I started thinking about how the web will eventually progress into next generations of technology.  As I look at many web 1.0 and even 2.0 sites, I see regurgitations of CRUD programming, now with some fancy little interface, but little else in terms of thought process outside of somehow employing social, tagging, etc. into their repetoire of tools for a site. 

Sure things like Crowd Sourcing, etc. are great but there will a time when those systems hit a limit of information and in turn new tools will be needed to make sense of all that information.  That's where having fundamentals in computer science, math, and other modeling techniques start to really come into play.  I mean, Crowd Sourcing is probably a brute force necessary step to get to the next level of system development, which will most likely involve AI, probability, statistics, etc. and other complex models to determine how to extract that information.

If that's the case, mere CRUD skills and some simple-but-cool UI tricks are not going to be enough to evolve the web further.  People are focusing on the low hanging fruits rather than going after some larger issues that will be the true challenges of evolving what this web 2.0 foundation has setup.

In short, we're going to have to start hitting the books again and re-examine the seemingly forgotten disciplines of math and algorithms to see how we can apply those to this flood of information at our doorstep.  As issues of scale decrease and become problems for large hosting facilities like Amazon, Sun, and Google, the next generation of computer engineers will need to cross examine how data (large scales of data) that has been organized slightly better will come together in even more meaningful, discrete units.  Or perhaps what the challenge is not simply do trivial SQL statements like "Select * from BLAH...." but try to answer some real questions that require multitudes of levels of information.  Mere SQL joins themselves will perform abysmally no matter what type of system you'd look at, so it's going to require databases of algorithms themselves to be able to solve certain patterns.

Maybe the next generation of engineers will be employed to brute force design a multitude of permutations on algorithms for answering numerous variations on questions.  Things like Yahoo Answers itself is far too primitive to be a truly scalable solution.  "select * from BLAH where foo = bar" is a terrible solution.  People have to stop looking for a one stop Golden Hammer and realize that a lot more groundwork is needed to compile the new design patterns to answer human questions.   That's what I see at least.