I'll start this off by saying this is pure opinion. I don't have any statistics. Instead I have some gut intuition based on numerous "enterprise Java" projects within my company, and through observing the development community.
I'm currently involved in architecting a next generation systems management framework. It has fairly hefty requirements... monitor and manage millions of objects distributed across the globe in real time. Take out the "millions" bit, and this isn't so bad. There are a bunch of standards out there (JMX, SNMP), there are some off the shelf tools, and you can knock something together pretty quickly that covers a reasonable subset of equipment for small installs. But we need to do better. We need to manage more than just "most" of the equipment, and we need to do it in some of the world's biggest data centers. And as I talk about this with my peers from other groups, I'm getting more convinced that J2EE has poisoned a good chunk of the current generation of architects.
I know this isn't how J2EE has to work, but in most projects, you start them like this...
1) Build your object model, use magic to map it to the DB
2) Build your business logic
3) Build your front end
4) Ship it!
And there you are. You have a nice central database so you don't have to worry about distributed data so much. It's all nice and safe on the disk. You have some adapters to get stuff in and out, and a cozy UI that serves it all up to the user. And it scales... for as much load as you can throw at your desktop.
Then you take it to some place big. And it just flat out doesn't work. It's not a matter of getting a bigger database, because you just can't scale it up past a certain point. You can cluster, but now you need a professional services group to install your product, and the performance gains to be had aren't crystal clear.
It becomes a matter of deciding what goes in there. And when, and how. But looking forward to these issues is met with fierce resistance in design meetings... Building distributed systems is much harder than building something around a central store. Objections are typically based on reading marketing propaganda from database vendors. Responses of the form "well, we'll partition the data when we hit that load. Or use a cluster... yeah, a cluster will fix everything" are extremely painful to work through. After all, some "expert" from VendorX says it will work. Who are you, my peer, to assume you know more than me?
But guess what? Each of these objectors has a product in the field, based a central database, handling far less load than our targets, and none of them scale well.
Scaling out is beating scaling up. Processors are going multi-core, not taking huge leaps forward in the GHz war. Scala and Erlang are gaining in popularity, and with them a different model of parallelism (not new, just different) than that offered by J2EE. The actor model embraced by both of those languages is primarily targeted at small processes within a service. But as you look at larger chunks of the architecture, I think the sanest way forward is to embrace a similar model.
ESB has been a buzzword for years now. I don't buy all the marketing hype around it, but at its core there is a model that the current generation of architects needs to get a handle on. It's very similar in concept to the Erlang/Scala actor model. Yes, it's harder to design around asynchronous messages and a large number of distributed components. But that's how you handle the big problems. And that's why you hire good developers to work on them, not just some random CS grad who happens to be able to regurgitate the latest Spring book.
I'm not saying there isn't a place for the traditional J2EE buildout (rails has a similar model, and works wonderfully for small-medium sized projects). And in fact, even with these asynchronous systems there are almost certainly services that need a DB backing them. But there are other tools that should be in your toolbelt.
My suggestion to Java architects today would be to pick up another language that fosters a totally different architecture. I'm currently biased towards Ruby, Erlang or Scala. There are no guarantees that any of these scale better than Java (and it's fairly easily arguable that Ruby doesn't). But any of these will make you see the world in a different way, and allow you to more effectively consider and evolve your current architecture.
I'm currently involved in architecting a next generation systems management framework. It has fairly hefty requirements... monitor and manage millions of objects distributed across the globe in real time. Take out the "millions" bit, and this isn't so bad. There are a bunch of standards out there (JMX, SNMP), there are some off the shelf tools, and you can knock something together pretty quickly that covers a reasonable subset of equipment for small installs. But we need to do better. We need to manage more than just "most" of the equipment, and we need to do it in some of the world's biggest data centers. And as I talk about this with my peers from other groups, I'm getting more convinced that J2EE has poisoned a good chunk of the current generation of architects.
I know this isn't how J2EE has to work, but in most projects, you start them like this...
1) Build your object model, use magic to map it to the DB
2) Build your business logic
3) Build your front end
4) Ship it!
And there you are. You have a nice central database so you don't have to worry about distributed data so much. It's all nice and safe on the disk. You have some adapters to get stuff in and out, and a cozy UI that serves it all up to the user. And it scales... for as much load as you can throw at your desktop.
Then you take it to some place big. And it just flat out doesn't work. It's not a matter of getting a bigger database, because you just can't scale it up past a certain point. You can cluster, but now you need a professional services group to install your product, and the performance gains to be had aren't crystal clear.
It becomes a matter of deciding what goes in there. And when, and how. But looking forward to these issues is met with fierce resistance in design meetings... Building distributed systems is much harder than building something around a central store. Objections are typically based on reading marketing propaganda from database vendors. Responses of the form "well, we'll partition the data when we hit that load. Or use a cluster... yeah, a cluster will fix everything" are extremely painful to work through. After all, some "expert" from VendorX says it will work. Who are you, my peer, to assume you know more than me?
But guess what? Each of these objectors has a product in the field, based a central database, handling far less load than our targets, and none of them scale well.
Scaling out is beating scaling up. Processors are going multi-core, not taking huge leaps forward in the GHz war. Scala and Erlang are gaining in popularity, and with them a different model of parallelism (not new, just different) than that offered by J2EE. The actor model embraced by both of those languages is primarily targeted at small processes within a service. But as you look at larger chunks of the architecture, I think the sanest way forward is to embrace a similar model.
ESB has been a buzzword for years now. I don't buy all the marketing hype around it, but at its core there is a model that the current generation of architects needs to get a handle on. It's very similar in concept to the Erlang/Scala actor model. Yes, it's harder to design around asynchronous messages and a large number of distributed components. But that's how you handle the big problems. And that's why you hire good developers to work on them, not just some random CS grad who happens to be able to regurgitate the latest Spring book.
I'm not saying there isn't a place for the traditional J2EE buildout (rails has a similar model, and works wonderfully for small-medium sized projects). And in fact, even with these asynchronous systems there are almost certainly services that need a DB backing them. But there are other tools that should be in your toolbelt.
My suggestion to Java architects today would be to pick up another language that fosters a totally different architecture. I'm currently biased towards Ruby, Erlang or Scala. There are no guarantees that any of these scale better than Java (and it's fairly easily arguable that Ruby doesn't). But any of these will make you see the world in a different way, and allow you to more effectively consider and evolve your current architecture.
