Web Services Discussion

From my standpoint, there are two kinds of ways that data in the two-way web will flow in a distributed ecosystem that has the scale of the Internet: query-based and real-time.

Query-based - relational or non-relational; you send a request for and receive a snapshot of data. Sometimes, languages like SQL allow you to define what you want. Sometimes, it's a URL. Sometimes, it's a method invocation on a component. Semantically, there tends to be overlap. E.g., stored procedures are like method invocations, and URL requests indicate where you want a GET or POST invoked.
Real-time - continuous or snapshot based. You indicate interest in living data, and notifications come flying in your direction. The initial notification might act much like a snapshot, but then all subsequent notifications consist only of deltas. At the application level, you may not want every single update for usability or performance reasons. So it's cached and you request snapshots of the live instance of this data in that cache when you're ready for it.

Many standards, technologies, and products exist for these data movement strategies. The question is how something might emerge - as pervasive as the web - that gives you both of them on any connected device. One thing I truly believe is that it cannot involve software that must run on big, expensive servers. (Earth will go flat before a Tibco/Rendezvous daemon runs on my Palm IV.) That's where peer computing comes in. Lightweight, cross-platform WebServices will emerge that allow any device to interact at Internet scale.

I think there is great promise in peer computing as a distributed metaphor, but we just need to get people away from thinking it's limited to stealing songs ;-) To me, peer computing is as much about stealing music as living life itself is about stealing music.

-- PhilipEskelin

Philip, I generally agree with you, and my experiences resonate with query-based vs realtime-based. Working in finance IT, one of the problems I have with "distributed component" specs like EJB is that they seem to be query-biased and didn't have explicit support (until recently) for "realtime" systems, such as those involving market data, positions, or risk calculators that talk through Tibco/Rendezvous.

Interesting that you mention Rendezvous though. I actually think it's one of the most "lightweight" messaging products available, even though the cost certainly isn't lightweight. :-) If you look at the network overhead of RVD compared to other messaging beasts (MQSeries, etc), or even SOAP-over-HTTP, it's quite slim. A model like that would be quite nice on a wireless device, assuming we had wireless availability to handle real time updates. RVD is also much more suited to peer-to-peer than almost any other approach I've seen (barring now-defunct alternatives like ISIS, which was part of the NYSE's communication infrastructure).

The trick is to standardize another "semantic data stream" that fits over wireless-sized bandwidth so we can make these peer-to-peer interoperability dreams into reality. XML-over-wireless just seems to me too "pie in the sky" right now - we can afford to waste bandwidth on land-lines, but not through the air, at least for the next few years. I think some seem to agree with this - I've heard a lot of humming & hawing over the usefulness of WAP/WML compared to custom approaches for delivering content (using, say, the Java 2 Micro Edition - which I believe NTT DoCoMo? is using here in Tokyo for their latest phones). -- StuCharlton

I realize I threw out a controversial statement without providing some background. I too have worked with Rendezvous at a financial organization. The main problem I had in working with RV 4.2 was that the RV daemon had a 2MB footprint, it resided in a separate process, and we had to restart it whenever configuration changes were made. RV has a structured, very packed and proprietary format that it uses to send data from publisher to subscriber. We had to create a hidden window where the RV API would post all updates for all subjects we were listening on. To use the RV API directly was a big pain, and the TibMsg? API made it slightly easier, but the Market Data API made things even easier. Yet we had to work with the MD API team at TIBCO and pistol whip them into fixing bugs we found that rendered it virtually useless.

RV just didn't seem that elegant. It wasn't flexible enough to work across a multitude of devices in an unstructured environment like the Internet, through firewalls and network address translation services, and in a non-proprietary manner that makes e-markets and inter-company business processes more interoperable. Can you imagine IP multicast to a cellphone in a car going through a tunnel at 75 mph? ;-)

Standards like WAP/WML didn't catch as well as I thought they would. There are certain some nascent issues. Connections that break a lot and little bandwidth are a reality today. But I think it will improve over time; software will become more intelligent and active, and our understanding of its application in a distributed sense will evolve to the extent that we won't be treating wireless communication like it's wired. Actually . . . isn't most of wireless communication infrastructure actually wired, with only the last few hundred yards or so between your device and the cell doing RF?

-- PhilipEskelin

I think the thing that caught my attention about RV was the concept more than the product: it was a compact, self-describing protocol for efficient peer-to-peer messaging. Sure, it was proprietary and the older API's were rough, but the newer Java API is a cinch to work in, which is why I'm less negative about it as a product. XML has the self-describing part, but not the efficiency & compactness. Though perhaps we'll find a way to make it more efficient, or we'll just throw our arms up and use a hand-crafted protocol when certain QoS demands are needed. -- StuCharlton

I don't have any experience with the newer Java API, but I've heard it's pretty cool. Still . . . try stamping it onto an embedded device running a KVM that may or may not be reliable or support the services you need to do distributed computing at the scale we're discussing here. SOAP seems to be riding a wave right now, but you're right about efficiency & compactness - it suffers there. One strategy off the cuff might be to compress and base 64 encode it. Still, QoS for SOAP is lacking at this point, but other overlapping protocols such as HTTP do as well. Maybe this is something we can look forward to in an "EJB 3.0" spec (or equivalent). -- PhilipEskelin

Philip - I'm glad you mentioned real-time services in the context of a WebServicesDiscussion. It's been on my mind since reading Microsoft's and Sun's offerings. You hardly hear talk about anything but HTTP. But two-way asynchronous communications, both between two software components and between one software component and its user, are crucial for lots of applications (try implementing instant messaging or Napster without it). Because of this, it seems to me that HTTP just doesn't cut it. We need a more symmetrical protocol. -- RussAtkind

Guys - SOAP itself doesn't mandate the protocol - it just requires that at LEAST HTTP be supported. Personally, I'm seeing a lot of interest in SOAP over JMS (through IBM MQSeries) and even (imagine this) SOAP over SMTP! And yes, there is significant ongoing work on compressing SOAP and providing additional support for QoS and other useful things like Transactions... -- KyleBrown

We are doing SOAP over JMS. -- sg

: I don't get SOAP-over-JMS. JMS is an API, is it not? Not comparable to HTTP. Or does JMS also specify a new network protocol?? SOAP-over-MQ I get. SOAP-over-JMS I do not get.

How are you going with it, Sam? A while back, I had to do a feasibility study of integrating a Java/JMS-based environment with BizTalk, and didn't think it would be too much of a problem (just a lot of work replicating the BizTalk Server in Java) -- RobertWatkins

Can't say much because its NDA but in general, its going extremely well. We have SonicMQ JMS busses geographically dispersed and my group is totally MicrosoftDotNet. We simply deal with everything as XML WebServices. Java components in other places format SOAP and put a SOAP call onto the JMS bus, which the MicrosoftDotNet WebServices and CsharpLanguage components pull off the JMS bus and call the right WebService. By making HTTP, XML and Web Services our standard all throughout we can talk to *anything*. The JMS bus gives us great store and forward capability and robustness. We are having no problems. SOAP messages travel on the JMS busses, we pull it off and we can talk to BizTalk Server, SQLServer, wrapped Oracle databases, whatever. Its a great combination and one where all the different technology groups can share equally. -- sg

: Something I don't quite understand - is it the JMS that gives you store/forward, or is it the SonicMQ product? I thought JMS was just the API you would use from Java to put/get to/from SonicMQ? Does SonicMQ have an alternative API for apps written in not-Java? How does CSharp code pull off SonicMQ? Can't be through the Java classes, can it? -- DinoChiesa

By the way, people should look at what IBM is also doing with SOAP and WebServices. I am delighted to see their stuff. They are doing some good work. -- sg

And for once, I might say the same for Microsoft. The great thing that about SOAP and WSDL is that it provides a common way for Microsoft's and Sun/IBM's programming models to interoperate. I can't wait to see the first application delivered with a VB client and a WebSphere back-end! -- KyleBrown

Yes! It's a whole new world -- sg

It's weird how the EspeakPlatform by HewlettPackard was really the forerunner of WebServices but no-one really hears anything about it anymore.

Charging

How does charging work with WebServices? Presumably there is some way to make a profit ;-) -- GlynNormington

Heh. That's always the problem isn't it. If you bring that up to the standards committees, though, it's as if you made a rude noise in a crowded room. -- KyleBrown

The question of charging has come up a few times in the UDDI forum (for example http://groups.yahoo.com/group/uddi-general/message/200), and there is description of a license-based solution at http://msdn.microsoft.com/library/techart/ssf1lic.htm -- IanMitchell?

Is this a new anti-piracy thing? I mean, it's like, "It used to be that I had to worry about you copying my software if I gave it to you, but now I no longer have to give you a copy of the software I write. With the ubiquity of the Internet, I can run a server, and give you a mere stub that securely contacts my server for everything, and I can charge you a nice monthly fee. Want to use my Large Integer implementation? I'll give you an object that, whenever you want to do some Large Integer arithmetic, just contacts my Large Integer server, sends the numbers to the server along with the operation, and receives the result. Not only that, I'll throw in Quaternion arithmetic, Regular Expressions, and LR(1) parser generation, and live chat-room tech support, all for just $8.99 per month! You'll never have to worry about bugs or upgrades, because I'll always be cleaning up the code on my end!" Of course if the network goes down, you're screwed, and if it's slow, you're screwed, and if the server is full or overloaded, you're screwed, and if I introduce a new bug into the code for a while, you may be screwed... but in the internet of the future, maybe many of these will never happen. Sometimes a server can do things an individual computer can't do. Maybe you can trade Large Integers with other people, or deal with integers larger than you can represent on your local machine (for a substantial additional fee, of course). I can also imagine open standards where you could run your own server of a certain kind and, perhaps, get paid... -- EdwardKiser

For non-RealTime critical apps where you can assume Internet access, this does appear to be the next model of choice. It'll be interesting to see what metering and billing technologies emerge to collect that $8.99 per month. -- MichaelLeach

As a RelationalWeenie, I like the idea of using database queries for "distributed" information. It reduces the burden of having to create behavioral interfaces in advance of potential uses. A relational interface would say: "Here's the possible data, take as needed and you do what you want to with it once you get it". There is less it has to anticipate in advance with regard to "features", because it is generally non-behavioral (other than things that queries can do like joins, sum, average, group, etc.). Whether there are protocols or techniques to make such secure to the public or a wide range of businesses has yet to be seen that I know of. A front-end would have to prevent things like cartesian joins, billion-record dumps, etc. I wonder what an Oracle sales-person would propose?

Is it just me, or are there some others who feel a bit skeptical about WebServices? As if all the hype about them is so great it cannot possibly be true.

Does anyone have some success stories to share? (Or some "failure stories").