Subscribe: Real-Time Enterprise
Preview: Real-Time Enterprise

Real-Time Enterprise

This blog is devoted to what is new and exciting in technologies that enable the Real-Time Enterprise including Rich Internet Applications, Active Data Warehousing, Enterprise Dashboards and other topics.

Last Build Date: Fri, 15 Jun 2007 19:18:20 -0700

Copyright: Copyright 2011

Flex 2 Developers Conference(FlexManiacs) Presentation on Coordinated Visualizations

Fri, 15 Jun 2007 19:18:20 -0700

Highly Coordinated Visualizations using Charting
Type: Lecture
Presented by: Arshak Navruzyan
June 25 - 26, 2007 in

Coordinated visualizations allow users to simultaneously view inter-related data from multiple perspectives to perform correlation analysis and discover patters that might otherwise go overlooked. The session will discuss how Flex 2 Charting component coupled with a coordination API can deliver sophisticated coordination actions including hovering/filtering, coordinated selection, coordination paging/scrolling and others. It will also discuss coordinations with custom developed visualization components such as Treemaps.

Fig Leaf Software/FlexManiacs 2007 Conference
RFF Conference Center
Fig Leaf Software Training Center
1400 16th Street, NW
Washington, DC 20036

AppFusion Case Presentation at MIT Enterprise Forum

Wed, 30 May 2007 02:29:57 -0700

AppFusion - Case Presentation
Starting a Business Intelligence Revolution - How to Rise Above the Noise

Arshak Navruzyan, CEO

Wednesday, June 20 , 2007

The Salk Institute

BO finally delivers zero footprint! ...ohh they mean carbon

Sun, 20 May 2007 14:52:46 -0700

Business Objects press release about a new Online Community should be read carefully. Zerofootprint doesnt mean "enterprise-wide deployment without having to install software on desktops they are referring to carbon emissions.

The visualization tool that BO is positioning as a possible solution still unfortunately has a substantial desktop footprint.

SAP Acquires Outlooksoft

Tue, 15 May 2007 17:59:52 -0700

As previously predicted SAP has decided not to roll over and play dead in response to Oracles acquisition of Hyperion. Last week SAP announced its One Small Step for Desktop RIA

Sun, 13 May 2007 13:57:50 -0700

Adobe has released the Alpha version of its much-anticipated cross-OS runtime for deploying desktop Rich Internet Applications Apollo.

Apollo enables people to more easily maintain a connection with information in their web applications. Just like a desktop app, Apollo applications have an icon on the desktop, in the Windows start menu, or in the OS X dock. Also, when youre running a web application today, its a separate world from your computer. You cant easily integrate local data with your web app. For example, you cant just drag and drop your local contacts onto a web-based mapping application to get directions to your friends house. With Apollo applications you will be able to this kind of integration as it bridges the chasm between your computer and the Internet.

Adding to the capabilities of Adobes existing Flex 2.0.1 platform, Apollo is an attempt to put RIAs on an equal footing with Desktop applications. Although exact specifications for Apollo are still somewhat tenuous, Adobes product management has outlined the following areas for Apollo to build on the capabilities of Flex:

  • PDF support
  • Online/offline APIs
  • Full top-level HTML application support
  • Settings/data persistence APIs
  • Drag and drop support
  • Copy and paste support
  • Native file picker dialog boxes
  • Full native window support
  • File extension registration
  • Launching an application to handle a file type
  • Full control of the right-click menu
  • Transparency in HTML

It has been over three years since the release of Flex 1.0 by Macromedia. Only Adobe knows for certain how big the Flex community really is, but perhaps one can get an basic indication from the following stats:

Adobe can cause a fundamental shift in Desktop applications if:

  • Adobe manages to build a community orders of magnitude bigger for Apollo than it has for Flex
  • Adobe makes Apollo SDK open source and opens up the various protocols for data services such as AMF, RTMP (not advocating making LifeCycle Data Service Open source but allowing others to build open source alternatives easily)
  • the $100 million that Adobe plans to invest in venture capital over the next 3-5 years goes into pipes and smokestacks type of technology rather than to frivolous startups in Web 2.0

I have seen the future of BI and it is…Desktop RIA

Sun, 29 Apr 2007 02:07:48 -0700

LinkedIns new Toolbar for Outlook gives us a glimpse into the future of BI. Whats so unique and exciting about this little desktop utility is it takes an application you already use and adds some very useful information without any additional effort.

A few days ago, I received an unsolicited email that normally I would force myself to read perchance that its from someone I know or a referral , in which case I make it a point to respond even if the email is a blatant sales attempt.

The LinkedIn integrated window (which comes back instantaneously) told me that the email is from someone whos job is lead generation (so it wasnt specifically addressed to me), second the person is 3 degrees away from anyone I know and as an aside, the person is actually in LA even though his signature read PA.

Whats exciting about this utility is that it marries a desktop application (Outlook) with a web service (LinkedIn Website) and in the process eliminates any disconnects that current desktop-only and web-only applications tend to have.

In order for Business Intelligence to undergo a similar revolution in usability and productivity, two things have to happen:

  1. Databases need to become more web friendly (Michael Franklins presentation which touches on DataSpaces)
  2. A desktop Rich Internet Application platform (such as Adobes Apollo) is needed to truly bridge the gap between the web and the desktop

This is how an Adobe blog describes desktop RIA

Apollo enables people to more easily maintain a connection with information in their web applications. Just like a desktop app, Apollo applications have an icon on the desktop, in the Windows start menu, or in the OS X dock. Also, when you’re running a web application today, it’s a separate world from your computer. You can’t easily integrate local data with your web app. For example, you can’t just drag and drop your local contacts onto a web-based mapping application to get directions to your friend’s house. With Apollo applications you will be able to this kind of integration as it bridges the chasm between your computer and the Internet.

Adobes commitment to desktop RIA will make #2 into reality in the very near future. #1 is going to be much harder. The folks that you would expect to be innovating in these areas are still obsessing over spindle speeds and cpu cycles.

If at first you don't succeed, try, try again! Business Objects acquires Cartesis

Tue, 24 Apr 2007 02:37:54 -0700

Less than two years after acquiring SRC (Aug &'05), Business Objects has announced its intention to acquire another enterprise performance management (EPM) player, this time it's Cartesis S.A. As an indication of how serious BO is about getting into EPM, it paid $100M for SRC; now BO is raising the stakes by paying $300M for Cartesis a French company with its roots in Compagnie Générale des Eaux and PricewaterhouseCoopers. Unlike Cognos' acquisition of Adaytum, the BO/SRC/Infommersion combination didn't seem to get a lot of traction in the market and coupled with the recent Oracle acquisition of Hyperion, BO management probably felt pressured to revisit EPM.

It remains to be seen how well Cartesis will integrate into the BO stack, which other than a database, now has one (or two) of everything. Furthermore, although Cartesis certainly had Global and US based customers, the core of the customer base appear to be European companies and if there is one thing that is fairly unique between regions it's finance systems: planning, consolidation, statutory reporting, etc.

Lingua Franca for Event Stream Processing

Tue, 24 Apr 2007 01:42:29 -0700

Event Stream Processing (aka: Complex Event Processing) enables companies to monitor event streams, detect and analyze event patterns, and cause actions. Although not entirely dissimilar to rules-based engines such as Corticon, Blaze Advisor and ILOG with ESP emphasis is placed on processing events "mid flight i.e. as they occur in originating sources and before they hit any type of permanent storage. Tower Group estimates that spending on ESP will grow tenfold from $67 million in 2006 to over $600 million by 2010.

An application that clearly demonstrates the benefits of ESP is Algorithmic Trading on Wall Street. An ESP application can take data coming from ticker plants and route the first arriving quote from across multiple exchanges that meets a traders specific criteria.

As ESP is still in its infancy, vendors have developed very distinct approaches for how ESP processes are set up. The one approach that stands out is that of SteamBase, which by working with various academic groups, has created a language called StreamSQL. StreamSQL syntax is similar to relational SQL and both Data Definition (DDL) and Data Manipulation (DML) elements are present however unlike relational SQL which returns query results upon reaching an "end of table condition, StreamSQL relies on time windows to return results when desired conditions are met.

Other predominant approaches are that of products such as Aleri, Apama (now part of Progress), Coral8 and Portware which orchestrate ESP through either a proprietary scripting language or a set of APIs for a handful of environments such as Java or .NET.

Aside from providing the familiar paradigm of SQL, StreamSQL also has the potential of being generated by third-party products such as tools for data visualization and analysis. Although some vendors (for example Apama) have chosen to provide "complete solutions themselves including the IDE for orchestration and dashboards for monitoring, StreamBase welcomes third-party generated StreamSQL as evidenced by a recent statement by

Lure of the Multifunctions

Fri, 23 Mar 2007 23:59:21 -0700

An increasing number of vendors (particularly those positioned in Operational BI) are advocating their low-latency data stores as a way of monitoring Key Performance Indicators from various enterprise sources. Vendors typically name this their “Analytic Engine” which provides such features as:

  • Agents that collect, filter and transform content from various sources (databases, messages, applications)
  • Memory-resident (or disk if the recovery option is turned on) storage of information harnessed by the above Agents
  • Business rules that set off actions (such as Alerts) when certain conditions are met

Although such solutions may provide a method for an organization to be up and running quickly because the extraction, storage, business rules & presentations (sometimes even the hardware) are all packaged together, their true role in the Enterprise needs to be closely examined.

Upon reflection one may come to the conclusion that these solutions do not openly compete with the Enterprise Data Warehouse yet they may be furtively discouraging the organization from making the larger investment in building a comprehensive reporting infrastructure based on a relational data model. Each of the functions provided by these products assumes the role of some part of the EDW stack:

Function Replaces
Agents ETL
Store Database
Business Rules SQL

However organizations need to realize that the price of this seeming simplicity and accelerated delivery may by be these solutions’ inability to handle complexity and their less than enterprise-class performance and scalability characteristics.

Conversely as activity increases in the EDW space, (Netezza IPO announcement and DATAllegro 3.0 release) , EDW vendors should take note of what these vendors are up to and respond by including greater presentation and analysis capabilities in their stacks to avoid losing customer mind-share.

Barrage of Capital Markets Apps

Sat, 17 Mar 2007 18:23:56 -0700

The resume of a Derivatives Business Analyst caught my eye because it made reference to more systems than the typical IT resume:

Bloomberg, Open Link Endur, Charles River Compliance, Charles River Investment Management, Charles River IMS, Sungard InTrader, Reuters, SunGard Global Trader, Summit, Portia, Calypso, Triple Point energy deal valuation application, Charles River Compliance, Latent Zero (Sentinel, Minerva and Tesseract modules), Wall Street Systems, Quotron, Dow-Jones Telerate, Bloomberg Gateway, Bloomberg Portfolio Order Management System, SunGard Middle Office Manager

and if thats not enough, the person is also proficient in generic analysis tools:

SQL Query, business intelligence tools such as Business Objects, Cognos, MS Excel Pivot Tables, SQL, and Show Case Query and Report Writer

Becoming truly proficient in such a multitude of tools has to be a career in itself, which makes you wonder when do they find the time to do the actual analysis?

It can be a sobering experience to see what analysts and traders use on a daily basis to do their jobs: a combination of home-grown web apps, highly sophisticated excel applications that may have more macro code than data, industry apps, etc. which all leaves you with the impression that one place where time is not money is on Wall Street.

Enterprise Web 2.0 can change all this by creating composite applications that do everything that a Wall Street analyst or trader needs from one user interface in real-time and there is a pot of gold awaiting the software company that gets it done in less than a decade.

Sudden Rise of Appliances

Fri, 16 Mar 2007 08:36:27 -0700

In the same week that Gartner declared the term “Open Source” meaningless, “Appliance” is starting to run the same risk.

Vendors ranging from Enterprise Data Warehouse to Business Intelligence have made acquisitions and put together alliance programs to hastily position themselves in the appliance market.

  • Cognos acquired Celequest (LAVA appliance)
  • Business Objects Open Appliance Initiative (OAI)
  • DATAllegro announced V3 on commodity hardware made up of Dell, EMC

The BO announcement doesn’t make it clear if the appliance vendors are going to allow the BI apps to run on their hardware. The statement “customers will have the flexibility to choose the right mix of data warehouse and business intelligence technology” suggests that “mix” means “blades” whereby an a certain number of blades are devoted to (or shared with) data warehouse and the remaining blades are for the BI engine.

If this is in fact what’s behind the BO announcement, there is a name for this type of a “multi-purpose” appliance, it’s called a “server.”

Of course the real reason could be much more sinister such as some obscure accounting rule that let’s you immediately recognize revenues from software if it comes pre-installed on a piece of hardware.

Nonetheless, whatever the motivation, it has to be self-serving. By coupling BI with a DW appliance, both vendors hope to do more business than they can independently.

Lack of customer involvement in the BO announcement is a good indication that this wasn’t customer driven.

Loading the Active Warehouse

Fri, 16 Mar 2007 06:39:39 -0700

Numerous options are available for loading data from operational systems into the Active Warehouse in real-time or near real-time. The options vary based on cost, complexity and not all options are available for every environment.

Most techniques have been put together to overcome the limitations of application level triggers, timestamps and table differencing. Almost all include some level of filtering and transformations.

The following three categories represent the main approaches. Some products may be a slight variance on one or more of these categories.

Application Messaging Change Data Capture, Transactional Data Management Replication
Method Publish / Subscribe messages via a message queue (JMS, ActiveMQ, etc) Log sniffing Database replication (converted to messages)
Synch / Asynch Asynchronous Asynchronous Synchronous
Source system cost System Resources System Resources Transaction delay
Latency Near real time however varies w / topology Near real time however varies w / topology, checkpoint & log switch interval Real-Time
Transformation Application Server Database or TDM Database
Advantage Messages can be also pushed to BI applications. Can handle extremely large data volumes Closest to real-time
Limitations Provided only by ultra-new enterprise applications. Data Warehouse vendor may not support the particular the messaging facility used. May not be supported by RDBMS vendor and possible security issues Database specific. May not support all message formats 1
Examples SAP JMS Provider, iWay Adapter for Oracle E-Business Suite GoldenGate, Informatica, Oracle Change Data Capture, PowerExchange: Real-Time Option, DataMirror Sybase RepConnector, Oracle Synch CDC

1. IBM MQSeries, Java JMS, Microsoft MSMQ, Oracle AQ, Apache ActiveMQ, HTTP, TCP/IP, and SOAP

Gartner's View of Open Source BI

Thu, 15 Mar 2007 15:50:55 -0700

Not surprisingly Gartner is not enthusiastic about Open Source BI. At the Summit, Conference Chair Bill Hostmann is quoted as saying:

"[the term is becoming ] kind of like the word "organic" in the grocery business. It's starting to lose its meaning, with some "open-source" vendors demanding licensing fees. Open source is promising, but the business models and products haven't kept up with the commercial products."

The point is well taken and by looking at other software projects that have succeeded in OSS, you can see that OSBI (JasperAnalysis, OpenI, Pentaho, and SpagoBI and others) are still missing two key ingredients:

  1. Product superior in price/performance, price/functionality to that of commercial counterparts
  2. A community of outside developers that genuinely believes and is devoting time & resources to the betterment of the software

A further hindrance could be the fact that other famous OSS projects have been geared toward IT and particularly the developer whereas by and large BI starts out by making its pitch to the business user.

Undoubtedly you can argue against many of these statements by pointing to large organizations that have standardized on this or that OSBI product. But where those honest wins over the commercial counterparts or wins because there was a CIO that is philosophically OSS ?

IBM's Dynamic Warehousing Announcement

Wed, 14 Mar 2007 02:49:38 -0700

IBM Dynamic Warehousing announcement is interesting however its difficult to tell what exactly is net new. Further perplexing are the case studies that theyre siting of, Omnium (they have to tell you who Omnium is) and some unidentified law enforcement agencies.

Determined to get to the bottom of whats behind the announcement I found the following white-paper:

which contains the following statement:

Dynamic warehousing is not a product, tool or simple one-off solution. It is an approach that enables you to deliver more dynamic business insights by integrating,
transforming, harvesting and analyzing insights from structured and unstructured information.

Which begs the question, when did companies start putting out press releases about "an approach and if they must, why not make that clear in the announcement to save time?

Column-Oriented Databases Come of Age

Tue, 13 Mar 2007 00:31:41 -0700

Vertica's column-oriented database which claims a 3.5 order of magnitude improvement on price/performance compared to row-stores (traditional relational engines) is getting serious attention from high-profile VC sources and industry leaders.

Last month Vertica Systems, Inc. ( announced $16.5 million in Series B financing from New Enterprise Associates (NEA) and Kleiner Perkins Caufield & Byers on top of the $7 million it had already raised from Bessemer and Highland. Also as part of the announcement Vertica said that Kleiner Perkins general partner Ray Lane, formerly president and chief operating officer of Oracle will serve as a special adviser to the company.

In addition to the world-class investment team, the company has an experienced management team including Dr. Michael Stonebraker, CTO who was the main architect of the INGRES relational DBMS.

Paper that discusses a benchmark they ran against an appliance RDBMS

Also note an MIT Open Source project in the same area (may or may not be related)
C-Store: A Column Oriented DBMS