How North Korea is Like a Rogue Computer Process

Recognizing North Korea and Kim Jong-un’s recent actions as probable bluster has parallels to assessing a rogue computer process or questionable user activity on a network. When a process goes wrong in a system, log monitoring software gives off a real-time alert as a warning. With a less-than-enterprise class solution, this alert might be all that happens, which forces systems administrators to decide on an action based on isolated, incomplete information. With lives at stake rather than system and network resources, the result could be tragic.

Context is Key for Real Understanding

The U.S. government has the benefit of a sophisticated infrastructure providing correlated analysis of any situation from multiple angles. The direct threats from North Korea are correlated with data on their lack of actual troop movements, no missile facilities preparations, and in context of historical data of frequent threats right around national holidays. Similar to the U.S. government, a true enterprise-class log management and data analytics system should enable IT managers to have a fully informed view of any specific event with all the pertinent information available at once to enable fully intelligent action. [Read more...]

You’re Being Stingy With Your Data

With all the talk about how big data should be used, what for, and why, rarely do we hear about who uses “it.” All the recent buzz around big data is not because data has all of a sudden become more valuable, it’s that people are now realizing and discussing how to use new technologies and architectures to derive value from these large data sets.

All too often, organizations have looked at the log management problem from an application-centric point of view. Unfortunately, this approach typically results in an “accidental architecture” of redundant connections to log services, inefficient use of network resources, and valuable data “siloed” into distinct, unrelated, and difficult to traverse data stores. This causes log data to become less valuable than it could otherwise be.

Put Your Money Where Your Data Is

Real deep log data use has historically been prohibitively expensive. Due to the complexity involved and expensive solutions, getting this valuable data unlocked wasn’t a priority for this quarter’s earnings while it was understood to be a long-term advantage. Companies weren’t purposely withholding information; it was just a shortsighted solution to a problem they didn’t know they had. Log management is supposed to protect data from bad guys with an agenda, but it shouldn’t make data inaccessible from people at a company who can gain value. Businesses need an enterprise-class platform that anyone can easily see across the enterprise. [Read more...]

What Does Hiking Have to Do with Innovation?

Three friends and I went hiking last week in a dense forest. The objective was clear: to reach a small fortress about six miles from base camp. Without maps, or smartphones and no real clue how to get there, we were on our own. With  no rules, or parameters to validate our moves and literally no support or back-up, we had to make instant decisions based on events as they occurred. We were trailblazers, quite literally, and had to innovate in creating a trail for others to follow just to reach our goal.

Trailblazing is the process of leaving markings that follow each other at certain — though not necessarily exactly defined — distances, and marking the direction of the trail. The markings left by previous hikers help others follow the best trail.

This is essentially what today’s organizations have to face. They know the end objectives (mitigate risks and comply) and apply forensics to determine what went wrong so it can be amended the next time. If you’re lucky, you can isolate the event and put a mark up so others don’t follow that path in the future, but the ability to make decisions in real time or leverage trailblazing is what differentiate organizations from being average to becoming outstanding.

[Read more...]

Three Key Security Observations from the 2013 RSA Conference in San Francisco

At the RSA Conference in San Francisco, it’s all security, all of the time. When one topic is the only focus over such a short period, it becomes easy to see current trends. Here are three that have caught my attention:

1. Advanced persistent threats (APTs) – APTs will continue to be an issue for enterprises in 2013 and beyond. The machine layer of defense is excellent at catching threats that are known ahead of time, where rules can be written, filters created and bad things can be “bucketed” from good things. Sometimes a human eye is brought in to spot things computers don’t “see” so easily. In a perfect world, that is enough.

But the significant problem is the A in APT. Advanced threats haven’t been identified, and only by collecting all data available and using user and machine activity monitoring can these threats be identified and blocked.

2. Bring Your Own Device (BYOD) – BYOD brings problems as well when people using their own devices can break any policy at any time. Enforcing policies across disparate devices (some more secure, some more vulnerable) requires monitoring of systems and user activity. [Read more...]

Conflicting Data or the Data Divide?

This week there have been two reports released. One from Ofcom (independent regulator and competition authority for the UK communications industries), which has reported that the UK’s mobile users are consuming more data on their phones and tablets than any other leading nation for the first time. A second from the Office for National Statistics has reported that 7.63 million adults in the UK have never used the internet, which is 15% of the population. They have coined a moniker for these people – “The Internots.”

So are the reports wrong? Or is there something else happening?

Let’s explore the reports in a little more detail.

Ofcom’s report, which you can download here, shows that the UK has one of the highest levels of penetration of smartphones in the world at 58%of the population, while just fewer than one in five owns a tablet computer. As a result, British consumers are downloading the most data on mobiles and tablets. In December 2011, the average UK mobile connection used 424 megabytes of data, higher than any other leading country, pushing Japan into second place at 392 megabytes and the US into sixth at 319 megabytes.

One-sixth of all website traffic in the UK was on a mobile, tablet or other connected device, higher than any other country in Europe. James Thickett, Ofcom’s director of research, said: “Our research shows that UK consumers continue to benefit from one of the most advanced markets for communications products and services.” [Read more...]

TIBCO Innovates in Real Time for Global Carrier CargoSmart

 

“Never before in history has innovation offered promise of so much to so many in so short a time.” Bill Gates uttered these words and they could not be truer for TIBCO. The famously successful businessman is no stranger to innovation, and today there is great promise for new and exciting technological advancements. As a leading provider of infrastructure software, TIBCO knows it is important to innovate and help other companies innovate to become event-enabled enterprises. One such company is Hong Kong-based CargoSmart Limited, an ocean-carrier transportation portal that allows customers to master the challenging “end-to-end” shipping process that requires multi-carrier shipping to connect seller and buyer.

Gates’ words are not just inspiring, but an idea CargoSmart realizes is vital to the success in the transportation industry. Not only does innovation promise to bring “so much to so many” in this day and age, but CargoSmart has to ensure that their shipments are being received by “so many” of their customers in a timely and efficient manner. CargoSmart uses a TIBCO-based platform to create products and respond to events in real time. Also, CargoSmart built a collaborative platform for the company on tibbr – TIBCO’s social computing tool for the workplace. Because of TIBCO’s help, CargoSmart has the ability to cut transportation management costs, automate and streamline operations, and reduce the risk of late or mismanaged shipments. [Read more...]

Closing the Big Data Loop

It has been two weeks since TUCON 2012, the TIBCO user conference held annually in Las Vegas, Nevada. TIBCO is well-known as an integration company but used the event to demonstrate its broad platform approach to the biggest challenges of today, like digital customer experience, loyalty, and the topic of this article, Big Data.

The TIBCO CTO, Matt Quinn spoke about the patterns hiding within the rapidly increasing amounts of data flowing across the enterprise. Quinn made an important distinction between data at rest, information sitting in databases or flat files waiting to be queried, and data in motion, which includes streaming data and data stored in-memory (also known as cache).

Quinn made the point that it takes analytics like those offered within TIBCO’s Spotfire product to be able to see what would be invisible to people trying to keep up with the increasing deluge of information. To Quinn, the smart enterprise finds patterns in historical and machine data (log files that up until recently weren’t mined for patterns) that provide insight that can be applied to data that’s coming at today’s “full speed.” [Read more...]

Big Data in Real Time

Big Data was first characterized in 2001 as having three Vs: Volume, Velocity and Variety.  Volume refers to the sheer size of data that you need to work with, whether it’s Gigabytes, Terabytes or Petabytes.  Velocity is about the speed at which new data is generated, coming from more and faster streams of events.  Variety talks to the many different ways that data is represented, whether it has different structures, or has no structure at all.

To these three Vs, I like to add a fourth: Volatility.  When I talk about data Volatility, I’m talking less about the actual data, and more about what it represents.  Events occur every day in your enterprise that are digital representations of threats to, or opportunities for, your organization.  Perhaps the event represents the chance to help a customer in your store find – and buy – an item he’s looking for.  Or maybe the event is telling you that a cyber-thief is making off with your sensitive information.

In either case, the situation isn’t waiting around for you to respond to it; it’s on its own schedule.  If you aren’t ready to respond in the appropriate amount of time, then your customer – or his personal information he entrusted you with – has left your premises.

So what do you do about these four Vs?  It all boils down to just three words:  Understand.  Anticipate.  Act. [Read more...]

Gartner Makes the Case for Complex Event Processing to Keep Up with Real-time Big Data

TIBCO’s complex event processing and in-memory data grid solutions are the perfect way to solve the problem that Gartner’s Roy W. Schulte and Bill Gassman call, “the conventional save and process paradigm” that isn’t fast enough for today’s big data challenges.

The enterprise world is becoming more time-critical by the moment. The need to analyze real-time data for opportunities, risks and efficiencies before putting it into a database is moving companies to complex event processing combined with in-memory storage.

“We found Roy Schulte and Bill Gassman’s report on complex event processing’s role in Big Data to be very insightful,” said Ivan Casanova, Senior Director of Product Marketing, TIBCO. “We could not agree more that changing conditions, including greater volume, velocity and variety of data is making the conventional save-and-process paradigm obsolete for big data.” [Read more...]

DEBS2012 on Event Patterns

At DEBS last month, a few members of the EPTS Reference Architecture team tutored on the latest Functional Event Patterns list – with sample and pseudocode implementations – covering all aspects of event preparation, analysis, (complex event) detection, and reaction. From a TIBCO CEP perspective, this version mostly covers TIBCO BusinessEvents rule patterns (i.e. how these functional patterns map to a standard event-based production rule pseudocode), with a few references to BE State Models and the odd BE Continuous Query [*1]. These can be viewed alongside examples from Oracle EP, IBM WODM, IBM Stream Insight and the PROLOG-based Prova.

This remains very much a work-in-progress [*2], but should give a good idea of where we are heading. In the “real world,” it should be noted that often many of these functions are combined into a single operation (eg: covering preparation/filtering, analysis/transformation, detection/composition and reaction/assessment in a single rule or query). [Read more...]